Databricks Secures Apache Spark, Launches Community Edition
Also at Spark Summit, Databricks announced the general availability of Databricks Community Edition (DCE), a free version of the company's data platform. The announcement comes just four months after the beta launch of Databricks Community Edition at Spark Summit East in New York. "This year we've seen explosive growth for the Apache Spark project and all signs indicate the pace will only accelerate as the community expands even more," Matei Zaharia, co-founder and CTO at Databricks, said in a statement. "Databricks Community Edition has created an ideal environment for learning Apache Spark. Developers of all backgrounds can now use Databricks Community Edition to learn Spark and mitigate the acute Spark skills gap." In a blog post on the GA release, Ion Stoica, executive chairman of Databricks, said more than 8,000 users have signed up for DCE, many of them using the service heavily. The top 10 most active users are averaging more than six hours per week with the platform and are executing more than 10,000 commands on average. Moreover, DCE is attracting a wide user base, Stoica said. According to a recent survey, 25 percent of DCE users have never used Spark before, and 60 percent of the users are neither data scientists nor data engineers. "This demonstrates the effectiveness of DCE to grow the open-source Apache Spark user community by bringing new users into the fold, as well as its ability to train new data scientists and engineers," he said."Today's enterprises have an insatiable demand for data skills, which is exacerbated by the scarcity of qualified talent," Stoica said in a statement. "Education is in Databricks' DNA, and our birthplace at the UC Berkeley AMPLab gives the company significant experience in educating students and users. Databricks Community Edition augments that effort as a learning platform. More than 2,200 students have already taken courses using Databricks Community Edition since its beta release, and with its general availability, we expect widespread adoption by universities across the world." Databricks Community Edition users will have access to a 6GB micro-cluster as well as a cluster manager and the notebook environment to prototype simple applications. As a learning tool, DCE comes with a portfolio of Apache Spark learning resources, including a set of Massive Open Online Courses (MOOC) and sample notebooks.
The same survey also indicates that 90 percent of the users employ DCE for learning Apache Spark, "which establishes the role of DCE as a learning platform for Spark. Indeed, since its launch, tens of universities have already used DCE for teaching, including UC Berkeley and Stanford," Stoica said. Also, the GA comes with new introductory materials and sample applications to make learning Apache Spark even easier.