Apache Spark
78 items from awesome-spark/awesome-spark ★1,902
-
Apache Beam beam.apache.org
A unified programming model that implements both batch and streaming data processing jobs that run on many execution engines.
-
Spark in Action manning.com
New book in the Manning's "in action" family with +400 pages. Starts gently, step-by-step and covers large number of topics. Free excerpt on how to setup Eclipse for Spark application development and how to bootstrap a new application using the provided Maven Archetype. You can…
-
Apache Zeppelin zeppelin.incubator.apache.org
Web-based notebook that enables interactive data analytics with plugable backends, integrated plotting, and extensive Spark support out-of-the-box.
-
Apache SystemML systemml.apache.org
Declarative machine learning framework on top of Spark.
-
Advanced Analytics with Spark shop.oreilly.com
Useful collection of Spark processing patterns. Accompanying GitHub repository: sryza/aas.
-
PredictionIO prediction.io
Machine Learning server for developers and data scientists to build and deploy predictive applications in a fraction of the time.
-
Polynote polynote.org
Polynote: an IDE-inspired polyglot notebook. It supports mixing multiple languages in one notebook, and sharing data between them seamlessly. It encourages reproducible notebooks with its immutable data model. Originating from Netflix.
-
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing people.csail.mit.edu
Paper introducing a core distributed memory abstraction.
-
Learning Spark, 2nd Edition oreilly.com
Introduction to Spark API with Spark 3.0 covered. Good source of knowledge about basic concepts.
-
MLflow mlflow.org
Machine learning orchestration platform.
-
ModelDB mitdbg.github.io
A system to manage machine learning models for spark.ml and scikit-learn .
-
Mahout Spark Bindings mahout.apache.org
[status unknown] - linear algebra DSL and optimizer with R-like syntax.
-
KeystoneML keystone-ml.org
Type safe machine learning pipelines with RDDs.
-
Mastering Apache Spark jaceklaskowski.gitbooks.io
Interesting compilation of notes by Jacek Laskowski. Focused on different aspects of Spark internals.
-
datamechanics/spark hub.docker.com
An easy to setup Docker image for Apache Spark from Data Mechanics.
-
apache/spark hub.docker.com
Apache Spark Official Docker images.
-
Spark with Scala Gitter channel gitter.im
"A place to discuss and ask questions about using Scala for Spark programming" started by @deanwampler.
-
jupyter/docker-stacks/pyspark-notebook github.com
PySpark with Jupyter Notebook and Mesos client.
-
Apache DataFu github.com
A library of general purpose functions and UDF's.
-
Data Science and Engineering with Apache Spark (edX XSeries) edx.org
Series of five courses (Introduction to Apache Spark, Distributed Machine Learning with Apache Spark, Big Data Analysis with Apache Spark, Advanced Apache Spark for Data Science and Data Engineering, Advanced Distributed Machine Learning with Apache Spark) covering different…
-
lakeFS docs.lakefs.io
Integration with the lakeFS atomic versioned storage layer.
-
Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark cs.stanford.edu
Structured Streaming is a new high-level streaming API, it is a declarative API based on automatically incrementalizing a static relational query.
-
Big Data Analysis with Scala and Spark (Coursera) coursera.org
Scala oriented introductory course. Part of Functional Programming in Scala Specialization.
-
Apache Bahir bahir.apache.org
Collection of the streaming connectors excluded from Spark 2.0 (Akka, MQTT, Twitter. ZeroMQ).
-
Large-Scale Intelligent Microservices arxiv.org
Microsoft paper that presents an Apache Spark-based micro-service orchestration framework that extends database operations to include web service primitives.
-
Apache Spark User List apache-spark-user-list.1001560.n3.nabble.com
and Apache Spark Developers List - Mailing lists dedicated to usage questions and development topics respectively.
-
Spark SQL: Relational Data Processing in Spark amplab.cs.berkeley.edu
Paper introducing relational underpinnings, code generation and Catalyst optimizer.
-
AMP Camp ampcamp.berkeley.edu
Periodical training event organized by the UC Berkeley AMPLab. A source of useful exercise and recorded workshops covering different tools from the Berkeley Data Analytics Stack.
-
almond almond.sh
A scala kernel for Jupyter.
