Big Data
1,992 items in 9 lists
-
Facebook McDipper facebook.com
key/value cache for flash storage.
-
Facebook Haystack facebook.com
object storage system.
-
Evrything evrythng.com
Making products smart
-
EventQL eventql.io
a distributed, column-oriented database built for large-scale event collection and analytics.
-
etcML etcml.com
text classification with machine learning.
-
LinkedIn Galene engineering.linkedin.com
search architecture at LinkedIn.
-
LinkedIn Espresso engineering.linkedin.com
horizontally scalable document-oriented NoSQL data store.
-
Facebook TAO engineering.fb.com
TAO is the distributed data store that is widely used at Facebook to store and serve the social graph.
-
Logstash elastic.co
a tool for managing events and logs.
-
Metricbeat elastic.co
lightweight shipper for system and service metrics.
-
Apache Edgent (Incubating) edgent.apache.org
a programming model and micro-kernel style runtime that can be embedded in gateways and small footprint edge devices enabling local, real-time, analytics on the edge devices.
-
Eclipse BIRT eclipse.org
Eclipse-based reporting system.
-
Apache Eagle eagle.apache.org
real time monitoring solution
-
Drizzle drizzle.org
evolution of MySQL 6.0.
-
Dremio dremio.com
an open-source, SQL-like Data-as-a-Service Platform based on Apache Arrow.
-
Apache Doris doris.apache.org
real-time analytical database for high-concurrency SQL analytics, search, and warehousing.
-
Invantive SQL documentation.invantive.com
SQL engine for online and on-premise use with integrated local data replication and 70+ connectors.
-
Soda Core docs.soda.io
open-source Python library and CLI for data quality tests.
-
Azure Data Factory docs.microsoft.com
cloud-based pipeline orchestration for on-prem, cloud and HDInsight
-
DataKitchen Open Source Data Observability docs.datakitchen.io
open-source data observability for monitoring data journeys, data quality, and pipeline events.
-
Nokia Disco discoproject.org
MapReduce framework developed by Nokia.
-
Disco DDFS disco.readthedocs.org
distributed filesystem.
-
Delta Lake delta.io
open-source storage framework for building lakehouse architectures on data lakes.
-
Dekart dekart.xyz
Large scale geospatial analytics for Google BigQuery based on Kepler.gl.
-
Debezium debezium.io
open-source distributed platform for change data capture.
-
Apache SystemML systemml.apache.org
Declarative machine learning framework on top of Spark.
-
2013 db.disi.unitn.eu
Facebook - Unicorn: A System for Searching the Social Graph.
-
Advanced Analytics with Spark shop.oreilly.com
Useful collection of Spark processing patterns. Accompanying GitHub repository: sryza/aas.
-
2013 db.disi.unitn.eu
Facebook - Scuba: Diving into Data at Facebook.
-
PredictionIO prediction.io
Machine Learning server for developers and data scientists to build and deploy predictive applications in a fraction of the time.
-
2013 db.disi.unitn.eu
Google - Online, Asynchronous Schema Change in F1.
-
2013 db.disi.unitn.eu
Google - MillWheel: Fault-Tolerant Stream Processing at Internet Scale.
-
Polynote polynote.org
Polynote: an IDE-inspired polyglot notebook. It supports mixing multiple languages in one notebook, and sharing data between them seamlessly. It encourages reproducible notebooks with its immutable data model. Originating from Netflix.
-
HadoopDB db.cs.yale.edu
hybrid of MapReduce and DBMS.
-
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing people.csail.mit.edu
Paper introducing a core distributed memory abstraction.
-
GraphLab Create dato.com
A machine learning platform in Python with a broad collection of ML toolkits, data engineering, and deployment tools.
-
Learning Spark, 2nd Edition oreilly.com
Introduction to Spark API with Spark 3.0 covered. Good source of knowledge about basic concepts.
-
DataTorrent StrAM datatorrent.com
real-time engine is designed to enable distributed, asynchronous, real time in-memory big-data computations in as unblocked a way as possible, with minimal overhead and impact on performance.
-
MLflow mlflow.org
Machine learning orchestration platform.
-
Datasalt Splout SQL datasalt.com
full SQL query engine for big datasets.
-
ModelDB mitdbg.github.io
A system to manage machine learning models for spark.ml and scikit-learn .
-
datapine datapine.com
self-service business intelligence tool in the cloud.
-
Mahout Spark Bindings mahout.apache.org
[status unknown] - linear algebra DSL and optimizer with R-like syntax.
-
Datapallas datapallas.com
BI and data platform with AI exploration, dashboards, and pixel-perfect report generation; formerly ReportBurster.
-
KeystoneML keystone-ml.org
Type safe machine learning pipelines with RDDs.
-
Monitoring Kafka performance datadoghq.com
Guide to monitoring Apache Kafka, including native methods for metrics collection.
-
Mastering Apache Spark jaceklaskowski.gitbooks.io
Interesting compilation of notes by Jacek Laskowski. Focused on different aspects of Spark internals.
-
Monitoring Cassandra performance datadoghq.com
Guide to monitoring Cassandra, including native methods for metrics collection.
-
datamechanics/spark hub.docker.com
An easy to setup Docker image for Apache Spark from Data Mechanics.
-
AMPLab SIMR databricks.github.io
run Spark on Hadoop MapReduce v1.
- next page of items loading…
