Big Data
645 items from 0xnr/awesome-bigdata ★14,672
-
ORYX github.com
Lambda architecture platform built on Apache Spark and Apache Kafka with specialization for real-time large scale machine learning.
-
Feast github.com
A feature store for the management, discovery, and access of machine learning features. Feast provides a consistent view of feature data for both model training and model serving.
-
Aim github.com
open-source AI metadata tracker for experiments and training runs.
-
dgraph github.com
Scalable, Distributed, Low Latency, High Throughput Graph Database.
-
cubism github.com
JavaScript library for time series visualization.
-
Blazer github.com
business intelligence made simple.
-
Infinity github.com
The AI-native database built for LLM applications, providing incredibly fast vector and full-text search. [Apache2]
-
peity github.com
Progressive SVG bar, line and pie charts.
-
Apache Pulsar (⭐15k) github.com
Versatile pub/sub and streaming platform offering scalable, low-latency messaging for modern applications.
-
cayley github.com
An open-source graph database. Google.
-
Bolt github.com
an embedded key-value database for Go.
-
Heka github.com
Data Acquisition and Processing Made Easy. Deprecated.
-
pytorch_geometric_temporal github.com
Temporal Extension Library for PyTorch Geometric.
-
vega github.com
a visualization grammar.
-
Keras github.com
An intuitive neural net API inspired by Torch that runs atop Theano and Tensorflow.
-
arbor github.com
graph visualization library using web workers and jQuery.
-
yugabyteDB github.com
open source, high-performance, distributed SQL database compatible with PostgreSQL.
-
Tile38 github.com
Geolocation DB with spatial index and realtime geofencing.
-
ElastAert github.com
ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in ElasticSearch.
-
H2O github.com
statistical, machine learning and math runtime with Hadoop. R and Python.
-
snowplow/snowplow :fire::fire::fire::fire::fire: github.com
enterprise-strength web and event analytics, powered by Hadoop, Kinesis, Redshift and Postgres.
-
Freeboard github.com
pen source real-time dashboard builder for IOT and other web mashups.
-
Materialize github.com
Streaming SQL database powered by Timely Dataflow :heavy_dollar_sign:
-
NuPIC github.com
Numenta Platform for Intelligent Computing: a brain-inspired machine intelligence platform, and biologically accurate neural network based on cortical learning algorithms.
-
Lightdash github.com
The open source Looker alternative built on dbt
-
TDengine github.com
open-source time-series database with high-performance ingestion, SQL support, and IoT-oriented storage.
-
buntdb github.com
Fast, embeddable, in-memory key/value database for Go with custom indexing and spatial support.
-
RudderStack github.com
A warehouse-first Customer Data Platform that enables you to collect data from every application, website and SaaS platform, and then activate it in your warehouse and business tools.
-
Riak github.com
a decentralized datastore.
-
ingestr github.com
CLI tool to copy data between databases with a single command. Supports 50+ sources including PostgreSQL, MySQL, MongoDB, Salesforce, Shopify to any data warehouse.
-
TiKV github.com
a distributed key-value database powered by Rust and inspired by Google Spanner and HBase.
-
Tarantool github.com
An in-memory database and application server.
-
Scalding github.com
Scala library for Map Reduce jobs, built on Cascading.
-
Thanos github.com
Thanos is a set of components to create a highly available metric system with unlimited storage capacity using multiple (existing) Prometheus deployments.
-
Druid github.com
Column oriented distributed data store ideal for powering interactive applications
-
Little Ball of Fur github.com
A library for sampling graph structured data.
-
chdb github.com
in-process OLAP SQL engine powered by ClickHouse, callable from Python with native pandas/Arrow DataFrame interop.
-
Twemproxy github.com
A fast, light-weight proxy for memcached and redis.
-
Numaflow (⭐2.8k) github.com
Kubernetes-native, serverless platform for massively parallel stream and data processing, with exactly-once semantics and language-agnostic user code.
-
PigPen github.com
map-reduce for Clojure which compiles to Apache Pig.
-
Gobblin from LinkedIn github.com
linkedin's universal data ingestion framework.
-
Summing Bird github.com
Streaming MapReduce with Scalding and Storm, by Twitter.
-
Secor github.com
Pinterest's Kafka to S3 distributed consumer.
-
Gaffer github.com
A large-scale graph database.
-
Bruin github.com
End-to-end data pipeline tool that combines ingestion, transformation (SQL + Python), and data quality in a single CLI. Connects to BigQuery, Snowflake, PostgreSQL, Redshift, and more. Includes VS Code extension with live previews.
-
kairosdb github.com
Fast scalable time series database.
-
Substation github.com
A cloud native data pipeline and transformation toolkit written in Go.
-
envisionjs github.com
dynamic HTML5 visualization.
-
HiBench github.com
a Hadoop benchmark suite.
-
HyperDex github.com
A scalable, searchable key-value store. Deprecated.
- next page of items loading…
