Big Data
1,992 items in 9 lists
-
.Net FlumeNG Clients github.com
-
tnetwork github.com
-
algo.graph github.com
-
data-genie github.com
High-performance, streaming-first ETL engine for Node.js and TypeScript with constant memory footprint.
-
faucet-stream github.com
Config-driven data-movement platform for Rust with pluggable source and sink connectors, running ETL, CDC, and streaming pipelines declaratively from YAML or embedded as a library.
-
-
https://github.com/chimpler/hive-solr github.com
-
netUtils github.com
-
https://github.com/dvasilen/Hive-Cassandra github.com
-
Micro S3 persistence github.com
Docker microservice for saving/restoring volume data to S3.
-
-
SnackFS github.com
A bite-sized, lightweight HDFS compatible file system built over Cassandra.
-
-
npartite github.com
-
Scherlok github.com
Zero-config data quality CLI. Profiles every table on first run, then auto-detects anomalies (volume drops, schema drift, freshness misses, distribution shifts) on subsequent runs. No YAML, no rules to write. Works with Postgres, BigQuery, Snowflake, and dbt.
-
-
Rawbbit github.com
Open-source self-hosted game analytics pipeline. HTTP event collector with NATS JetStream buffering, raw Parquet in object storage you own, and ClickHouse for queries via Metabase, SQL, or a read-only MCP server for AI agents. Designed for teams that want to own their raw game…
-
Dotflow github.com
A lightweight Python library for building execution pipelines with retry, parallel execution, cron scheduling, and async support.
-
levelnet github.com
-
NetLytics github.com
Analytics platform to process network data on Spark.
-
koala-diff github.com
A high-performance Python library for comparing large datasets (CSV, Parquet) locally using Rust and Polars. It features zero-copy streaming to prevent OOM errors and generates interactive HTML data quality reports.
-
concoR github.com
-
Bistro github.com
A light-weight engine for general-purpose data processing including both batch and stream analytics. It is based on a novel unique data model, which represents data via functions and processes data via columns operations as opposed to having only set operations in conventional…
-
Flume UDP Source github.com
-
https://github.com/recruitcojp/OdbcHive github.com
-
2012 vldb.org
Twitter - The Unified Logging Infrastructure for Data Analytics at Twitter.
-
Twitter Finagle twitter.github.io
asynchronous network stack for the JVM.
-
Trino trino.io
distributed SQL query engine for querying large datasets across heterogeneous data sources.
-
Terraform terraform.io
infrastructure as code tool for provisioning and managing cloud and on-premises infrastructure.
-
LMDB symas.com
ultra-fast, ultra-compact key-value embedded data store developed by Symas.
-
Google BigTable static.googleusercontent.com
column-oriented distributed datastore.
-
Open Graph protocol ogp.me
The Open Graph protocol enables any web page to become a rich object in a social graph.
-
Cytoscape.js js.cytoscape.org
-
Palladio hdlab.stanford.edu
Visualize complex historical data with ease.
-
Splice Machine splicemachine.com
a full-featured SQL-on-Hadoop RDBMS with ACID transactions.
-
Serf serf.io
decentralized solution for service discovery and orchestration.
-
2007 read.seas.harvard.edu
Amazon - Dynamo: Amazon’s Highly Available Key-value Store.
-
Query.me query.me
collaborative SQL notebooks for querying, scheduling, and sharing reporting workflows.
-
OpenMPI open-mpi.org
message passing framework.
-
Lily HBase Indexer ngdata.github.io
quickly and easily search for any content stored in HBase.
-
Metricsgraphic.js metricsgraphicsjs.org
a library built on top of D3 that is optimized for time-series data
-
Apache Mesos mesos.apache.org
cluster manager.
-
Kylin kylin.apache.org
open source Distributed Analytics Engine from eBay.
-
Apache JMeter jmeter.apache.org
load testing tool for measuring performance of services and distributed systems.
-
JanusGraph janusgraph.org
open-source, distributed graph database with multiple options for storage backends (Bigtable, HBase, Cassandra, etc.) and indexing backends (Elasticsearch, Solr, Lucene).
-
IPython ipython.org
provides a rich architecture for interactive computing.
-
Indicative indicative.com
Web & mobile analytics tool, with data warehouse (AWS, BigQuery) integration.
-
Apache Helix helix.apache.org
cluster management framework.
-
HASH hash.ai
open source simulation and visualization platform.
-
Graylog graylog.org
log management platform for collecting, storing, searching, and alerting on machine data.
- next page of items loading…
