Big Data
1,992 items in 9 lists
-
drt github.com
OSS Reverse ETL CLI. Sync data from warehouses to business tools via YAML.
-
https://github.com/yc-huang/Hive-mongo github.com
-
weighted-modularity-LPAwbPLUS github.com
-
NetworkViz.jl github.com
-
streamsx.topology github.com
Libraries to enable building IBM Streams application in Java, Python or Scala.
-
Network Science CheatSheets github.com
-
CHSZLabLib github.com
-
ProxySQL github.com
High Performance Proxy for MySQL.
-
-
LinkedIn Kamikaze github.com
utility package for compressing sorted integer arrays.
-
LightGraphsExtras.jl github.com
-
Google Sheets ETL github.com
Live import all your Google Sheets to your data warehouse.
-
ankush github.com
-
AKF github.com
The AI native file format. Trust scores, source provenance, and compliance metadata that embed into 20+ formats (DOCX, PDF, images, code). EXIF for AI.
-
https://github.com/recruitcojp/WebHive github.com
-
-
Bloomery github.com
Web UI for Impala.
-
Hortonworks Schema Registry github.com
-
OneQuery github.com
Self-hosted gateway for safe, auditable queries for agents across approved data sources.
-
Provero github.com
A vendor-neutral, declarative data quality engine. Define checks in YAML, run anywhere. Includes 16 built-in check types, SQL batch optimizer, anomaly detection, and data contracts.
-
.Net FlumeNG Clients github.com
-
tnetwork github.com
-
algo.graph github.com
-
data-genie github.com
High-performance, streaming-first ETL engine for Node.js and TypeScript with constant memory footprint.
-
faucet-stream github.com
Config-driven data-movement platform for Rust with pluggable source and sink connectors, running ETL, CDC, and streaming pipelines declaratively from YAML or embedded as a library.
-
-
https://github.com/chimpler/hive-solr github.com
-
netUtils github.com
-
https://github.com/dvasilen/Hive-Cassandra github.com
-
Micro S3 persistence github.com
Docker microservice for saving/restoring volume data to S3.
-
-
SnackFS github.com
A bite-sized, lightweight HDFS compatible file system built over Cassandra.
-
-
npartite github.com
-
Scherlok github.com
Zero-config data quality CLI. Profiles every table on first run, then auto-detects anomalies (volume drops, schema drift, freshness misses, distribution shifts) on subsequent runs. No YAML, no rules to write. Works with Postgres, BigQuery, Snowflake, and dbt.
-
-
Rawbbit github.com
Open-source self-hosted game analytics pipeline. HTTP event collector with NATS JetStream buffering, raw Parquet in object storage you own, and ClickHouse for queries via Metabase, SQL, or a read-only MCP server for AI agents. Designed for teams that want to own their raw game…
-
Dotflow github.com
A lightweight Python library for building execution pipelines with retry, parallel execution, cron scheduling, and async support.
-
levelnet github.com
-
NetLytics github.com
Analytics platform to process network data on Spark.
-
koala-diff github.com
A high-performance Python library for comparing large datasets (CSV, Parquet) locally using Rust and Polars. It features zero-copy streaming to prevent OOM errors and generates interactive HTML data quality reports.
-
concoR github.com
-
Bistro github.com
A light-weight engine for general-purpose data processing including both batch and stream analytics. It is based on a novel unique data model, which represents data via functions and processes data via columns operations as opposed to having only set operations in conventional…
-
Flume UDP Source github.com
-
https://github.com/recruitcojp/OdbcHive github.com
-
Rhombus github.com
A time-series object store for Cassandra that handles all the complexity of building wide row indexes.
-
Zilla github.com
- An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT, and the native Kafka protocol.
-
networkdiffusion github.com
-
crdt-merge github.com
Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs.
-
PFC-JSONL github.com
Specialized JSONL log compressor with block-level timestamp indexing and DuckDB integration. Achieves ~9% compression ratio (better than gzip) with time-range random access queries.
- next page of items loading…
