Big Data
1,992 items in 9 lists
-
ContentStructure github.com
-
CCAS github.com
-
GraphVisualize.jl github.com
-
-
https://github.com/lovelysystems/ls-hive github.com
-
https://github.com/hrp/sequel-hive-adapter github.com
-
-
GraphCentrality.jl github.com
-
Aegis DQ github.com
Open-source agentic data quality framework with LLM-powered diagnosis, root-cause analysis, SQL auto-fix proposals, and 31 rule types — DuckDB, Postgres, BigQuery, Databricks, Athena, Snowflake.
-
parag github.com
-
Networks.jl github.com
-
NetworkFlows.jl github.com
-
Estuary Benchmark Report github.com
reproducible, vendor-neutral data warehouse benchmark.
-
DataSpoc Pipe github.com
Data ingestion engine that connects 400+ Singer taps to Parquet files in cloud buckets (S3, GCS, Azure). Streaming, incremental, with auto-catalog.
-
Smile.jl github.com
-
A tutorial for network centrality in R github.com
-
gdMetriX github.com
-
spark-connect-csharp github.com
C# bindings.
-
CARQ github.com
Context-Aware RAG Processing Queue for high availability and adaptive rate-limiting.
-
Signals CLI github.com
Intent signal monitoring CLI. Track LinkedIn engagers, keyword posters, job changers, funding events. JSON output for data pipelines.
-
Mindweave Synthetic Business Data github.com
42-table synthetic SME dataset with double-entry accounting, tax compliance (AU/US/UK), and temporal realism. CSV, SQL, Parquet, SQLite. Ideal for ETL pipeline testing.
-
db2lake github.com
Lightweight Node.js ETL framework for databases → data lakes/warehouses.
-
Mixer github.com
-
Lambdo github.com
Lambdo is a workflow engine which significantly simplifies the analysis process by unifying feature engineering and machine learning operations.
-
StreamSets Data Collector github.com
continuous big data ingest infrastructure with a simple to use IDE.
-
Argus github.com
Time series monitoring and alerting platform.
-
Kestrel github.com
distributed message queue system.
-
RL4J github.com
Reinforcement learning for Java and Scala. Includes Deep-Q learning and A3C algorithms, and integrates with Open AI's Gym. Runs in the Deeplearning4j ecosystem.
-
ND4J github.com
A matrix library for the JVM. Numpy for Java.
-
Deeplearning4j Benchmarks github.com
-
DataVec github.com
A vectorization and data preprocessing library for deep learning in Java and Scala. Part of the Deeplearning4j ecosystem.
-
CartoDB github.com
open-source or freemium hosting for geospatial databases with powerful front-end editing capabilities and a robust API.
-
Miscellaneous Scripts for fixing issues with the Universal Forwarder github.com
This kit was compiled based on common issues with Splunk deployments and managing idiosyncrasies that tend to naturally occur.
-
SwimOS github.com
A framework for building real-time streaming data processing applications that supports a wide range of ingestion sources.
-
CronQ github.com
An application cron-like system. Used w/Luigi. Deprecated.
-
GreenPlum github.com
The Greenplum Database (GPDB) - An advanced, fully featured, open source data warehouse. It provides powerful and rapid analytics on petabyte scale data volumes.
-
Helium MCP github.com
Remote MCP server for real-time financial data, 3.2M+ news articles, ML options pricing, and news bias analysis. Free, no API key. MCP
-
BottledWater github.com
Change data capture from PostgreSQL into Kafka. Deprecated.
-
https://github.com/klout/brickhouse github.com
-
gohadoop github.com
-
enrich-companies github.com
CLI tool to enrich CSV files with company data (financials, contacts, metadata) from 250M+ company records. Available on npm.
-
