Big Data
1,992 items in 9 lists
-
Scriptella ETL github.com
Open-source, Java-based ETL and script execution tool for transferring and transforming data between databases, files, and other sources.
-
spark-connect-rs github.com
Rust bindings.
-
mpich2-yarn github.com
-
Oligrapher github.com
-
pg-kafka github.com
A PostgreSQL extension to produce messages to Apache Kafka.
-
MatrixNetworks.jl github.com
-
Velox github.com
System for serving machine learning predictions.
-
NetworkLayout.jl github.com
-
neo4r github.com
-
BDA github.com
The vulnerability detector for Hadoop and Spark
-
https://github.com/livingsocial/HiveSwarm github.com
-
IGraph/M github.com
-
-
OrionBelt Semantic Layer github.com
Open-source semantic sidecar that compiles YAML-defined dimensions, measures, and metrics into optimized SQL across 8 engines (BigQuery, ClickHouse, Databricks, Dremio, DuckDB, MySQL, PostgreSQL, Snowflake). Unified REST, MCP, and Postgres wire protocol; one model powers AI…
-
JPMML-Spark github.com
PMML transformer library for Spark ML.
-
Schedoscope github.com
Scala DSL for agile scheduling of Hadoop jobs.
-
https://github.com/forward3d/rbhive github.com
-
NetworkCube github.com
-
GraphStore github.com
-
HIHO github.com
framework for connecting disparate data sources with Hadoop.
-
pdfmux github.com
Python PDF-to-Markdown orchestrator. Classifies each page and routes to the optimal backend (PyMuPDF, Docling, RapidOCR, Gemini Flash), emitting Markdown plus a per-page confidence score so ingestion pipelines can quarantine low-trust pages before feeding LLMs or retrieval.
-
TikzGraphs github.com
-
Apache Slider github.com
is a YARN application to deploy existing distributed applications on YARN.
-
akela github.com
-
Beetest github.com
-
Flume MongoDB Sink github.com
-
goldfish github.com
-
MetaGraphs github.com
-
EcologicalNetwork.jl github.com
-
qgis-edge-bundling github.com
-
SnapVX github.com
-
Hive_test github.com
-
itachi github.com
A library that brings useful functions from modern database management systems to Apache Spark.
-
sparkly github.com
Helpers & syntactic sugar for PySpark.
-
https://github.com/forward/node-hive github.com
-
UCSB github.com
extended Yahoo Cloud Serving Benchmark for NoSQL databases.
-
Tuktu github.com
Easy-to-use platform for batch and streaming computation, built using Scala, Akka and Play!
-
daffy github.com
Decorator-first DataFrame contracts/validation (columns/dtypes/constraints) at function boundaries. Supports Pandas/Polars/PyArrow/Modin.
-
Flume RabbitMQ source and sink github.com
-
Datasalt Pangool github.com
alternative MapReduce paradigm.
-
FutureSearch SDK github.com
Python SDK that dispatches parallel web-research agents across table rows, synthesizing multi-agent findings into structured columns.
-
vahara github.com
-
Snowflake Emulator github.com
A Snowflake-compatible emulator for local development and testing.
-
EvolvingGraphs github.com
-
Graphy github.com
-
datatrax github.com
Data engineering and classic ML toolkit with batch processing, type coercion, and 7 algorithms in pure Go with zero dependencies.
-
Kafka-logger github.com
Kafka-winston logger for Node.js from Uber.
-
PACE github.com
An open source framework that allows you to enforce agreements on how data should be accessed, used, and transformed, regardless of the data platform (Snowflake, BigQuery, DataBricks, etc.)
-
ig.degree.betweenness github.com
-
QueryGPT github.com
Natural language database query interface with automatic chart generation, supporting Chinese and English queries.
- next page of items loading…
