Big Data
1,992 items in 9 lists
-
JPMML-Spark github.com
PMML transformer library for Spark ML.
-
Schedoscope github.com
Scala DSL for agile scheduling of Hadoop jobs.
-
https://github.com/forward3d/rbhive github.com
-
NetworkCube github.com
-
GraphStore github.com
-
HIHO github.com
framework for connecting disparate data sources with Hadoop.
-
pdfmux github.com
Python PDF-to-Markdown orchestrator. Classifies each page and routes to the optimal backend (PyMuPDF, Docling, RapidOCR, Gemini Flash), emitting Markdown plus a per-page confidence score so ingestion pipelines can quarantine low-trust pages before feeding LLMs or retrieval.
-
TikzGraphs github.com
-
Apache Slider github.com
is a YARN application to deploy existing distributed applications on YARN.
-
akela github.com
-
Beetest github.com
-
Flume MongoDB Sink github.com
-
goldfish github.com
-
MetaGraphs github.com
-
EcologicalNetwork.jl github.com
-
qgis-edge-bundling github.com
-
SnapVX github.com
-
Hive_test github.com
-
itachi github.com
A library that brings useful functions from modern database management systems to Apache Spark.
-
sparkly github.com
Helpers & syntactic sugar for PySpark.
-
https://github.com/forward/node-hive github.com
-
UCSB github.com
extended Yahoo Cloud Serving Benchmark for NoSQL databases.
-
Tuktu github.com
Easy-to-use platform for batch and streaming computation, built using Scala, Akka and Play!
-
daffy github.com
Decorator-first DataFrame contracts/validation (columns/dtypes/constraints) at function boundaries. Supports Pandas/Polars/PyArrow/Modin.
-
Flume RabbitMQ source and sink github.com
-
Datasalt Pangool github.com
alternative MapReduce paradigm.
-
FutureSearch SDK github.com
Python SDK that dispatches parallel web-research agents across table rows, synthesizing multi-agent findings into structured columns.
-
vahara github.com
-
Snowflake Emulator github.com
A Snowflake-compatible emulator for local development and testing.
-
Grafana grafana.com
graphite dashboard frontend, editor and graph composer.
-
Metamarkets Druid druid.io
framework for real-time analysis of large datasets.
-
PostgreSQL postgresql.org
The world's most advanced open source database.
-
MySQL mysql.com
The world's most popular open source database.
-
Census getcensus.com
A reverse-ETL tool that let you sync data from your cloud data warehouse to SaaS applications like Salesforce, Marketo, HubSpot, Zendesk, etc. No engineering favors required—just SQL.
-
FluentD fluentd.org
An open source data collector for unified logging layer.
-
Apache Beam beam.apache.org
A unified programming model that implements both batch and streaming data processing jobs that run on many execution engines.
-
Apache Tez tez.apache.org
An application framework which allows for a complex directed-acyclic-graph of tasks for processing data.
-
Apache Sqoop sqoop.apache.org
A tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases.
-
Apache Samza samza.apache.org
A distributed stream processing framework.
-
OpenTSDB opentsdb.net
distributed time series database on top of HBase.
-
Apache Oozie oozie.apache.org
A workflow scheduler system to manage Apache Hadoop jobs.
-
Apache NiFi nifi.apache.org
An easy to use, powerful, and reliable system to process and distribute data.
-
Apache Mahout mahout.apache.org
An environment for quickly creating scalable performant machine learning applications.
-
Apache Solr lucene.apache.org
Search platform for Apache Lucene.
-
Apache Hive hive.apache.org
Data warehouse software facilitates querying and managing large datasets residing in distributed storage.
-
Apache HBase hbase.apache.org
The Hadoop database, a distributed, scalable, big data store.
-
Hue gethue.com
web application for interacting with Hadoop.
-
Apache Drill drill.apache.org
Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage.
-
Cascading cascading.org
Java based application development platform.
-
Apache Calcite calcite.apache.org
framework that allows efficient translation of queries involving heterogeneous and federated data.
- next page of items loading…
