Big Data
1,992 items in 9 lists
-
RAPIDS cuGraph github.com
-
.NET for Apache Spark github.com
.NET bindings.
-
Mara github.com
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
-
d3-force: Force-directed graph layout github.com
-
Elasticsearch Hadoop github.com
-
Spark Cassandra Connector github.com
Cassandra support including data source and API and support for arbitrary queries.
-
Gremlin github.com
graph traversal Language.
-
Baidu Tera github.com
an Internet-scale database, inspired by BigTable.
-
ActorDB github.com
a distributed SQL database with the scalability of a KV store, while keeping the query capabilities of a relational database.
-
Robinhood's Faust github.com
Forever scalable event processing & in-memory durable K/V store as a library with asyncio & static typing.
-
Smooks (⭐421) github.com
Extensible Java framework for processing and transforming structured data such as EDI, XML, CSV, and JSON.
-
Ambry github.com
a distributed object store that supports storage of trillion of small immutable objects as well as billions of large objects.
-
Genie github.com
-
SlothDB github.com
an embedded SQL database that runs everywhere: on your laptop, on a server, and in the browser. [MIT] website
-
Kuiper github.com
An edge lightweight IoT data analytics/streaming software implemented by Golang, and it can be run at all kinds of resource-constrained edge devices.
-
Elassandra github.com
is a fork of Elasticsearch modified to run on top of Apache Cassandra in a scalable and resilient peer-to-peer architecture.
-
Timely github.com
A time series database application that provides secure access to time series data based on Accumulo and Grafana.
-
LocustDB github.com
an experimental analytics database aiming to set a new standard for query performance on commodity hardware.
-
Nika github.com
Intent-as-code workflow engine for AI data pipelines: reviewable YAML DAGs statically checked (schema, permits, cost floor) before execution, with tamper-evident run traces.
-
spark-testing-base github.com
Collection of base test classes.
-
MLeap github.com
Execution engine and serialization format which supports deployment of o.a.s.ml models without dependency on SparkSession.
-
Hetionet: an integrative network of disease github.com
Heterogeneous network integrating genes, diseases, drugs, pathways, and more.
-
Optimus github.com
Data Cleansing and Exploration utilities with the goal of simplifying data cleaning.
-
Comdb2 github.com
a clustered RDBMS built on optimistic concurrency control techniques.
-
Project Nessie github.com
A Transactional Catalog for Data Lakes with Git-like semantics. Works with Apache Iceberg tables.
-
Conjecture github.com
scalable Machine Learning in Scalding.
-
AgensGraph github.com
transactional graph database based on PostgreSQL.
-
Duckle github.com
open-source visual ETL/ELT platform built on DuckDB with connectors, data quality checks, and lineage.
-
FiloDB github.com
Distributed. Columnar. Versioned. Streaming. SQL.
-
SummitDB github.com
an in-memory, NoSQL key/value database, with disk persistence and using the Raft consensus algorithm.
-
hdfs - A native go client for HDFS github.com
-
Hydrosphere Mist github.com
a service for exposing Apache Spark analytics jobs and machine learning models as realtime, batch or reactive web services.
-
neo4j-spark-connector github.com
Bolt protocol based, Neo4j Connector with RDD, DataFrame and GraphX / GraphFrames support.
-
sparkmagic github.com
Jupyter magics and kernels for working with remote Spark clusters, for interactively working with remote Spark clusters through Livy, in Jupyter notebooks.
-
Eventhub github.com
open source event analytics platform.
-
Twitter Fatcache github.com
key/value cache for flash storage.
-
S3QL github.com
A file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack.
-
Omnigraph github.com
Typed graph database where agents branch and merge like Git. S3-native, Rust, traversal + vector + BM25 in one runtime.
-
CCM github.com
A script to easily create and destroy an Apache Cassandra cluster on localhost.
-
AthenaX github.com
a streaming analytics platform that enables users to run production-quality, large scale streaming analytics using Structured Query Language (SQL).
-
FlexViz github.com
Python library for interactive, cross-filtered dashboards that stay responsive on 100M+ rows by aggregating with Polars on the server.
-
GraphFrames github.com
Data frame based graph API.
-
isolation-forest github.com
distributed Spark and Scala implementation of isolation forest for unsupervised outlier detection.
-
dna-claude-analysis github.com
Personal genome analysis toolkit with Python scripts analyzing raw DNA data across 17 categories (health risks, ancestry, pharmacogenomics, nutrition, psychology, etc.) and generating a terminal-style single-page HTML visualization.
-
Pinball github.com
DAG based workflow manager. Job flows are defined programmatically in Python. Support output passing between jobs.
-
Bistro github.com
general-purpose data processing engine for both batch and stream analytics. It is based on a novel data model, which represents data via functions and processes data via column operations as opposed to having only set operations in conventional approaches like MapReduce or SQL.
-
graspologic github.com
-
sparklyr github.com
An alternative R backend, using dplyr.
-
411 github.com
an web application for alert management resulting from scheduled searches into Elasticsearch.
-
Livy github.com
REST server with extensive language support (Python, R, Scala), ability to maintain interactive sessions and object sharing.
- next page of items loading…
