Big Data
1,992 items in 9 lists
-
GitHub Archive gharchive.org
GitHub's public timeline since 2011, updated every hour.
-
Estuary Flow estuary.dev
No/low-code data pipeline platform that handles both batch and real-time data ingestion.
-
Embulk embulk.org
An open source bulk data loader that helps data transfer between various databases, storages, file formats, and cloud services.
-
YData Profiling docs.profiling.ydata.ai
A general-purpose open-source data profiler for high-level analysis of a dataset.
-
DataRaven dataraven.io
Managed cloud object storage transfers for ingestion workflows.
-
Akka Toolkit akka.io
runtime for distributed, and fault tolerant event-driven applications on the JVM.
-
Aerospike aerospike.com
NoSQL flash-optimized, in-memory. Open source and "Server code in 'C' (not Java or Erlang) precisely tuned to avoid context switching and memory copies."
-
Riak-TS basho.com
Riak TS is the only enterprise-grade NoSQL time series database optimized specifically for IoT and Time Series data.
-
AWS EMR aws.amazon.com
A web service that makes it easy to quickly and cost-effectively process vast amounts of data.
-
ArcadeDB arcadedb.com
Open-source multi-model database with native graph, document, key-value, and vector support. SQL, Cypher, and Gremlin query languages. Apache 2.0 license.
-
Alluxio alluxio.org
A memory-centric distributed storage system enabling reliable data sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.
-
Ably ably.com
Pub/sub messaging platform for IoT
-
Apache Zeppelin zeppelin.incubator.apache.org
Web-based notebook that enables interactive data analytics with plugable backends, integrated plotting, and extensive Spark support out-of-the-box.
-
Apache Trafodion trafodion.apache.org
Transactional SQL-on-Hadoop/HBase.
-
Apache Tajo tajo.apache.org
distributed data warehouse system on Hadoop.
-
Apache Spark spark.apache.org
Makes it easy to build scalable fault-tolerant streaming applications.
-
Spark Packages spark-packages.org
A community index of packages for Apache Spark.
-
HBase: The Definitive Guide shop.oreilly.com
Comprehensive guide to HBase.
-
Spring for Apache Hadoop projects.spring.io
Spring-Hadoop integration, including HBase support.
-
Presto prestodb.io
distributed SQL query engine.
-
Apache Pig pig.apache.org
high level language to express data analysis programs for Hadoop.
-
Apache Phoenix phoenix.apache.org
SQL layer on top of HBase.
-
Apache Nutch nutch.apache.org
open source web crawler.
-
Apache Livy (incubating) livy.incubator.apache.org
The REST Spark Server.
-
Apache Kudu kudu.apache.org
Hadoop's storage layer to enable fast analytics on fast data.
-
Apache Knox Gateway knox.apache.org
single point of secure access for Hadoop clusters.
-
Kite Software Development Kit kitesdk.org
High-level data layer for Hadoop/HBase.
-
Apache Ignite ignite.apache.org
High-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies.
-
Apache Hadoop hadoop.apache.org
framework for distributed processing. Integrates MapReduce (parallel processing), YARN (job scheduling) and HDFS (distributed file system).
-
Apache Flume flume.apache.org
service to manage large amount of log data.
-
Apache Falcon falcon.apache.org
data management framework.
-
How to monitor Hadoop metrics datadoghq.com
Guide to monitoring Hadoop, with an overview of Hadoop architecture, and native methods for metrics collection.
-
Apache Curator curator.apache.org
Java libraries for Apache ZooKeeper.
-
Apache Crunch crunch.apache.org
a simple Java API for tasks like joining and data aggregation that are tedious to implement on plain MapReduce.
-
Lingual cascading.org
SQL-like query language for Cascading.
-
Apache Bigtop bigtop.apache.org
system deployment framework for the Hadoop ecosystem.
-
Big Data Benchmark amplab.cs.berkeley.edu
Benchmark of Redshift, Hive, Shark, Impala and Stiger/Tez.
-
Apache Accumulo accumulo.apache.org
distributed key/value store, built on Hadoop.
-
Apache SINGA - A General Distributed Deep Learning Platform singa.incubator.apache.org
-
Jupyter Notebook jupyter.org
Notebook and project application for interactive data science and scientific computing across all programming languages.
-
DexPaprika API api.dexpaprika.com
DEX pool, token, OHLCV, and trade data via API and MCP. Keyless access has rate and usage limits; see plans.
-
Spark Machine Learning spark.apache.org
Spark’s ML library consisting of common learning algorithms and utilities.
-
Cube square.github.io
uses MongoDB to store time series data.
-
The Analytics Engineering Podcast roundup.getdbt.com
How analytics engineers build and maintain data pipelines at scale.
-
NetworkX networkx.org
-
Data Analysis with Python and PySpark manning.com
Early Access
-
The Quiet-Broke Index jeevesagency.github.io
A 30-metro composite ranking of how much of a $400K household income gets consumed by housing, taxes, childcare, healthcare, and transport. Open methodology, free, no email gate.
-
enigma.com enigma.com
Navigate the world of public data - Quickly search and analyze billions of public records published by governments, companies and organizations.
-
Data Engineering Podcast dataengineeringpodcast.com
The show about modern data infrastructure.
-
DataDriven - SQL, Python, and Data Modeling Interview Questions datadriven.io
Interview practice with SQL query execution, Python, and data modeling exercises.
- next page of items loading…
