Programming
Software development: programming languages, coding, libraries and frameworks, software engineering and architecture, source control, developer tools, databases, and infrastructure.
-
Spark MLlib spark.apache.org
Spark's scalable machine learning library consisting of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as underlying optimization primitives.
-
Software Engineering Daily softwareengineeringdaily.com
Daily interviews about technical software topics, including data infrastructure.
-
SmoothieCharts smoothiecharts.org
A JavaScript Charting Library for Streaming Data.
-
Sling slingdata.io
CLI data integration tool specialized in moving data between databases, as well as storage systems.
-
SchemaCrawler schemacrawler.com
Open-source and free relational database schema discovery and comprehension tool. Documents and diagrams relational database schemas from your Java programs, build tools and the command line. Find database design issues with lint, and write scripts against the database…
-
RunSQL runsql.com
Free online SQL playground for MySQL, PostgreSQL, and SQL Server. Create database structures, run queries, and share results instantly.
-
Rivestack rivestack.io
Managed PostgreSQL with pgvector for AI workloads. HNSW indexing, sub-4ms latency, and built-in SQL editor with automatic embedding generation.
-
/r/dataengineering reddit.com
News, tips, and background on Data Engineering.
-
Presto prestodb.github.io
A distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources.
-
Prefect prefect.io
An orchestration and observability platform. With it, developers can rapidly build and scale resilient code, and triage disruptions effortlessly.
-
Percona XtraBackup percona.com
A free, open source, complete online backup solution for all versions of Percona Server, MySQL® and MariaDB®.
-
Percona Server for MongoDB percona.com
Percona Server for MongoDB® is a free, enhanced, fully compatible, open source, drop-in replacement for the MongoDB® Community Edition that includes enterprise-grade features and functionality.
-
ParquetKit parquetkit.com
Browser-based viewer, SQL workbench and converter for Parquet files powered by DuckDB-WASM. Fully client-side, no upload.
-
Nakadi nakadi.io
An open source event messaging platform that provides a REST API on top of Kafka-like queues.
-
LunaPad lunapad.dev
Open-source analytics notebook for reusable SQL workflows, interactive reports, and AI-assisted data exploration.
-
LizardFS lizardfs.com
Software Defined Storage is a distributed, parallel, scalable, fault-tolerant, Geo-Redundant and highly available file system.
-
ImageLayers imagelayers.io
Visualize Docker images and the layers that compose them.
-
Ilum ilum.cloud
A modular Data Lakehouse platform that simplifies the management and monitoring of Apache Spark clusters across Kubernetes and Hadoop environments.
-
Spark RDD API Examples homepage.cs.latrobe.edu.au
Examples by Zhen He.
-
Hadoop MapReduce hadoop.apache.org
A software framework for easily writing applications which process vast amounts of data (multi-terabyte data-sets) - in-parallel on large clusters (thousands of nodes) - of commodity hardware in a reliable, fault-tolerant manner.
-
Apache Geode geode.apache.org
An open source, distributed, in-memory database for scale-out applications.
-
Fixzi fixzi.ai
JSON/XML validation and API contract monitoring tool for debugging and testing structured data.
-
Enrich.sh enrich.sh
Managed event ingestion service that converts JSON sent to a REST API into Hive-partitioned Parquet on Cloudflare R2, queryable from DuckDB, ClickHouse, BigQuery, Snowflake, and Python.
-
Wikipedia dumps.wikimedia.org
Wikipedia's complete copy of all wikis, in the form of Wikitext source and metadata embedded in XML. A number of raw database tables in SQL form are also available.
-
Riak docs.basho.com
A distributed database designed to deliver maximum data availability by distributing data across multiple servers.
-
dlt dlthub.com
A fast&simple pipeline building library for Python data devs, runs in notebooks, cloud functions, airflow, etc.
-
Twitter Realtime developer.twitter.com
The Streaming APIs give developers low latency access to Twitter's global stream of Tweet data.
-
DataScreenIQ datascreeniq.com
Real-time data quality firewall for pipelines and APIs. Screens rows in milliseconds for schema drift, null spikes, type mismatches, and data anomalies with PASS / WARN / BLOCK decisions.
-
Data Mechanics datamechanics.co
A cloud-based platform deployed on Kubernetes making Apache Spark more developer-friendly and cost-effective.
-
DataKitchen datakitchen.io
Open Source Data Observability for end-to-end Data Journey Observability, data profiling, anomaly detection, and auto-created data quality validation tests.
-
Dataform dataform.co
An open-source framework and web based IDE to manage datasets and their dependencies. SQLX extends your existing SQL warehouse dialect to add features that support dependency management, testing, documentation and more.
-
Crustdata crustdata.com
A real-time B2B data API for company and people intelligence, providing firmographics, headcount signals, job listings, web traffic, and funding events via REST API and webhooks for data enrichment pipelines.
-
Couchbase couchbase.com
The highest performing NoSQL distributed database.
-
Common Crawl commoncrawl.org
Open source repository of web crawl data.
-
ClickHouse clickhouse.tech
Distributed columnar DBMS for OLAP. SQL.
-
chDB chdb.io
Embedded ClickHouse — full ClickHouse SQL dialect, ~80 data formats, and 12+ source connectors (S3, Postgres, MongoDB, Kafka, Iceberg) in core. Python, Go, Rust, Node, Bun, Zig, and Ruby bindings.
-
Bonobo bonobo-project.org
A data-processing toolkit for python 3.5+.
-
Actian Ingres actian.com
commercially supported, open-source SQL relational database management system.
-
Actian Versant actian.com
commercial object-oriented database management systems .
-
Actian SQL for Hadoop actian.com
high performance interactive SQL access to all Hadoop data.
-
Actian Vector actian.com
column-oriented analytic database.
-
Arpe.io arpe.io
High-speed CLI tools for database export, import, replication and migration with parallel streaming to CSV, Parquet, JSON and cloud storage, supporting PostgreSQL, MySQL, Oracle, SQL Server and 80+ sources.
-
LinkedIn Jobs Scraper apify.com
Crawlee-based actor extracting structured LinkedIn job listings at scale without API keys.
-
Sorsa API api.sorsa.io
Real-time X (Twitter) data API providing tweets, profiles, search, communities and engagement metrics. Up to 50x cheaper than the official X API with 20 req/sec rate limit, JSON output.
-
AI for Database aifordatabase.com
Agentic AI platform to connect any database (PostgreSQL, MySQL, MongoDB, etc.) and query in plain English; includes self-refreshing intelligent dashboards and action workflows triggered by data changes.
-
Docker based Hadoop provisioning slideshare.net
-
Programming Hive shop.oreilly.com
-
Programming Pig shop.oreilly.com
-
Parametarized Algorithms by Warsaw youtube.com
-
CMU Theory youtube.com
Aims for a mathematical understanding of fundamental issues in Computer Science, and to use this understanding to produce better algorithms, protocols, and systems, as well as identify the inherent limitations of efficient computation.
- next page of items loading…
