OutbackCDX

github.comtool

RocksDB-based capture index (CDX) server supporting incremental updates and compression. Can be used as backend for OpenWayback, PyWb and Heritrix. (Stable)

from

Web Archiving

added

2026-10-10

likes

0

similar

1. ▲
   duckdb-web-archive-cdx github.com

   tool category #web-archiving link ↗

   DuckDB extension to query the Internet Archive and CommonCrawl CDX APIs directly from SQL. (In Development)

2. ▲
   cdx-toolkit pypi.org

   site category #web-archiving link ↗

   Library and CLI to consult cdx indexes and create WARC extractions of subsets. Abstracts away Common Crawl's unusual crawl structure. (Stable)

3. ▲
   Waybackpy github.com

   tool category #web-archiving link ↗

   Wayback Machine Save, CDX and availability API interface in Python and a command-line tool (Stable)

   
4. ▲
   OpenWayback github.com

   tool category #web-archiving link ↗

   The open source project aimed to develop Wayback Machine, the key software used by web archives worldwide to play back archived websites in the user's browser. (Stable)

5. ▲
   gowarcserver github.com

   tool category #web-archiving link ↗

   BadgerDB-based capture index (CDX) and WARC record server, used to index and serve WARC files (Go).

   
6. ▲
   warcrefs github.com

   tool category #web-archiving link ↗

   Web archive deduplication tools. (Stable)

   

Web Archiving › Tools & Software > Utilities: “RocksDB-based capture index (CDX) server supporting incremental updates and compression. Can be used as backend for OpenWayback, PyWb and Heritrix. (Stable)”