dogear

enter for all results · esc to close

grab-site

github.comtool

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns. (Stable)

visit github.com ↗
0
sign in to dogear this
from
Web Archiving
added
2026-10-10
likes
0

similar

  1. webarchive-discovery github.com
    tool category #web-archiving link ↗

    WARC and ARC full-text indexing and discovery tools, with a number of associated tools capable of using the index shown below. (Stable)

  2. Web Curator Tool webcuratortool.org
    site category #web-archiving link ↗

    Open-source workflow management for selective web archiving. (Stable)

  3. Web2Warc github.com
    tool category #web-archiving link ↗

    An easy-to-use and highly customizable crawler that enables anyone to create their own little Web archives (WARC/CDX). (Stable)

  4. gowarcserver github.com
    tool category #web-archiving link ↗

    BadgerDB-based capture index (CDX) and WARC record server, used to index and serve WARC files (Go).

  5. Warchaeology nationallibraryofnorway.github.io
    site category #web-archiving link ↗

    A collection of tools for inspecting, manipulating, deduplicating and validating WARC-files. (Stable)

  6. Web Archiving Community github.com
    site category #web-archiving link ↗

Web Archiving › Tools & Software > Acquisition: “The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns. (Stable)”