The Web as History uclpress.co.uk

An open-source book that provides a conceptual overview to web archiving research, as well as several case studies.

#WebArchiving twitter.com

#WebArchiveWednesday twitter.com

@commoncrawl twitter.com

Official Common Crawl Foundation handle.

@WebSciDL twitter.com

ODU Web Science and Digital Libraries Research Group.

@NetPreserve twitter.com

Official IIPC handle.

Tempas v2 tempas.l3s.de

Temporal web archive search based on links and anchor texts extracted from the German web from 1996 to 2013 (results are not limited to German pages, e.g., Obama@2005-2009 in Tempas). (Stable)

Tempas v1 tempas.l3s.de

Temporal web archive search based on Delicious tags. (Stable)

Windows Snipping Tool support.microsoft.com

Windows built-in for partial screen capture and annotation. On macOS you can use Command + Shift + 4 (keyboard shortcut for taking partial screen capture).

Glossary of Archive-It and Web Archiving Terms support.archive-it.org

StormCrawler stormcrawler.net

A collection of resources for building low-latency, scalable web crawlers on Apache Storm. (Stable)

Tutorial for Humanities researchers about how to explore Arquivo.pt sobre.arquivo.pt

Smarsh smarsh.com

cdx-toolkit pypi.org

Library and CLI to consult cdx indexes and create WARC extractions of subsets. Abstracts away Common Crawl's unusual crawl structure. (Stable)

PlayOnMac playonmac.com

For running Xenu and Notepad++ on macOS.

PlayOnLinux playonlinux.com

For running Xenu and Notepad++ on Ubuntu.

PageFreezer pagefreezer.com

Reconstructive oduwsdl.github.io

A Service Worker module for client-side reconstruction of composite mementos by rerouting resource requests to corresponding archived copies (JavaScript).

Definition of Web Archivability nullhandle.org

This describes the ease with which web content can be preserved. (Archived version from the Stanford Libraries)

IIPC Blog netpreserveblog.wordpress.com

IIPC and DPC Training materials: module for beginners (8 sessions) netpreserve.org

IIPC netpreserve.org

Warchaeology nationallibraryofnorway.github.io

A collection of tools for inspecting, manipulating, deduplicating and validating WARC-files. (Stable)

UK Government Web Archive nationalarchives.gov.uk

Main page for the UKGWA.

MirrorWeb mirrorweb.com

SiteStory mementoweb.github.io

A transactional archive that selectively captures and stores transactions that take place between a web client (browser) and a web server. (Stable)

WARCreate matkelly.com

A Google Chrome extension for archiving an individual webpage or website to a WARC file. (Stable)

Retrieving and Archiving Information from Websites by Wael Eskandar and Brad Murray kit.exposingtheinvisible.org

Common Crawl CDX API index.commoncrawl.org

Search Common Crawl's CDX URL index.

IIPC Slack iipc.slack.com

Ask @netpreserve for access.

The warc-specifications iipc.github.io

A community HTML version of the official specification and hub for new proposals.

httpreserve.info httpreserve.info

Service to return the status of a web page or save it to the Internet Archive. HTTPreserve includes disambiguation of well-known short link services. It returns JSON via the browser or command line via CURL using GET. Describes web sites using earliest and latest dates in the…

Xenu home.snafu.de

Desktop link checker for Windows.

Hanzo hanzo.co

WASAPI groups.google.com

OpenWayback groups.google.com

Common Crawl groups.google.com

Webrecorder US GovArchive govarchive.us

High-fidelity replay.

Wget gnu.org

An open source file retrieval utility that of version 1.14 supports writing warcs. (Stable)

GLAM Workbench: Web Archives glam-workbench.github.io

See also this related blog post on 'Asking questions with web archives'.

warcdedupe gitlab.com

WARC deduplication tool (and WARC library) written in Rust. (In Development)

WebMemex github.com

Browser extension for Firefox and Chrome which lets you archive web pages you visit. (In Development)

here github.com

.

Heritrix github.com

An open source, extensible, web-scale, archival quality web crawler. (Stable)

Heritrix Q&A github.com

A discussion forum for asking questions and getting answers about using Heritrix.

Comparison of web archiving software github.com

Web Archiving Community github.com

Crawl git.autistici.org

A simple web crawler in Golang. (Stable)

End of Term Archive eotarchive.org

WARCs, CDX files, parquet url index.

Wikipedia's List of Web Archiving Initiatives en.wikipedia.org