Common Crawl Foundation discord.gg

Common Crawl files data.commoncrawl.org

WARCs, CDX files, parquet url index, parquet host index, etc.

The Web Crawl section of COPTR coptr.digipres.org

Conifer conifer.rhizome.org

From Rhizome, source available at https://github.com/Rhizome-Conifer. The hosted service was discontinued in June 2026; existing collections remain accessible.

Community Archive community-archive.org

Open Twitter Database and API with tools and resources for building on archived Twitter data.

Common Crawl Columnar Index commoncrawl.org

SQL-queryable index, with CDX info plus language classification. (Stable)

Common Crawl Web Graph commoncrawl.org

A host or domain-level graph of the web, with ranking information. (Stable)

Common Crawl Foundation Blog commoncrawl.org

rss

Chrome Revolver chromewebstore.google.com

Browser extension: switches between browser tabs.

Chrome Open Multiple URLs chromewebstore.google.com

Browser extension: opens multiple URLs and also extracts URLs from text.

Chrome link checker chromewebstore.google.com

Browser extension: basic link checker.

Chrome Check My Links chromewebstore.google.com

Browser extension: a link checker with more options.

Chrome link gopher chromewebstore.google.com

Browser extension: link harvester on a page.

Continuing Education to Advance Web Archiving (CEDWARC) cedwarc.github.io

Common Crawl Foundation Partners ccfpartners.slack.com

(ask greg zat commoncrawl zot org for an invite)

UK Web Archive Blog blogs.bl.uk

DSHR's Blog blog.dshr.org

David Rosenthal regularly reviews and summarizes work done in the Digital Preservation field.

Offical ISO 28500 WARC specification homepage bibnum.bnf.fr

Archives Unleashed Toolkit documentation aut.docs.archivesunleashed.org

Arkiwera arkiwera.se

The WARC Ecosystem archiveteam.org

Archives Unleashed Slack archivesunleashed.slack.com

Fill out this request form for access to a researcher group of people working with web archives.

Archivers Slack archivers.slack.com

Invite yourself to a multi-disciplinary effort for archiving projects run in affiliation with EDGI and Data Together.

Archive Ready archiveready.com

tool, for estimating how likely a web page will be archived successfully.

The Web Archiving Lifecycle Model archive-it.org

An attempt to incorporate the technological and programmatic arms of the web archiving into a framework that will be relevant to any organization seeking to archive content from the web. Archive-It, the web archiving service from the Internet Archive, developed the model based…

Archive-It archive-it.org

From the Internet Archive.

Unfair enumeration youtube.com

A program that outputs all even numbers and then all odd numbers generates an unfair enumeration of the natural numbers because some numbers are never reached.

CockroachDB web.archive.org

Database application, that is marketed as being so fault-tolerant and resilient as a cockroach.

Optimistic UI uxplanet.org

User interfaces that assume expensive operations will complete successfully, thereby improving the perceived performance.

tldr tldr.sh

Simplified man pages with practical examples.

caffeinate theapplegeek.co.uk

Terminal app to keep macOS from "sleeping".

Bento layout saasframe.io

A grid-based layout resembling the compartmentation of bento boxes.

Israeli Queue rapidapi.com

A type of priority queue and a reference to the infamously unorganized queues in Israel. Here items can cut in line when they already have waiting friends.

Carousel nngroup.com

A kind of animated slideshow looping back on itself.

ACID vs. BASE johndcook.com

Acronyms describing competing database ideologies (aka. SQL vs. NoSQL). Note that acid and base are also opposites in chemistry.

git bayesect hauntsaninja.github.io

Like git bisect but can cope with flaky tests using Bayesian inference.

munch hackage.haskell.org

Parser function that greedily consumes an input stream until it's satisfied.

zip hackage.haskell.org

Merges two lists into one list of pairs like the interlocking teeth of a zipper.

Floating point number floating-point-gui.de

This representation can encode numbers at very different magnitudes with a limited amount of digits by letting the radix point float instead of being fixed in place.

Yoda condition eslint.org

When you write if ("red" === color) { instead of if (color === "red") { because it reads as, “if red equals the color”, similar to the way the Star Wars character Yoda speaks.

Trojan horse en.wikipedia.org

Malware which misleads users of its true intent. The term is derived from the Ancient Greek story of the deceptive Trojan Horse.

Tree shaking en.wikipedia.org

Shake the dependency tree until all the dead parts are falling off and you end up with a nice lean tree.

Tree en.wikipedia.org

A hierarchically organized data structure. From the root item, the other items branch out into nodes and leaves. A collection of trees is often called a forest.

Training en.wikipedia.org

The process of showing the machine a bunch of examples, until it learns what we want from it.

Time travel debugging en.wikipedia.org

Stepping back in time through source code to understand execution and sometimes even to change history.

Technical debt en.wikipedia.org

The implied future cost of shortcuts taken today, which accumulates like financial debt and must eventually be paid back — often with interest.

Syntactic sugar en.wikipedia.org

Syntax that makes the language "sweeter" for human use. Usually a shorthand for common operations that can also be expressed in a more verbose form.

Process starvation en.wikipedia.org

A problem where a process is perpetually denied resources to do its work.

Canary en.wikipedia.org

A secret value placed on the stack to detect buffer overflow attacks. If it has been altered, danger is near — just like a canary in a coal mine that warns miners of toxic gas.

Stack en.wikipedia.org

Like with a stack of pancakes, you can only add and remove items from the top of this data structure.