Heritrix

From Archiveteam
Revision as of 14:59, 15 June 2026 by Klea (talk | contribs) (Link to sample WARC extracts.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Heritrix is a WARC-writing web crawler created by the Internet Archive. It is written in Java and can be found on the IA's GitHub page.

Sample WARC segments in text format are available for inspection at https://gist.github.com/ibnesayeed/ddb17bb2958fcff37f3dafc9a3a86793, though note it was made in 2018-03-16.