SmolNet

The SmolNet consists of content available through alternative protocols outside the web such as gemini:// gopher:// finger:// spartan:// text:// nex://.

At this time the WARC format does not support these protocols and the WBM does not support them, so the SmolNet is not archivable nor can archives be accessed.

Fortunately their are proxies to HTTP and HTML that can be used. The most prominent one is https://portal.mozz.us/ and it doesn't require JavaScript.

ArchiveBot job 84rmt67gwgaah8r70zqtjzw84 is crawling the entire SmolNet (and outlinks to HTTP) via the portal.mozz.us proxy. Unfortunately some sites do not allow there content to be downloaded via proxies, and some sites are of course down, so a minority of sites just give errors. Some sites contain git commits, those are ignored in favour of Codearchiver/SWH. Since SmolNet folks are often data hoarders/packrats, there may be large archives of resources already saved via HTTP, those should be ignored when they are encountered.

Should the WARC format ever add support for the SmolNet protocols, then the AB job could be useful for seeding a native recrawl of all known SmolNet sites, including those that block proxies to HTTP.