Mailing Lists
Mailing lists | |
Status | Unknown |
Archiving status | Not saved yet |
Archiving type | Unknown |
Project source | mailing-lists-grab |
IRC channel | #flaming-lists (on hackint) |
Mailing lists are a type of discussion group that is a precursor to forums (like Discourse) and is declining in popularity as email declines in use, being replaced by forums, and then social media. Existing mailing lists have a lot of history that would be useful to save.
Requirements
- Some sites only support TLS 1.0, or maybe even SSL 3 or earlier
- Some software (notably Sympa) or sites need POST plus a cookie to get otherwise public archives
Sources
There are some things that could be useful to find mailing list services/software:
- Wikidata
- instance of (P31) electronic mailing list (Q304863)
- might be a few more with other links to that item via refs etc
- Search engine enumeration
- https://en.wikipedia.org/wiki/List_of_mailing_list_software
- https://en.wikipedia.org/wiki/Category:Mailing_list_software
- https://alternativeto.net/tag/mailing-list/
- https://builtwith.com/mailman
Services
There are a number of notable services that include mailing lists, some have public archives:
- Usenet
- Gmane (now NNTP-only)
- The Mail Archive
- Mailing list ARChives (MARC)
- raw or partially raw mails public
- some lists archived elsewhere too, but sole archive source for some lists (eg some KDE lists)
- thr3ads.net
- archives lists on Google Groups, Mailman 2 instances and other services
- Nabble
- supports custom domains, examples: http://forum.world.st/
- Narkive
- main site is Usenet
- lists on subdomains, for eg https://debian-knoppix.linuxtag.narkive.com/
- FreeLists
- Google Groups
- raw messages are not available on most lists, anti-eg [1] [2] [3]
- some lists have Message-Id based URLs in the mail's footer, eg bitcoindev
- groups.io
- Has support for custom domains, for e.g. https://lists.lfenergy.org/g/main and https://lists.hubmapconsortium.org/
- Simplelists
- Has support for custom domains
- https://lists.security.metacpan.org/
- https://srfi-email.schemers.org/ - protected account - index of lists here instead https://srfi.schemers.org/srfi-list-subscribe.html
- Has support for custom domains
- SourceForge
- Derived from Mailman
- example https://lists.sourceforge.net/lists/listinfo/valgrind-announce
- Framasoft
- LWN
- notmuch
- PostgreSQL
- LibreOffice
- ActiveState
- SecLists
Message-Id lookup
There are some services that offer Message-Id lookup. It is important that we archive both the Message-Id based URLs and the URLs they redirect to. We should also extract the Message-Ids from every message to develop a mapping between Message-Ids and URLs, and later create a service for searching that.
http://mid.gmane.org/alpine.BSF.2.00.1004291655110.61392@x.fncre.vasb nntp://news.gmane.io/65491cda.c80a0220.5fa79.7688SMTPIN_ADDED_BROKEN@mx.google.com http://news.gmane.org/find-root.php?message_id=alpine.BSF.2.00.1004291655110.61392@x.fncre.vasb https://marc.info/?i=Prayer.1.0.18.0811191202370.4107@sabinus.cs.umn.edu http://mid.mail-archive.com/9dc4201d0807272359r1f0ad9d6ta44f31f439de58b7@mail.gmail.com https://www.mail-archive.com/search?l=mid&q=9dc4201d0807272359r1f0ad9d6ta44f31f439de58b7@mail.gmail.com https://www.freebsd.org/cgi/mid.cgi?db=mid&id=199802242058.MAA24843@monk.via.net https://docs.FreeBSD.org/cgi/mid.cgi?db=mid&id=199802242058.MAA24843@monk.via.net https://mail-archive.freebsd.org/cgi/mid.cgi?db=mid&id=199802242058.MAA24843@monk.via.net https://lists.debian.org/msgid-search/20050606213954.GC884@finlandia.infodrom.north.de https://lists.debian.org/msgid-search/?m=YvogMm/B0cM/AUMu@einval.com https://lists.debconf.org/cgi-lurker/keyword.cgi?doc-url=/lurker&format=en.html&query=id:87vey4ogcj.fsf@vorlon.ganneff.de https://www.w3.org/mid/alpine.BSF.2.00.1004291655110.61392@x.fncre.vasb https://www.postgresql.org/message-id/20060419051850.96025.qmail@web31802.mail.mud.yahoo.com https://public-inbox.org/git/Pine.LNX.4.58.0504150753440.7211@ppc970.osdl.org https://article.olduse.net/40478@ucbvax.BERKELEY.EDU http://al.howardknight.net/msgid.cgi?STYPE=msgid&A=0&MSGI=<usenet-hier-faq-1516003264%246620%40hope.eyrie.org> https://lwn.net/ml/linux-kernel/20181020134908.GA32218@kroah.com/ https://lwn.net/ml/all/20181020134908.GA32218@kroah.com/ https://lore.kernel.org/all/20181020134908.GA32218@kroah.com/ https://yhbt.net/lore/all/20181020134908.GA32218@kroah.com/ https://public-inbox.org/git/1486427537.16949.42.camel@bonedaddy.net/ https://www.43-1.org/app/nntp/mid/20050606213954.GC884@finlandia.infodrom.north.de https://forum.dlang.org/post/d19o8m$1jlj$1@digitaldaemon.com https://www.nntp.perl.org/group/perl.beginners/;msgid=01041316021102.03716%40localhost.localdomain https://nmbug.notmuchmail.org/nmweb/show/156832191038.4937.14365134928759130092%40fireG0 https://www.ctan.org/ctan-ann/id/mailman.3878.1440779487.4386.ctan-ann@dante.de lists.sr.ht graphql API https://archives.spi-inc.org/message-id/14d2a7e9-3269-d881-7f15-506bbf78299f%40spi-inc.org https://groups.google.com/d/msgid/bitcoindev/aDWfDI03I-Rakopb%40petertodd.org
Software
There are lots of software for mailing lists that have instances with public archives:
- Mailman 2 (pipermail archives)
- Requires a very special strategy for Mailman 2 instances
- pabs is monitoring ArchiveBot for Mailman 2 URL patterns
- Message-Id in three places:
<HEAD> <LINK REL="made" HREF="mailto:...?...In-Reply-To=...">
<BODY> <A HREF="mailto:...?...In-Reply-To=...">
(may get corrupted by Cloudflare/etc email address protection- in the compressed mbox archives
- Mailman 3 (usually HyperKitty archives)
- Message-Id in mailto:...?...In-Reply-To links (may be broken by email protection), might be an API somewhere to avoid that
- https://lists.darmstadt.ccc.de/hyperkitty/ https://lists.fedoraproject.org/ https://lists.hacknology.de/
- Sympa
- May need POST plus a cookie to get otherwise public archives
- http://lists.ee.ethz.ch/wws/ http://lists.sameswifi.fr/wws/ https://framagroupes.org/abc/en/ https://framalistes.org/sympa/ https://groupes.renater.fr/sympa/ https://liste.normandie-univ.fr/sympa/ https://listes.april.org/wws/ https://listes.aquilenet.fr/ https://listes.asso.univ-rennes.fr/wws/ https://listes.centrale-marseille.fr/ https://listes.creachlabs.fr/wws https://listes.doctorat-bretagne.fr/wws/ https://listes.ens-lyon.fr/sympa/ https://listes.ens-paris-saclay.fr/wws/ https://listes.ensc-rennes.fr/wws https://listes.entrouvert.com/ https://listes.etudiant.univ-rennes.fr/wws/ https://listes.existrans.org/ https://listes.hadoly.fr/ https://listes.iiens.net/sympa/ https://listes.infini.fr/ https://listes.renater.fr/wws/ https://listes.sciencespo-rennes.fr/wws/ https://listes.services.cnrs.fr/wws/ https://listes.u-bretagneloire.fr/wws/ https://listes.univ-rennes.fr/wws/ https://listes.univ-reunion.fr/wws/ https://lists.actux.eu.org/sympa/ https://lists.cam.ac.uk/sympa/ https://lists.digi-newb.eu/wws/ https://lists.ens.psl.eu/wws/ https://lists.fdn.fr/ https://lists.fit.edu/sympa/ https://lists.grnet.gr/wws/ https://lists.laglab.org/wws/ https://lists.linuxfromscratch.org/sympa/ https://lists.nnedv.org/lists/ https://lists.piratenpartei.de/ https://lists.puscii.nl/ https://lists.riseup.net/ https://lists.squat.net/wws/ https://lists.uniroma2.it/ https://lists.vis.ethz.ch/sympa/ https://lists.vmp.ethz.ch/sympa/ https://lists.vvn-bda.de/ https://lists36.net/wws/ https://mail.linuxtag.org/wws/ https://mailing.sogo.nu/sympa/ https://mlistes.resel.fr/wws/ https://newsletter.ende-gelaende.org/ https://newsletters.univ-rennes.fr/wws/ https://post.bund.net/ https://psilists.ethz.ch/sympa/ https://sympa.inria.fr/sympa/ https://sympa.laas.fr/wws/ https://sympa.lix.polytechnique.fr/ https://sympa.obspm.fr/wws/ https://sympa.uvsq.fr/sympa/ https://www.iut-tlse3.fr/wws/ https://www.legi.grenoble-inp.fr/servlist/wws/ https://www.listes.upmc.fr/wws/ https://www.listserv.dfn.de/sympa/ https://lists.hefn.org/
- Apache Pony Mail
- https://lists.apache.org/ (huge)
- public-inbox
- https://inbox.sourceware.org/ (huge) https://inbox.vuxu.org/ https://list.orgmode.org/ https://lore.barebox.org/ https://lore.kernel.org/ (huge) https://lore.pengutronix.de/ https://lore.ptxdist.org/ https://yhbt.net/kgio-public/ https://yhbt.net/raindrops-public/ https://yhbt.net/unicorn-public/ https://yhetil.org/
- LISTSERV
- https://listserv.ihs.gov/ (saving in AB)
- lyris
- mlmmj
- seems to use external archivers
- lists.lighttpd.net
- lurker
- mhonarc
- Message-Id in encoded HTML comments
<!--X-Message-Id: 19190830-af45-4646-8344-4f788db48a0f@disroot.org -->
- https://identicalsoftware.com/maillist/xwpe/ https://archives.seul.org/ lists.gnu.org (saved by AB)
- lists.debian.org (not saved yet)
- lists.launchpad.net (shutting down, contacted Canonical)
- Message-Id in encoded HTML comments
- hypermail
- Message-Id in a HTML comment, the @ might be escaped to _at_ or something else
<!-- id="9C19DACF-698F-4AF8-8873-A2DB2C6CB875_at_yahoo.com" -->
<!-- id="199911012305.PAA05714@hawk.prod.itd.earthlink.net" -->
- http://lalists.stanford.edu/ http://lists.boost.org/boost-interest https://list.skarnet.org/lists/ https://lists.boost.org/Archives/boost/ https://lists.boost.org/boost-announce/ https://lists.boost.org/boost-bugs/ https://lists.boost.org/boost-build/ https://lists.boost.org/boost-cmake/ https://lists.boost.org/boost-commit https://lists.boost.org/boost-docs/ https://lists.boost.org/boost-gil/ https://lists.boost.org/boost-maint/ https://lists.suckless.org/ https://lists.w3.org/Archives/Public/ https://lkml.iu.edu/hypermail/linux/kernel/ https://potter-yachters.org/archive/ https://www.circlemud.org/maillist/ https://www.daml.org/listarchive/ https://www.informatik.uni-kiel.de/~curry/listarchive/ https://www.lehigh.edu/~wsm0/vse-l/vse-l_archives.shtml#hypermail https://www.w1.fi/hostap/
- Message-Id in a HTML comment, the @ might be escaped to _at_ or something else
- Majordomo2
- ezmlm/ezmlm-idx/ezmlm-web (associated with qmail/djbdns)
- SourceHut
- CREN ListProc (more info)
- Unknown
Tools
These are some pages of and individual tools that are useful for archiving mailing lists:
Projects
These are some former archiving projects by ArchiveTeam that involved mailing lists:
Pages
There are a number of wiki pages that mention mailing lists in some way.
- OWASP
- Gopher
- Spanish Revolution
- Hackerspaces
- WikiApiary
- OpenSolaris
- Internet infrastructure
- Chaos Computer Club
- AdelaideUniversityMerger
Articles
There are some articles about hosting mailing lists that may be useful.