Template:CTA URL lists

Options:

• regex, required, the PCRE regular expression to use for filtering, will get wrapped in single quotes for the grep command
  • Technically, this isn't actually required, but only for use on URLs.
• broad, optional, adding an extra bit about the regex being intentionally broad if non-empty

Example:

{{CTA URL lists|regex = <nowiki>\S*(foo|bar)\S*</nowiki>|broad = yes}}

renders as:

How to help if you have lists of URLs

This project requires lists of URLs for content on the target website. If you have a source of URLs, please:

1. Use the PCRE regular expression \S*(foo|bar)\S* for filtering.
   • Note that this regex is intentionally broad to cover many different URL formats. Please do not try to use a more narrow pattern as it may miss valid URLs. We can always filter or transform the results as needed later.
   • Enable case-insensitive matching (e.g. -i option on grep) to not miss URLs with capitalised domains or similar.
   • If you use grep, remember to include the -a (aka --text on GNU grep) option to ensure it will continue searching for matches when encountering binary data.
   • Example command (GNU grep): grep -Pahoi '\S*(foo|bar)\S*' FILENAME FILENAME...
2. If the output exceeds a few megabytes, please compress it, preferably using zstd -10.
3. Upload the file to https://transfer.archivete.am/.
4. Share the resulting URL in the project IRC channel.
   • If you would like to keep the list non-public instead, e.g. for privacy reasons or for not wanting to be publicly associated with it, please get in touch with a channel op (e.g. arkiver or JustAnotherArchivist). Note that the items generated from your list would still be processed publicly, of course, but they would be mixed with everything else.

See also Category:Projects requiring URL lists for other ArchiveTeam projects that necessitate URL lists.