Elections/2021 UK elections
The 2021 UK elections were not a general election, but were a series of elections held across the UK:
- Council, Constabulary and Mayoral elections in England
- Welsh Parliament elections in Wales
- Scottish Parliament elections in Scotland
Work based upon Democracy Club information
User:Betamax collected and processed data from Democracy Club, a UK non-profit organisation with crowdsourced information on candidates and political parties for the elections. In particular, the Candidate Database was particularly helpful, although several other spreadsheets were also provided by members of Democracy Club.
After processing this data, the end result is as follows:
- 3904 Twitter usernames of political parties / candidates
User:Betamax ran a manual scrape to grab the tweets (with media and outlinks). This produced a list (900MB in size) list containing 17.8 million URLs (13.1 million tweets plus outlinks and media links).
The list was split up into chunks of 1 million URLs, which were archived through ArchiveBot (currently still ongoing): 0 1 2 3 4 5 6
- 4175 Facebook pages
Rate limiting and banning by Facebook prevents any of these pages from being archived.
- 318 Instagram profiles
Rate limiting and banning by Facebook prevents any of these profiles from being archived.
- 89 Youtube channels (and a Dailymotion channel) used by parties / candidates
- 530 Political Party websites
This has been further processed down to 522 websites: reprocessed list.
- 1273 Candidate Websites
This has been further processed down to 1101 websites: reprocessed list.
Along with the Political Party websites (see above), these were processed as individual ArchiveBot jobs to avoid ArchiveBot's cross-linking / outlink handling from resulting in incomplete grabs.
- 2503 Candidate Web pages (these have been linked to separately and include manifestos, etc... - there will likely be a large overlap between these pages and the candidate websites, but I think it is best to archive these as well).
Completed as ArchiveBot Job ID 32g60eq89ir3jt7xm7htfrk81