Alive... OR ARE THEY
Like many sites before them, these places indicate a sunny outlook, a clean bill of health and a total sense of "all systems go". But as we've found out from those many sites before them, fortunes can change overnight.
Archive Team considers these sites specifically of interest because they solicit so much content, contain so many works and projects by a wide group of people, or have the internet particularly dependent on them. Consider this a fire drill.. know what you can do to get your data off these sites and back them off for later.
Owned by Yahoo! Imminent Demise!
- Yahoo and AOL properties are being sold by Verizon to hedge fund Apollo in 2021. The news comes immediately after the closure of Yahoo! Answers and will presumably be swiftly followed by radical disruption for the sake of short-term cash milking.
Flickr contains billions of files, hundreds millions of which are under a Creative Commons license or stored there by many museums and other cultural institutions. The site was tumblr-ised in 2013 and has been poorly functional ever since; pro users were removed, so it doesn't yet have a business model. Additionally, it's owned by Yahoo!, need to say more?!
- Flickr was sold to SmugMug in 2018, purged the biggest non-paying and non-freely licensed accounts and switched to a more predictable subscription-based model.
- Academic Earth ( • • • ) has been worryingly unloved for a while, and holds a mountain of free education that's invaluable to the world.
- A-Infos ( • • • ) a multi-lingual news service by, for, and about anarchists. Have stuff archived since the 90's that isn't available anywhere else.
- Encyclopedia Astronautica ( • • • ) is the most comprehensive collection of the history of space travel. Period. Seriously, the official NASA history folks will refer you this website if they can't answer your questions. However, Mark Wade (the sole creator/maintainer) abandoned his blog at the end of 2007, and the Encyclopedia has not been updated since May 2008, despite much happening in the space exploration world since then. A backup was made of the site as of 28/01/2017.
- Angelfire has been in constant decline for many years now.
- AnimeMusicVideos.org ( • • • ) is fine right now, but they rely on donations and host vast amounts of user-edited music videos on their server (presumably without mirrors). Hard to download as you have to be a member to get all the download links, and after downloading a handful you have to vode before you can d/l again (or you can donate which presumably gives you 1 year of free d/l access). Also, this site might be a grey area, copyright-wise, as the videos are all cut together from copyrighted material.
- Archive of Our Own ( • • • ) is stable but contains a large catalog of fanfiction that would likely be lost if the site were to shut down.
- The Believer Magazine ( • • • ) was just purchased by what appears to be a sex toy company with an interest in keeping the archives up but they do not appear to be reliable or trustworthy custodians.
- BetaArchive ( • • • ) has Kafkaesque requirements to be able to access it, and apparently refuses to be backed up, presumably so that they get more visitors. Valuable cultural library of historic software with no backups? Aargh.
- BioMedia Project ( • • • ) is a large archive of various BIONICLE media that has not had a notable update since 2015.
- Carrd ( • • • , • • • , • • • , and • • • ) is a free web host, mainly used by minorities (the LGBTQ+ community and people with mental illnesses, for example) and Twitter/Tumblr communities. Also used for activism (one of the most popular Carrd websites is about Black Lives Matter, for example). Pretty stable as of now, but could become a good historical resource in the future.
- Codecademy ( • • • ) has a large amount of valuable coding lessons.
- DatasheetArchive ( • • • ) hosts over 350 million PDF datasheets for integrated circuits, some of which are very old and hard to track down otherwise. The site is slow from time to time and uses a convoluted IFRAME-based online viewer, presumably to make scraping the site harder. Nevertheless, multiple other similar sites exist, with large parts of their PDFs non-overlapping, so that at some point, all should be saved. Similar sites include http://doc.chipfind.ru/ (1.6m datasheets), http://www.alldatasheet.com/ (20m datasheets), http://www.datasheets.com/ (250m datasheets), http://www.datasheetcatalog.com/, http://freedatasheets.com/ and several others
- Delicious ( • • • ) loves to change their API, which has a side effect of making it difficult to back up.
- Encyclopedia Dramatica ( • • • ) is frequently up, down, and changing domains due to general Internet drama.
- Facebook ( • • • ) seems stable at the moment.
- Fandom ( • • • ), the for-pay arm of Wikipedia (just kidding, it's a different company, but shares a lot of people) is a repository of directed, unsubject-to-wikipolitics wikis, many of them intense and completist. It'd be bad for them to go away.
- FanFiction ( • • • ) represents many thousands of user-generated stories, essays and huge amounts of work.
- Forrst ( • • • ) was shut down on April 14th, 2014, but all posts were archived by Forrst.
- FreewareFiles ( • • • ) is a treasure trove of free and open source completed softwares. It's been around for 20 years, but hasn't had its look updated in a long while.
- FurAffinity ( • • • )
- Google ( • • • ) wants you to think they will be here forever.
- h2g2 ( • • • ) was (among?) the first online, collaborative encyclopaedia(s).
- Identify Your Breyer - As of August 31st, 2022, "three curators" have come in to save the website. They're currently working on moving site hosts, but as of September 11th, 2022, they needed to push it back "a few weeks". While the fate of the site is still somewhat uncertain, it's no longer in danger of expiring within the next few months.
- IFTTT ( • • • ) is still growing.
- Internet Archive ( • • • ) seems stable at the moment, but its 45 petabytes of data (as of October 2018) aren't mirrored anywhere else. The code for their system isn't open source, and generally they're a single point of failure for a large amount of the web's history. Why should there be only 1 internet archive?
- There is a second instance of the Wayback Machine at Bibliotheca Alexandrina, but it appears to have been broken since several years as of October 2018 ("The Resource you have requested is temporarily unavailable" when trying to access a snapshot as of 2018-10-03). It also hasn't been updated since 2007, i.e. all crawl data since 2008 is only available at the Internet Archive.
- More discussion at INTERNETARCHIVE.BAK
- Invisionfree ( • • • ) is as far as I can tell not used by as many people nowadays as before, probably because other free forum hosts use better forum software like phpBB that has better layout and looks better. The copyright notice on zIFBoards.com, which Invisionfree.com redirects to, has not been updated since 2014.
- JanusVR is a company that aims to re-imagine the web as "rooms" interconnected with portals. These "rooms" can be hosted anywhere. Though the company and its technology are still fairly new, there are already cases of rooms with missing assets or even entire links broken, and ArchiveBot's ability to grab these rooms is somewhat limited.
- JSFiddle ( • • • ) is referenced in many StackOverflow answers, as well as other forums, etc. It shows no signs of going away, but should we archive it just in case?
- Know Your Meme ( • • • ) is at this point the de facto central repository for information on internet memes and culture. It is as popular as ever at the moment, but even with this popularity, former owners Rocketboom had trouble financing it. In the spring of 2011 was sold to Cheezburger Networks, a site which has been known to "reorganize" its properties, sometimes with a detrimental effect on content. Though it was quite a different story, I might remind people what happened to Encyclopedia Dramatica.
- Last.fm ( • • • ) is being cloned by free software developers in the form of Libre.fm -- they have a tool, Lastscrape which can get all your listening data out into a tab delimited text file.
- Literotica.com ( • • • ) Contains over 290,000 user-written stories and poems. First pass at a backup: part1.rar, part2.rar, part3.rar, part4.rar -- contains the text of all stories as of the backup date in XML format. (One page of one story is missing because it doesn't exist on the site; embedded images and audio are not included this time; non-English stories aren't labelled with their language).
- LiveJournal ( • • • ) fired a bunch of US-based developers, but is still serving from its new (presumably cheaper) data center in Montana.
- Mapillary ( • • • ) project similar to Google's Street View, but with CC BY-SA photos submitted by users. Particularly worrisome since Mapillary requires purchasing licenses to download a large number of photos, making it essentially a big silo. See more at http://blog.improve-osm.org/en/2016/11/a-glimpse-into-the-future-of-mapmaking-with-osm-2/
- Megalodon.jp ( • • • ) is a Japanese archiving website. Outside of Japanese users, reddit users use it sometimes.
- The Mod Archive ( • • • ) One of the largest collection of music modules.
- Mod DB ( • • • ) is the largest website dedicated to user generated game content, including mods (12,000) and addons (14,000) with a combined size of 4,5 TB of user-generated downloadable content.
- Moegirlpedia ( • • • ) is the biggest Chinese-language encyclopedia for popular culture.
- MUGEN Archive ( • • • ) Holds 1000's of user created addon content for the custom fighting game MUGEN. Much of the stuff uploaded here was thought lost forever. However, thanks to all the copyright infringing content and uploading a lot of content without the creators permission, the future of the site is questionable.
- nyaa ( • • • ) - Biggest torrent site of Japanese popular culture.
- Pastebin ( • • • ) is still getting filled with text.
- Pixiv ( • • • ) and deviantArt ( • • • ) are the largest Japanese and American (respectively) fanart (and valuable art in general) collections on the internet. Most of works are possibly included in many boorus. Someone also mentioned Minitokyo around here somewhere. IA has a 2008 wallpaper dump from there.
- Pouet ( • • • ) is an important site of the demoscene. It indexes and ranks demoscene productions ('prods') and also includes a free-for-all BBS-style forum. Demozoo is a site in the same vein with a slightly different focus. They both use the same ISP iirc so if that goes down a lot of user created content is lost.
- Reddit ( • • • ) is a content aggregator where many Digg users migrated in 2010. Attracted controversy in July 2015, with accusations of censorship and shadowbanning. Many controversial subreddits (with up to hundreds of thousands subscribers each) are "quarantined"; many of these have subsequently been deleted. Stable for now, but team is small.
- SourceForge ( • • • ) is a critical repository of open source code, information, and webpages. It is mirrored and maintained, but there are sure to be parts that are neither.
- The Pirate Bay ( • • • ) is one of the largest and most popular torrent search engines. It's still having persistent legal problems. The tracker went down in November 2012, but the site still serves torrents and magnet links. If a torrent is lost, it becomes impossible to connect to other computers distributing the shared files. Considering that there are links to TPB all over this wiki, this site is pretty dang important. After they were raided in December 2014, a project known as The Open Bay was launched, which lets anybody host a mirror of TPB with automatic database updates, so even if TPB goes down again, temporarily or not, its database is still available.
- Supercard download section should be a priority as some flashcarts rely on software hosted there. • • • ) is the website of a manufacturer of Nintendo DS flashcarts (notably the DSTWO). The website is fine but it has not been updated in years and some section of it are broken. Archiving the
- Tribe ( • • • ) hosts large amount of user-generated data, and has been having consistent uptime issues.
- Tumblr ( • • • ) is a highly popular blogging platform which was bought by Yahoo! in May, 2013.
- TVTropes ( • • • ) is a popular wiki dedicated to finding recurring patterns in fiction, and discussing fiction in general. No word on whether there are backups. The administrators have a tendency to delete things indiscriminately, usually to save on disk space: article edit histories are frequently purged, and old forum threads have been known to get deleted mercilessly. A backup and an alternate website with imported content from July 2012 are available.
- Urban Dictionary ( • • • ) is "a crowdsourced online dictionary for slang words and phrases". It's financed by ads and a web store, and there are no signs of serious trouble as of late 2019.
- WebCite ( • • • ) itself seems to be having trouble with funding, and is facing "possible discontinuation." As this site serves as a stable reference for fleeting Web references, it would be pretty disastrous if it went away.
- whitehouse.gov ( • • • ) is overhauled every time a new US president assumes office and changes continuously during the term. Old versions are preserved thanks to the Presidential Records Act (e.g. George W. Bush's), but we also want to watch out for site changes / disappeared pages that were embarrassing or whatnot.
- WikiLeaks ( • • • ) contains several thousand leaked documents from sources such as the Iraq War and the cables famously known under the label 'Cablegate'. Due to the content on the website, and that PayPal and Amazon (very) quickly dropped their hosting for them during Cablegate's opening days, it should be considered a potential target for any number of government committees for a quick shutdown. They have an uncertain financial situation, and the site was inaccessible for some time in 2010.
- Wikidot ( • • • ) has not had development work for years, there is no official presence on their support forum, just many complaints about the difficulty contacted staff, even from those with paid accounts. Backup and wiki conversion tools are poor, services like search and anti-spam bots have problems for some time, and the site has been up for sale since February 2020.
- SCP Foundation is hosted on Wikidot. They plan on moving off of the platform and there are multiple archives, but there are still over 10 years' worth of creative writing that could be lost forever if Wikidot were to shut down.
- Wikipedia ( • • • ) will surely be here forever and ever! Fortunately, we don't have to take their word for it as they offer dumps of the data minus the photos. However, no-one has verified that Wikipedia can actually be restored from these dumps. If disaster strikes then we could discover a serious problem.
- Writing ( • • • ) A website for writing that was big in the 2000's. More and more restricted to guests and free users, as they are in need of money to keep the site running. Less and less popular for that reason
- YTMND - Resurrected on March 31st, 2020 with an archive being made in June 2018. Still on the Watchlist because of its past, but mostly safe for the time being.
Did someone leave the oven on?
- Kiwi Farms ( • • • , • • • ) is a notorious forum where users mock "lolcows", or people who have attracted ridicule because of their behavior or their beliefs. On January 20, 2017, the forum was shut down without a warning by its owner, but was restored three weeks later. As of September 2022, the website has been excluded from the Wayback Machine and the owner is struggling to find a new host due to a dispute with Cloudflare.
- Ning in 2010 has laid off 40% of staff and seems to be running out of money . There is certainly some networks worth archiving among the 2 million networks they host. Grouply and Posterous say they are going to offer migration tools.
- As of 2014, ScraperWiki Classic is now read-only. But don’t worry! You can transfer this scraper to Morph.io if you want to continue editing it.
- debates.oireachtas.ie on September 18th, 2012 the Houses of Oireachtas website announced that it would no longer be updating its XML data for Irish parliamentary debates (1919-2012). Access to pre-existing data is still available, but is likely to disappear, if the current trend continues. It would be useful to at least capture the XML data that is there, while it is still available. Here's a WARC archive of the XML only.
- Groklaw will no longer be posting new articles, "due to government monitoring of the internet, particularly e-mail." Whether or not its archives will remain online is unclear, although it does seem rather unlikely it will 100% disappear. OTOH, better safe than sorry. Still up as of October 11th, 2022.
- World of Spectrum's current administrator announced that he's ceasing to support the website and forum within 8 weeks (as of July 3). The future of the website is uncertain. Still up as of October 11th, 2022.
- The Centralstation Community has closed. The site is a UK-based social network for artists and creatives that provides hosting for content and portfolio. Users are being advised to back up their work as the new version of their platform will rely on existing media hosting sites like Flickr, Vimeo, and Soundcloud. Still up as of October 11th, 2022.
- Most of the paid staff at The Escapist has been "relieved of their duties" as of October 20, 2017, and the future and longevity of the site is uncertain; it's currently run mostly through volunteer efforts.
- Yelp, Inc. lost 30% of its advertisers and people don't seem too happy about it.
- Cheezburger • • • , once a “network” of meme blogs but now one single centralized site, has barely been maintained by its parent company (Cracked.com) for years, with the account creation system being broken since at least 2019. A lot of old meme images (of the Impact-font/rage-face/advice-animal variety) can be found here, dating back to 2007, and it would be a loss to the unique culture of the Internet if all these were to disappear. A lot of its contemporaries (such as Lolcats.com) have been lost to domain expiries and the ensuing cyber-squatters. There are still some dedicated users posting brand-new meme material here, but some people held on to GeoCities till the bitter end as well.
- Twitter ( • • • ) is tweaking away, with a dire financial situation and controversial decisions. Now faces potential bankruptcy after being purchased by Elon Musk.
I smell smoke.
- Mozilla Hubs Blog mozilla hubs early access release. Mozilla Hubs is changing their payment structure from open source to a subscription service. They will be deleting open source content.
- The Correspondent, From 30 Sep 19 to 1 Jan 21 The Correspondent published member-funded journalism about the forces that shape our world. The organization has shut down and has put up a read-only archive of all publications that were previously only accessible to paying members. As keeping such an archive running will probably cost someone money it might not stay online for many more years.
- YoyoGames, developer of the GameMaker application, is planning to retire the old "GameMaker Sandbox" game hosting website in favor of the "GameMaker: Player" service, by late October. "Sandbox content will remain available for a period of time until the GameMaker: Player is fully live."
- Giphy: Bought by Facebook, to be "integrated" (assimilated) into Instagram https://news.knowyourmeme.com/news/facebook-to-buy-giphy
- Dayviews ( • • • ), the Swedish photo diary community contains quite a bit of 00s Swedish "youth culture". It's had "technical problems" for months now and has said that "many old photos were lost".
- Surrender at 20, a popular League of Legends news and teardown site, has suddenly ceased activity on November 14th, 2022. Site owner moobeat posted a tweet saying "life sucks major shit right now" and alluded to some personal woes he's been going through. moobeat's wife, Aznbeat, posted a comment on November 25th on the site's latest post stating that while she helped with running the site to pay their bills after moobeat was hospitalized for a while, she otherwise "had no interest in League, nor will I" and that "unless by some miracle he ever decides to take some responsibility for it, then don't expect any more coverage from us".