Digital Preservation 2013 Transcript

From Archiveteam
Jump to navigation Jump to search

Hello everyone, my name is Jason Scott. I don’t know how many people know of me, but my name’s Jason Scott – just remember the hat.

I am the harbinger of death. I am the angel of death, I am the sad grim reaper that sits at the crossroads of your lost and dying dreams. I am the boat man on the river Styx who takes your hard drive from you, and then rides with you across the river to your later destiny. When the handshakes no longer happen and when the smiles fade, that’s where I’m living.

I’m living in this world because I helped found something called Archive Team, And Archive Team is a preservationist, activist group who have been around for three years now, since we founded in 2009. Right at this moment we’ve got multiple projects going on. I believe so much in not just talking about things that might be, or whatever, but I’ll just leave this running because it’s hilarious…

Um, this is one of our trackers. This is a blog community called Xanga, X-A-N-G-A, they are going down at the end of the month because they couldn’t raise enough money. We’ve downloaded 866,000 users and there’s 3.4 million to go. This one’s not going to end well.

We’re downloading something called Snapjoy, that one was a little bit easier, that’s downloading at the end of the month. Turns out they only had 3,000 users, so that wasn’t so hard to do. Formspring continues to be downloaded, we’ve downloaded over 60 terrabytes of

We’ve just recently finished up Posterous, which was a blogging community that was purchased by Twitter and shot in the face. (AUDIENCE LAUGHTER) And it.. it is basically going away, and we have downloaded all of their users.

So Archive Team is, like I said, dedicated to preserving things, because so much of what we have online right now is disappearing. And not just in that weird “link rot” that people like to talk about, but actual, forceful, destruction. I’m sure that many of you work in institutional groups where things are done along a much longer de-accessioning plan, and these are cases where things are being done very quickly, people are knocking things down. In some cases they’re giving no warning. When Myspace took away their blogs, they literally gave no warning. Zero. Millions of blogs from ten years, gone, without any kind of a mention.

And activists, of course, are really anathema to the structure of what we’re trying to have right now, right? But unfortunately activism is what we need, because we made a dramatic, terrible mistake. We let corporations and startups define the public and civil space online. So as a result, we ended up with a lot of shared communities that are sitting on real shifting sand, that has really no guarantees, no laws, really. Things can just kind of go away.

I’ll say it again later, but the fundamental question is “Is an online presence a valid humanity, and a valid humanitarian concern?” I happen to believe so. I happen to think that all the things that we’re doing offline, we’re now putting online. We’re not going to stop using the wheel, we’re not going to stop using the internet. Unfortunately, we are now victims of the Brogrammer/Journalist complex, which has worked together to really convince us that the place to put all of our stuff is with people we don’t know, for reasons we don’t know, until they decide that they are done with us. Or they’ve sold it to Google.

So, that sense of – We have three virtues within Archive Team, rage, paranoia and kleptomania. So basically we’re very angry about these things going away, we have an enormous paranoia about things that might go away at any given time, and we take everything, as fast as we can. So what you’re seeing on the right, right here, is something called the Archive Team Tracker, or as we like to call it, the Archive Team Warrior. And the Archive Team Warrior is a virtual appliance, you throw it on your box, it can run in Mac, Linux or Windows, and it boots up and says “So, what are we working on today?”, we have Formspring, Posterous, URL Team and Xanga. You hit Formspring, and once again, if you look on the right, it scrolls every once in a while as somebody grabs another Formspring site, and at the bottom eventually we have graphs, just showing, because people love achievements, right? So we always have people who could possibly win.

We haven’t really been building communities online, we have been holding parties. People know how to hold parties, and parties are great until parties end. There’ve been a lot of parties that have ended up out of vodka, out of Red Bull, with their chairs on the front lawn and people being left aside. There’s no penalties for this, there’s no sadness for it, it is now a common thing that when a business buys another business, what they now call a hiring acquisition, they just destroy everything before it because that way there’s no legal liability for whatever that group was doing, patent-wise or otherwise. So there’s a great amount of effort to get rid of it.

What kind of things are we working on otherwise? Well, we’re backing up Detroit, because Detroit’s in trouble! (AUDIENCE LAUGHTER) So, I mean, of all the people here, right, I mean you just heard, you all saw on the news, Detroit’s going bankrupt, right, they’re fighting the bankruptcy… Why aren’t you downloading copies of everything? Because who knows what websites are going to stay up after September.

We’re downloading FTP sites, remember them? Well, they’re dying, we’re grabbing those as fast as we can. CD-ROMs.. 25, you know 25-30 packs coming in by the hundreds, we’re just trying to download and grab every CD-ROM that we possibly can.

And URL shorteners, because… we got a thing about URL shorteners. Erm.. So we have URLTE.AM, because that’s the way I guess you do everything, right? URL Team has been aggressively downloading every URL shortener from every shortener provider, hundreds of them, for three years, turning them into Bzipped single columns, so here’s the short, here’s the long, and then making it available as a torrent once every six months. Because in ten years, you’re going to realise that you put all of your web history in a one-time cryptographic key pad owned by people you don’t know. So we’re working on that.

So there’s a whole variety of things, and the thing is, is that anybody here, I hope, can take away from us, kind of, if not all of our virtues, at least some of our approach to how we’re looking at the world, which is to realise that we really are putting all of these things online, putting them on very fast, and we did it really really quickly, and we have to remember… we uploaded humanity, right? We’re doing that now. We are putting it online, and yes, there’s people who think that paper and everything else is going to save us, but realise at this moment, you know… Yeah, we can make judgements about what’s important and what isn’t, but that’s been going on for a thousand years, and it hasn’t always worked out, Alexandria. (AUDIENCE LAUGHTER)

So, we want to make sure that people realise that this is going on right now, we kind of fell into that, and I think it’s going to right itself within the next 20 years, we’ll start to see more of a civic duty that comes with being a host, but right now they are tenements. They are tenements that we occasionally set on fire because we want to make a parking lot. And I think that’s really something that we’re going to move away from, but in the short term, please realise where you’re putting your stuff, and always keep an eye on it.

Archive Team thinks that history is the Z axis. It’s not just the little linear, it’s the Z axis that tells us a little bit more. It’s the fading song that we can hear in the background, the cacophony of the now. And so, this is what we’re doing. I don’t have any set of specific technical stuff I want to go into here. I have a lot of wonderful volunteers who are working very hard to make sure that as many things get saved as quickly as possible. People who are autistic, people who work as archivists, people who are just folks. Who fell for, and believe in, the dream. And what is the dream? The dream is the idea that data has human meaning. It’s no longer a case of just being something that tells you where the loom goes, or tells you how to add 5 and make a 6, but it’s where our dreams are going, it’s where the things we believe are going, and when we look back it’ll help us understand what we were. Thanks.


I can quickly shout back an unpleasant response to any question anyone has.


There’s one right there, go ahead.



AUDIENCE MEMBER: Hi, I’m Kim Schroeder and I’m from Detroit.


JASON: Backing you up!

KIM: So I want to thank you so much for backing us up, we’ve needed for about 60 years, but I wondered how are you choosing what you’re backing up, so in the instance of Detroit, you say backing up Detroit, are you talking about the city official website, or -- ?

JASON: Any government site we can find, any site that references Detroit in terms of being, you know, like Detroit’s in the title of what it is, any mentions of Detroit on any blogs we can find, just in case those are things like Tumblr blogs run by a particularly egregiously happy and forthright looking Detroit employee. Anything where we can find something with a reference to Detroit in the file, sorry, the domain name. And we’re basically backing a lot of these up and putting them in the Internet Archive Wayback Machine, so that way there’s some copy of it. Now we’re not going to get everything, obviously, but the idea is, for instance, when.. uh.. um.. uh.. what's his name? He ran Libya, what’s his name? He was a cool guy, until he wasn’t. What? Gaddafi. So when Gaddafi like blew out, because it turned out he was a murderous thug, we went and we said “You know, I think his site’s probably going to go down.” And we downloaded it, and ten days later the ISP was told that if they didn’t take it down, they’d be murdered, which is the ultimate DMCA. So we have the copy of everything from Gaddafi, because in 50 years you’ll want to go “What was up with that guy? OK, wow, he talks about himself a lot.” And so, you know, that kind of thinking – and we’re trying to kind of infect young people. I mean, Archive Team, our youngest member is 15, our oldest member is 60, we try to have everybody feeling that this is something they need to do, and think in this way. And, you know, I think over time we’ll get better, but until then, backing up Detroit by hand. And I hope some people from Detroit are backing themselves up too. I think there’s gonna be – it’s going to be interesting. New York was like this for a while in the 70s, when they kind-of like started to lose a lot of their history because things were really going off. Actually right now I think it’s like – I think the court just said they can’t go bankrupt quite yet.

KIM: Well I’m here with eight of our students, from the University, so if you need some help...

JASON: Too Motown To Fail! Yeah, I’d love to help them. Anybody else have a question? Before I get pulled out. And I mean, literally frog-walked out of here. (PAUSE) I mean, I can’t see you because I’m literally staring into Rigel and Sol here. (PAUSE) There are three lights! No? OK, well, thanks a lot for your time!


v · t · e         Archive Team
Current events

Alive... OR ARE THEY · Deathwatch · Projects

Archiving projects

APKMirror · · BetaArchive · Government Backup (#datarefuge · ftp-gov· Gmane · Internet Archive · It Died · · · · OSBetaArchive · TEXTFILES.COM · The Dead, the Dying & The Damned · The Mail Archive · UK Web Archive · WebCite ·

Blogging · Blogger · Blogster · · · Fuelmyblog · Jux · LiveJournal · My Opera · · Open Diary · · Posterous · Powerblogs · Proust · Roon · Splinder · Tumblr · Vox · · Windows Live Spaces · · Xanga · Yahoo! Blog · Zapd

Cloud hosting/file sharing

aDrive · AnyHub · Box · Dropbox · Docstoc · · Google Drive · Google Groups Files · iCloud · Fileplanet · LayerVault · MediaCrush · MediaFire · Mega · MegaUpload · MobileMe · OneDrive · · RapidShare · Ubuntu One · Yahoo! Briefcase


Apple · IBM · Google · Loblaw · Lycos Europe · Microsoft · Yahoo!


Arab Spring · Great Ape-Snake War · Spanish Revolution

Font Repos

DaFont · Google Web Fonts · GNU FreeFont · Fontspace

Forums/Message boards

4chan · Captain Luffy Forums · College Confidential · Discourse · DSLReports · ESPN Forums · Facepunch Forums · · HeavenGames · JamiiForums · Invisionfree · NeoGAF · Textream · The Classic Horror Film Board · Yahoo! Messages · Yahoo! Neighbors · · Zetaboards


Atomicgamer · · City of Heroes · Club Nintendo · Clutch · Counter-Strike: Global Offensive · CS:GO Lounge · Desura · Dota 2 · Dota 2 Lounge · Emulation Zone · ESEA · GameBanana · GameMaker Sandbox · GameTrailers · Halo · Heroes of Newerth · · HQ Trivia · Infinite Crisis · joinDOTA · League of Legends · Liquipedia · · · Playfire · Raptr · SingStar · Steam · SteamDB · SteamGridDB · Team Fortress 2 · TF2 Outpost · Warhammer · Xfire

Image hosting

500px · AOL Pictures · Blipfoto · Blingee · · Camera+ · Cameroid · DailyBooth · Degree Confluence Project · DeviantART · · Flickr · · · Fotopedia · Frontback · Geograph Britain and Ireland · Giphy · GTF Képhost · ImageShack · · Imgur · Inkblazers · Instagram · · · · · · Madden GIFERATOR · MLKSHK · Microsoft Clip Art · Microsoft Photosynth · Nokia Memories · · Odysee · Panoramio · Photobucket · Picasa · Picplz · Pixiv · · PSharing · Ptch · · Rawporter · · · Sketch · Smack Jeeves · Snapjoy · Streetfiles · Tabblo · Tinypic · Trovebox · TwitPic · Wallbase · Wallhaven · Webshots · Wikimedia Commons


arXiv · Citizendium · · Deletionpedia · EditThis · Encyclopedia Dramatica · Etherpad · Everything2 · infoAnarchy · GeoNames · GNUPedia · Google Books (Google Books Ngram· Horror Movie Database · Insurgency Wiki · Knol · Lost Media Wiki · · · Nupedia · OpenCourseWare · OpenStreetMap · Orain · Pastebin · · Project Gutenberg · Puella Magi · Referata · Resedagboken · SongMeanings · ShoutWiki · The Internet Movie Database · TropicalWikis · Uncyclopedia · Urban Dictionary · Urban Exploration Resource · Webmonkey · Wikia · Wikidot · WikiHow · Wikkii · WikiLeaks · Wikipedia (Simple English Wikipedia· Wikispaces · Wikispot · · Wiki-Site · WikiTravel · Word Count Journal

Magazines/Blogs/News · Game Developer Magazine · Gigaom · Hardware Canucks · Helium · JPG Magazine · Make Magazine · The Escapist · · San Fransisco Bay Guardian · Scoop · Regretsy · Yahoo! Voices


Heello · · Jaiku · · Plurk · Sina Weibo · Tencent Weibo · Twitter · TwitLonger


8tracks · AOL Music · · Cinch · digCCmixter · · Earbits · exfm · Free Music Archive · Gogoyoko · Indaba Music · Instacast · Instaudio · Jamendo · · Music Unlimited · MOG · PureVolume · Reverbnation · ShareTheMusic · SoundCloud · Soundpedia · Spotify · This Is My Jam · TuneWiki · · WinAmp


Aaron Swartz · Michael S. Hart · Steve Jobs · Mark Pilgrim · Dennis Ritchie · Len Sassaman Project


FTP · Gopher · IRC · Usenet · World Wide Web
BitTorrent DHT


Askville · Answerbag · · · Askalo · Baidu Knows · Blurtit · ChaCha · Experts Exchange · Formspring · GirlsAskGuys · Google Answers · Google Baraza · JustAnswer · MetaFilter · Quora · Retrospring · StackExchange · The AnswerBank · The Internet Oracle · Uclue · WikiAnswers · Yahoo! Answers


Allrecipes · Epicurious · · Foodily · Food Network · Punchfork · ZipList

Social bookmarking

Addinto · Backflip · Balatarin · BibSonomy · Bkmrx · Blinklist · BlogMarks · BookmarkSync · CiteULike · Connotea · Delicious · Designer News · Digg · Diigo · · Evernote · Excite Bookmark · Faves · Favilous · folkd · Freelish · Getboo · · Gnolia · Google Bookmarks · Hacker News · HeyStaks · IndianPad · Kippt · Knowledge Plaza · Licorize · Linkwad · Menéame · Microsoft Developer Network · myVIP · Mister Wong · My Web · Mylink Vault · Newsvine · Oneview · Pearltrees · Pinboard · Pocket · · Reddit · · Scloog · Scuttle · Simpy · SiteBar · Slashdot · Squidoo · StumbleUpon · Twine · Voat · Vizited · Yummymarks · Xmarks · Yahoo! Buzz · Zootool · Zotero

Social networks

Bebo · BlackPlanet · · Cyworld · Dogster · Dopplr · douban · Ello · Facebook · Flixster · FriendFeed · Friendster · Friends Reunited · Gaia Online · Google+ · Habbo · hi5 · Hyves · iWiW · LinkedIn · Miiverse · mixi · MyHeritage · MyLife · Myspace · myVIP · Netlog · Odnoklassniki · Orkut · Plaxo · Qzone · Renren · Skyrock · · Storylane · Tagged · tvtag · Upcoming · Viadeo · Vine · VK · WeeWorld · Weibo · Wretch · Yahoo! Groups · Yahoo! Stars India · Yahoo! Upcoming · more sites...


Alibaba · AliExpress · Amazon · Apple Store · Barnes & Noble · DirectCanada · eBay · Kmart · NCIX · Printfection · RadioShack · Sears · Sears Canada · Target · The Book Depository · ThinkGeek · Toys "R" Us · Walmart

Software/code hosting

Android Development · Alioth · Assembla · BerliOS · Betavine · Bitbucket · BountySource · Codecademy · CodePlex · Freepository · Free Software Foundation · GNU Savannah · GitHost  · GitHub · GitHub Downloads · Gitorious · Gna! · Google Code · ibiblio · · JavaForge · KnowledgeForge · Launchpad · LuaForge · Maemo · mozdev · · OW2 Consortium · Openmoko · OpenSolaris · · Ovi Store · Project Kenai · RubyForge · · SourceForge · Stypi · TestFlight · · Transifex · TuxFamily · Yahoo! Downloads


ABC · Austin City Limits · BBC · CBC · CBS · Computer Chronicles · CTV · Fox · G4 · Global TV · Jeopardy! · NBC · NHK · PBS · Penn & Teller: Bullshit! · The Howard Stern Show · TV News Archive (Understanding 9/11)


ExtraTorrent · EZTV · isoHunt · KickassTorrents · The Pirate Bay · Torrentz · Library Genesis

Video hosting

Academic Earth · Bambuser · · Epic · Freshlive · Google Video · · Mixer · Niconico · Nokia Trailers · · Periscope · · Qwiki · Skillfeed · Stickam · TED Talks · · · Ustream · · Viddler · Viddy · Vidme · Vimeo · Vine · Vstreamers · Yahoo! Video · YouTube · Famous Internet videos (Me at the zoo)

Web hosting

Angelfire · · BT Internet · CableAmerica Personal Web Space · Claranet Netherlands Personal Web Pages · Comcast Personal Web Pages · · FortuneCity · Free ProHosting · GeoCities (patch· Google Business Sitebuilder · Google Sites · Internet Centrum · MBinternet · MSN TV · Nifty · Nwnyet · Parodius Networking · · Saunalahti Iso G · Swipnet · Telenor · Tripod · University of Michigan personal webpages · Verizon Mysite · Verizon Personal Web Space · Webs · Webzdarma · Virgin Media

Web applications

Mailman · MediaWiki · phpBB · Simple Machines Forum · vBulletin


A Million Ways to Die on the Web · Backup Tips · Cheap storage · Collecting items randomly · Data compression algorithms and tools · Dev · Discovery Data · DOS Floppies · Fortress of Solitude · Keywords · Naughty List · Nightmare Projects · Rescuing floppy disks · Rescuing optical media · Site exploration · The WARC Ecosystem · Working with ARCHIVE.ORG


ArchiveCorps · Audit2014 · Emularity · Faceoff · FlickrFckr · Froogle · INTERNETARCHIVE.BAK (Internet Archive Census· IRC Quotes · JSMESS · JSVLC · Just Solve the Problem · NewsGrabber · Project Newsletter · Valhalla · Web Roasting (ISP Hosting · University Web Hosting· Woohoo


ArchiveBot · ArchiveTeam Warrior (Tracker· Google Takeout · HTTrack · Video downloaders · Wget (Lua · WARC)


Bibliotheca Anonoma · LibreTeam · URLTeam · Yahoo Video Warroom · WikiTeam


800notes · AOL · Akoha · · April Fools' Day · Amplicate · AutoAdmit · · Circavie · Cobook · Co.mments · Countdown · Discourse · Distill · Dmoz · Easel · Eircode · Electronic Frontier Foundation · FanFiction.Net · Feedly · Ficlets · Forrst · · FurAffinity · Google Helpouts · Google Moderator · Google Poly · Google Reader · ICQmail · IFTTT · Jajah · JuniorNet · Lulu Poetry · Mobile Phone Applications · Mochi Media · Mozilla Firefox · MyBlogLog · NBII · Newgrounds · Neopets · Quantcast · Quizilla · Salon Table Talk · Shutdownify · Slidecast · Stack Overflow · SOPA blackout pages · · TechNet · Toshiba Support · USA-Gov · Volán · Widgetbox · Windows Technical Preview · Wunderlist · YTMND · Zoocasa

About Archive Team

Introduction · Philosophy · Who We Are · Our stance on robots.txt · Why Back Up? · Software · Formats · Storage Media · Recommended Reading · Films and documentaries about archiving · Talks · In The Media · FAQ