WikiTeam

We save wikis, from Wikipedia to tiniest wikis
100 wikis saved to date

WikiTeam
WikiTeam, a set of tools for wiki preservation and a repository of wikis
URL	http://code.google.com/p/wikiteam
Status	Online!
Archiving status	In progress...
Archiving type	Unknown
IRC channel	#wikiteam (on hackint)

Welcome to WikiTeam. A wiki is a website that allows the creation and editing of any number of interlinked web pages, generally used to store information on a specific subject or subjects. This is done with a day-to-day web browser using a simplified markup language (HTML as an example) or a WYSIWYG (what-you-see-is-what-you-get) text editor.

Examples of huge wikis:

• Wikipedia - arguably the largest and one of the oldest Wikis on the planet. It offers public backups: http://dumps.wikimedia.org
• Wikimedia Commons - a Wiki of media files available for free usage. It offers public backups: http://dumps.wikimedia.org
  • But there is no image dump available, only the image descriptions
• Wikia - a website that allows the creation and hosting of wikis. It offers public backups: http://wiki-stats.wikia.com

Most of the wikis don't offer public backups. How bad!

Tools and source code

• WikiTeam Google Code repository
• dumpgenerator.py to download MediaWiki wikis: python dumpgenerator.py --api=http://archiveteam.org/api.php --xml --images
• wikipediadownloader.py to download Wikipedia dumps from download.wikimedia.org: python wikipediadownloader.py
• There are many wiki engines, the most famous is MediaWiki. So, the tools must me ready to read data from almost every wiki engine to saved them all.
• Scripts of a guy who saved Wikitravel
• OddMuseWiki backup
• UseModWiki: use wget/curl and raw mode (might have a different URL scheme, like this)

Wiki dumps

For a more detailed list, visit the download section on Google Code.

There is another site of MediaWiki dumps located here on Scott's Website.

TODO lists:

• WikiTeam/Sites using MediaWiki (English)
• WikiTeam/Sites using MediaWiki (Multilingual)
• Backup your favorite wikis or leave the URL here.

Wiki	Wiki is online?	Dumps available? (official or home-made)	Comments/Details	Saved by us? Who? Where?
Anarchopedias	Yes	Official: no. Home-made: Yes	-	idiolect
Archive Team Wiki	Yes	Official: no. Home-made: yes	-	WikiTeam
Bulbapedia	Yes	Official: no. Home-made: no	-	dr-spangle is working on it with a self-built PHP downloader
Citizendium	Yes	Official: daily (no full history). Home-made: yes, April 2011	No image dumps available
EditThis	Yes	Official: no. Home-made: in progress
enciclopedia.us.es	Yes	Official: no. Home-made: no	Sysop sent me page text sql tables	emijrp
Encyclopedia Dramatica	No	Official: no. Home-made: partial	WebEcology Project Article Dump (~9000 Articles) Most of the Images probably Lost
Gentoo wikis	Yes	Official: no. Home-made: yes		WikiTeam
GNUpedia	No	Official: no. Home-made: no	No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed	-
MeatBall	Yes	Official: no. Home-made: yes (mirror)	No histories, no xml format	SDBoyd
Metapedia	Yes	Official: ?. Home-made: no	-	-
Neoseeker	Yes	Official: ?. Home-made: no	-	-
Nupedia	No	Official: ?. Home-made: Yes, saved from IA	-	-
OmegaWiki	Yes	Official: daily	-	-
OpenStreetMap	Yes	Official: Yes. Home-made: no
OpenSUSE wikis	Yes	Official: no. Home-made: in progress	-	-
OSDev	Yes	Official: weekly	-	Not yet
Scout wikis
TV Tropes	Yes	Official: No Unofficial: In progress	No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org"	DoubleJ
Uncyclomedias
Wikanda	Yes	Official: no. Home-made: yes	-	emijrp
Wikia	Yes	Official: on demand	No image dumps available	Not yet
WikiFur	Yes	Official: yes	No image dumps available	Not yet
WikiHow
Wikimedia Commons	Yes	Official: periodically	No image dumps available	Not yet
Wikipedia	Yes	Official: periodically	No image dumps available. English Wikipedia dump uses to be very old	Not yet
Wiki-site.com
WikiTravel	Yes	Official: not yet. Home-made: yes, another of 2010-06-14	-	WikiTeam
WikiWikiWeb	Yes	Home-made: yes	-	Ca7
(o:forum	Yes	No	-	Not yet, to figure out how
WikiWiki.de	Yes	No	-	Not yet, to figure out how
GruenderWiki	Yes	No	-	Not yet, to figure out how

Tips

Some tips:

• When downloading Wikipedia/Wikimedia Commons dumps, pages-meta-history.xml.7z and pages-meta-history.xml.bz2 are the same, but 7z use to be smaller (better compress ratio), so use 7z.

BitTorrent downloads

BitTorrent downloads are available for most, if not all, files posted to the WikiTeam Google Code Downloads. You can subscribe to a feed of the latest torrents:

• link coming shortly, keeps getting incorrectly blacklisted for "ru" domain (pipes.yahoo.com)

Please note that there may be some delay after a file is uploaded before the torrent appears on the feed, due to caching at Yahoo Pipes and other factors. You can subscribe to this feed for automatic downloads (this has been tested successfully in µTorrent on Windows).

Closing/In danger

• Gentoo wikis: Error 503 Service Unavailable as of 2011-04-06 http://s23.org/wikistats/gentoo_html.php
  • Again up. Saved! Emijrp 21:30, 10 April 2011 (UTC)

Offline wikis and wikifarms

elwiki.com

• 2011
  • wik.is
• 2010
• 2009
• 2008
  • Scribblewiki (wikifarm)

External links

• http://wikiindex.org - A lot of wikis to save
• http://wiki1001.com/
• http://meta.wikimedia.org/wiki/List_of_largest_wikis
• http://s23.org/wikistats/
• http://en.wikipedia.org/wiki/Comparison_of_wiki_farms
• http://en.wikipedia.org/wiki/User:Emijrp/Wikipedia_Archive
• http://blog.shoutwiki.com/
• http://wikiheaven.blogspot.com/
• List of largest wikis in the world
• Dump of nostalgia, an ancient version of Wikipedia from 2001, dump
• http://code.google.com/p/wikiteam/downloads/list?can=1 many dumps