WikiTeam
Jump to navigation
Jump to search
+80 wikis saved to date
Welcome to WikiTeam. A wiki is a website that allows the creation and editing of any number of interlinked web pages, generally used to store information on a specific subject or subjects. This is done with a day-to-day web browser using a simplified markup language (HTML as an example) or a WYSIWYG (what-you-see-is-what-you-get) text editor.
Examples of huge wikis:
- Wikipedia - arguably the largest and one of the oldest Wikis on the planet. It offers public backups: http://dumps.wikimedia.org
- Wikimedia Commons - a Wiki of media files available for free usage. It offers public backups: http://dumps.wikimedia.org
- But there is no image dump available, only the image descriptions
- Wikia - a website that allows the creation and hosting of wikis. It offers public backups: http://wiki-stats.wikia.com
Most of the wikis don't offer public backups. How bad!
Tools and source code
- WikiTeam Google Code repository
- dumpgenerator.py to download MediaWiki wikis: python dumpgenerator.py --api=http://archiveteam.org/api.php --xml --images
- wikipediadownloader.py to download Wikipedia dumps from download.wikimedia.org: python wikipediadownloader.py
- There are many wiki engines, the most famous is MediaWiki. So, the tools must me ready to read data from almost every wiki engine to saved them all.
- Scripts of a guy who saved Wikitravel
Coordination
Talk with WikiTeam at irc://efnet/wikiteam
Wiki dumps
For a more detailed list, visit the download section on Google Code.
There is another site of MediaWiki dumps located here on Scott's Website.
TODO lists:
- WikiTeam/Sites using MediaWiki (English)
- WikiTeam/Sites using MediaWiki (Multilingual)
- Backup your favorite wikis or leave the URL here.
Wiki | Wiki is online? | Dumps available? (official or home-made) | Comments/Details | Saved by us? Who? Where? |
---|---|---|---|---|
Anarchopedias | Yes | Official: no. Home-made: Yes | - | idiolect |
Archive Team Wiki | Yes | Official: no. Home-made: yes | - | WikiTeam |
Bulbapedia | Yes | Official: no. Home-made: no | - | dr-spangle is working on it with a self-built PHP downloader |
Citizendium | Yes | Official: daily (no full history). Home-made: yes, April 2011 | No image dumps available | |
EditThis | Yes | Official: no. Home-made: in progress | ||
enciclopedia.us.es | Yes | Official: no. Home-made: no | Sysop sent me page text sql tables | emijrp |
Encyclopedia Dramatica | No | Official: no. Home-made: partial | WebEcology Project Article Dump (~9000 Articles) Most of the Images probably Lost |
|
Gentoo wikis | Yes | Official: no. Home-made: yes | WikiTeam | |
GNUpedia | No | Official: no. Home-made: no | No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed | - |
Metapedia | Yes | Official: ?. Home-made: no | - | - |
Neoseeker | Yes | Official: ?. Home-made: no | - | - |
Nupedia | No | Official: ?. Home-made: Yes, saved from IA | - | - |
OmegaWiki | Yes | Official: daily | - | - |
OpenStreetMap | Yes | Official: Yes. Home-made: no | ||
OpenSUSE wikis | Yes | Official: no. Home-made: in progress | - | - |
OSDev | Yes | Official: weekly | - | Not yet |
Scout wikis | ||||
TV Tropes | Yes | Official: No Unofficial: In progress | No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org" | DoubleJ |
Uncyclomedias | ||||
Wikanda | Yes | Official: no. Home-made: yes | - | emijrp |
Wikia | Yes | Official: on demand | No image dumps available | Not yet |
WikiFur | Yes | Official: yes | No image dumps available | Not yet |
WikiHow | ||||
Wikimedia Commons | Yes | Official: periodically | No image dumps available | Not yet |
Wikipedia | Yes | Official: periodically | No image dumps available. English Wikipedia dump uses to be very old | Not yet |
Wiki-site.com | ||||
WikiTravel | Yes | Official: not yet. Home-made: yes, another of 2010-06-14 | - | WikiTeam |
WikiWikiWeb | Yes | Home-made: yes | - | Ca7 |
Tips
Some tips:
- When downloading Wikipedia/Wikimedia Commons dumps, pages-meta-history.xml.7z and pages-meta-history.xml.bz2 are the same, but 7z use to be smaller (better compress ratio), so use 7z.
Closing/In danger
- Gentoo wikis: Error 503 Service Unavailable as of 2011-04-06 http://s23.org/wikistats/gentoo_html.php
Offline wikis and wikifarms
elwiki.com
- 2011
- wik.is
- 2010
- 2009
- 2008
- Scribblewiki (wikifarm)
External links
- http://wikiindex.org - A lot of wikis to save
- http://wiki1001.com/
- http://meta.wikimedia.org/wiki/List_of_largest_wikis
- http://s23.org/wikistats/
- http://en.wikipedia.org/wiki/Comparison_of_wiki_farms
- http://en.wikipedia.org/wiki/User:Emijrp/Wikipedia_Archive
- http://blog.shoutwiki.com/
- http://wikiheaven.blogspot.com/
- List of largest wikis in the world
- Dump of nostalgia, an ancient version of Wikipedia from 2001, dump
- http://code.google.com/p/wikiteam/downloads/list?can=1 many dumps