Difference between revisions of "WikiTeam"
m (→Wiki dumps: Put "Legend" above Legend table) |
(→Wiki dumps: add mirror listing) |
||
Line 133: | Line 133: | ||
* [http://pipes.yahoo.com/lobstor/google_code_torrent?_render=rss&project=wikiteam WikiTeam Torrent Feed] (pipes.yahoo.com) | * [http://pipes.yahoo.com/lobstor/google_code_torrent?_render=rss&project=wikiteam WikiTeam Torrent Feed] (pipes.yahoo.com) | ||
Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in µTorrent on Windows). | Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in µTorrent on Windows). | ||
=== Mirrors === | |||
<span class="plainlinks"> | |||
# [https://sourceforge.net/projects/wikiteam/files/ Sourceforge] (also mirrored to another 26 mirrors) | |||
# [http://www.archive.org/details/WikiTeamMirror Internet Archive] ([http://ia700705.us.archive.org/16/items/WikiTeamMirror/ direct link] to directory) | |||
</span> | |||
== Closing/In danger == | == Closing/In danger == |
Revision as of 15:12, 8 November 2011
100 wikis saved to date
WikiTeam | |
WikiTeam, a set of tools for wiki preservation and a repository of wikis | |
URL | http://code.google.com/p/wikiteam |
Status | Online! |
Archiving status | In progress... |
Archiving type | Unknown |
IRC channel | #wikiteam (on hackint) |
Welcome to WikiTeam. A wiki is a website that allows the creation and editing of any number of interlinked web pages, generally used to store information on a specific subject or subjects. This is done with a day-to-day web browser using a simplified markup language (HTML as an example) or a WYSIWYG (what-you-see-is-what-you-get) text editor.
Examples of huge wikis:
- Wikipedia - arguably the largest and one of the oldest Wikis on the planet. It offers public backups: http://dumps.wikimedia.org
- Wikimedia Commons - a Wiki of media files available for free usage. It offers public backups: http://dumps.wikimedia.org
- But there is no image dump available, only the image descriptions
- Wikia - a website that allows the creation and hosting of wikis. It offers public backups: http://wiki-stats.wikia.com
Most of the wikis don't offer public backups. How bad!
Tools and source code
- WikiTeam Google Code repository
- dumpgenerator.py to download MediaWiki wikis: python dumpgenerator.py --api=http://archiveteam.org/api.php --xml --images
- wikipediadownloader.py to download Wikipedia dumps from download.wikimedia.org: python wikipediadownloader.py
- There are many wiki engines, the most famous is MediaWiki. So, the tools must be ready to read data from almost every wiki engine to saved them all.
- Scripts of a guy who saved Wikitravel
- OddMuseWiki backup
- UseModWiki: use wget/curl and raw mode (might have a different URL scheme, like this)
Wiki dumps
For a more detailed list, visit the download section on Google Code.
There is another site of MediaWiki dumps located here on Scott's Website.
TODO lists:
- WikiTeam/Sites using MediaWiki (English)
- WikiTeam/Sites using MediaWiki (Multilingual)
- Backup your favorite wikis or leave the URL here.
Legend | |
Good | |
Could be better | |
Bad | |
Unknown |
Wiki | Wiki is online? | Dumps available? (official or home-made) | Comments/Details | Saved by us? Who? Where? |
---|---|---|---|---|
Anarchopedias | Yes | Official: no. Home-made: Yes | - | idiolect |
Archive Team Wiki | Yes | Official: no. Home-made: yes | - | WikiTeam |
Bulbapedia | Yes | Official: no. Home-made: no | - | dr-spangle is working on it with a self-built PHP downloader |
Citizendium | Yes | Official: daily (no full history). Home-made: yes, April 2011 | No image dumps available | - |
EditThis | Yes | Official: no. Home-made: in progress | - | - |
enciclopedia.us.es | Yes | Official: no. Home-made: no | Sysop sent me page text sql tables | emijrp |
Encyclopedia Dramatica | No | Official: no. Home-made: partial | WebEcology Project Article Dump (~9000 Articles) Most of the Images probably Lost |
- |
Encyclopedia Dramatica.ch (new ED) |
Yes | Official: ? Home-made: ? | Slowly being rebuilt from old sources. Should be up for a while but for who knows how long? |
- |
Gentoo wikis | Yes | Official: no. Home-made: yes | - | WikiTeam |
GNUpedia | No | Official: no. Home-made: no | No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed | - |
MeatBall | Yes | Official: no. Home-made: yes (mirror) | No histories, no xml format | SDBoyd |
Metapedia | Yes | Official: ?. Home-made: no | - | - |
Neoseeker aka Scout wikis | Yes | Official: ?. Home-made: no | - | - |
Nupedia | No | Official: ?. Home-made: Yes, saved from IA | - | - |
OmegaWiki | Yes | Official: daily | - | - |
OpenStreetMap | Yes | Official: Yes. Home-made: no | - | - |
OpenSUSE wikis | Yes | Official: no. Home-made: in progress | - | - |
OSDev | Yes | Official: weekly | - | Not yet |
TV Tropes | Yes | Official: No Unofficial: In progress | No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org" | DoubleJ |
Uncyclomedias | - | - | - | - |
Wikanda | Yes | Official: no. Home-made: yes | - | emijrp |
Wikia | Yes | Official: on demand | No image dumps available | Not yet |
WikiFur | Yes | Official: yes | No image dumps available | Not yet |
WikiHow | - | - | - | - |
Wikimedia Commons | Yes | Official: periodically | No image dumps available | Not yet |
Wikipedia | Yes | Official: periodically | No image dumps available. English Wikipedia dump uses to be very old | Not yet |
Wiki-site.com | - | - | - | - |
WikiTravel | Yes | Official: not yet. Home-made: yes, another of 2010-06-14 | - | WikiTeam |
WikiWikiWeb | Yes | Home-made: yes | - | Ca7 |
(o:forum | Yes | No | - | Not yet, to figure out how |
WikiWiki.de | Yes | No | - | Not yet, to figure out how |
GruenderWiki | Yes | No | - | Not yet, to figure out how |
Tips
Some tips:
- When downloading Wikipedia/Wikimedia Commons dumps, pages-meta-history.xml.7z and pages-meta-history.xml.bz2 are the same, but 7z use to be smaller (better compress ratio), so use 7z.
BitTorrent downloads
A feed of BitTorrent downloads is available for the latest files posted to the WikiTeam Google Code Downloads.
- WikiTeam Torrent Feed (pipes.yahoo.com)
Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in µTorrent on Windows).
Mirrors
- Sourceforge (also mirrored to another 26 mirrors)
- Internet Archive (direct link to directory)
Closing/In danger
- Gentoo wikis: Error 503 Service Unavailable as of 2011-04-06 http://s23.org/wikistats/gentoo_html.php
Offline wikis and wikifarms
elwiki.com
- 2011
- wik.is
- 2010
- 2009
- 2008
- Scribblewiki (wikifarm)
External links
- http://wikiindex.org - A lot of wikis to save
- http://wiki1001.com/
- http://meta.wikimedia.org/wiki/List_of_largest_wikis
- http://s23.org/wikistats/
- http://en.wikipedia.org/wiki/Comparison_of_wiki_farms
- http://en.wikipedia.org/wiki/User:Emijrp/Wikipedia_Archive
- http://blog.shoutwiki.com/
- http://wikiheaven.blogspot.com/
- List of largest wikis in the world
- Dump of nostalgia, an ancient version of Wikipedia from 2001, dump
- http://code.google.com/p/wikiteam/downloads/list?can=1 many dumps