Difference between revisions of "WikiTeam"

From Archiveteam
Jump to navigation Jump to search
Line 23: Line 23:


== Tools and source code ==
== Tools and source code ==
=== Official WikiTeam tools ===
* [http://code.google.com/p/wikiteam/ WikiTeam Google Code repository]
* [http://code.google.com/p/wikiteam/ WikiTeam Google Code repository]
* [http://code.google.com/p/wikiteam/source/browse/trunk/dumpgenerator.py dumpgenerator.py] to download MediaWiki wikis: <tt>python dumpgenerator.py --api=http://archiveteam.org/api.php --xml --images</tt>
* '''[http://code.google.com/p/wikiteam/source/browse/trunk/dumpgenerator.py dumpgenerator.py] to download MediaWiki wikis:''' <tt>python dumpgenerator.py --api=http://archiveteam.org/api.php --xml --images</tt>
* [http://code.google.com/p/wikiteam/source/browse/trunk/wikipediadownloader.py wikipediadownloader.py] to download Wikipedia dumps from download.wikimedia.org: <tt>python wikipediadownloader.py</tt>
* [http://code.google.com/p/wikiteam/source/browse/trunk/wikipediadownloader.py wikipediadownloader.py] to download Wikipedia dumps from download.wikimedia.org: <tt>python wikipediadownloader.py</tt>
* There are many wiki engines, the most famous is MediaWiki. So, the tools must be ready to read data from almost every wiki engine to saved them all.
 
=== Other ===
* [http://dl.dropbox.com/u/63233/Wikitravel/Source%20Code%20and%20tools/Source%20Code%20and%20tools.7z Scripts of a guy who saved Wikitravel]
* [http://dl.dropbox.com/u/63233/Wikitravel/Source%20Code%20and%20tools/Source%20Code%20and%20tools.7z Scripts of a guy who saved Wikitravel]
* [http://www.communitywiki.org/en/BackupThisWiki OddMuseWiki backup]
* [http://www.communitywiki.org/en/BackupThisWiki OddMuseWiki backup]

Revision as of 22:51, 4 December 2011

We save wikis, from Wikipedia to tiniest wikis
130+ wikis saved to date
WikiTeam
WikiTeam, a set of tools for wiki preservation and a repository of wikis
WikiTeam, a set of tools for wiki preservation and a repository of wikis
URL http://code.google.com/p/wikiteam
Status Online!
Archiving status In progress...
Archiving type Unknown
IRC channel #wikiteam (on hackint)

Welcome to WikiTeam. A wiki is a website that allows the creation and editing of any number of interlinked web pages, generally used to store information on a specific subject or subjects. This is done with a day-to-day web browser using a simplified markup language (HTML as an example) or a WYSIWYG (what-you-see-is-what-you-get) text editor.

Examples of huge wikis:

There are also several wikifarms with hundreds of wikis.

Most of the wikis don't offer public backups. How bad!

Tools and source code

Official WikiTeam tools

Other


Wiki dumps

For a more detailed list, visit the download section on Google Code.

There is another site of MediaWiki dumps located here on Scott's Website. More dumps are available as a collection in the Internet Archive.

TODO lists:

Legend
     Good
     Could be better
     Bad
     Unknown
Wiki Wiki is online? Dumps available? (official or home-made) Comments/Details Saved by us? Who? Where?
Anarchopedias Yes Official: no. Home-made: Yes - idiolect
Archive Team Wiki Yes Official: no. Home-made: yes - WikiTeam
Bulbapedia Yes Official: no. Home-made: no - dr-spangle is working on it with a self-built PHP downloader
Citizendium Yes Official: daily (no full history). Home-made: yes, April 2011 No image dumps available -
EditThis Yes Official: no. Home-made: in progress - -
enciclopedia.us.es Yes Official: no. Home-made: no Sysop sent me page text sql tables emijrp
Encyclopedia Dramatica No Official: no. Home-made: partial WebEcology Project Article Dump (~9000 Articles)
Most of the Images probably Lost
-
Encyclopedia Dramatica.ch
(new ED)
Yes Official: ? Home-made: ? Slowly being rebuilt from old sources.
Should be up for a while but for who knows how long?
-
Gentoo wikis Yes Official: no. Home-made: yes - WikiTeam
GNUpedia No Official: no. Home-made: no No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed -
MeatBall Yes Official: no. Home-made: yes (mirror) No histories, no xml format SDBoyd
Metapedia Yes Official: ?. Home-made: no - -
Neoseeker aka Scout wikis Yes Official: ?. Home-made: no - -
Nupedia No Official: ?. Home-made: Yes, saved from IA - -
OmegaWiki Yes Official: daily - -
OpenStreetMap Yes Official: Yes. Home-made: no - -
OpenSUSE wikis Yes Official: no. Home-made: yes - Hydriz
OSDev Yes Official: weekly - Not yet
TV Tropes Yes Official: No Unofficial: In progress No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org" DoubleJ
Uncyclomedias - - - -
Wikanda Yes Official: no. Home-made: yes - emijrp
Wikia Yes Official: on demand No image dumps available Not yet
WikiFur Yes Official: yes No image dumps available Not yet
WikiHow - - - -
Wikimedia Commons Yes Official: periodically No image dumps available Not yet
Wikipedia Yes Official: periodically No image dumps available. English Wikipedia dump uses to be very old Not yet
Wiki-site.com - - - -
WikiTravel Yes Official: not yet. Home-made: yes, another of 2010-06-14 - WikiTeam
WikiWikiWeb Yes Home-made: yes - Ca7
(o:forum Yes No - Not yet, to figure out how
WikiWiki.de Yes No - Not yet, to figure out how
GruenderWiki Yes No - Not yet, to figure out how

Tips

Some tips:

  • When downloading Wikipedia/Wikimedia Commons dumps, pages-meta-history.xml.7z and pages-meta-history.xml.bz2 are the same, but 7z use to be smaller (better compress ratio), so use 7z.

BitTorrent downloads

A feed of BitTorrent downloads is available for the latest files posted to the WikiTeam Google Code Downloads.

Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in µTorrent on Windows).

Mirrors

  1. Sourceforge (also mirrored to another 26 mirrors)
  2. Internet Archive (direct link to directory)

Closing/In danger

Offline wikis and wikifarms

elwiki.com

  • 2011
    • wik.is
  • 2010
  • 2009
  • 2008
    • Scribblewiki (wikifarm)

External links