Difference between revisions of "IRC Quotes"

From Archiveteam
Jump to navigation Jump to search
(Scraped 13 more QDBs, added info about a QdbS scraper)
(Linkified all the scraped QDBs, sorted alphabetically.)
Line 33: Line 33:
|The quote database that pretty much created all others.
|The quote database that pretty much created all others.
|-
|-
|[http://www.deaddyingdamned.com/qdb/ DeadDyingDamned.com/QDB/]
|[http://bomblol.net/ornot/ BombLol.net/ornot]
|Yes
|Auguste
|Generic QdbS QDB
|-
|[http://www.deaddyingdamned.com/qdb/ DeadDyingDamned.com/qdb/]
|No
|No
|
|
|The unofficial ArchiveTeam QDB.  I'll have the server automatically save these somewhere. --[[User:Auguste|Auguste]] 13:36, 7 April 2011 (UTC)
|The unofficial ArchiveTeam QDB.  I'll have the server automatically save these somewhere. --[[User:Auguste|Auguste]] 13:36, 7 April 2011 (UTC)
|-
|-
|[http://www.i-rox.com I-Rox.com]
|[http://deanyderkheiser.net/qdb/ DeanyDerkheiser.net/qdb]
|Yes
|Yes
|Auguste
|Auguste
|
|Generic QdbS QDB
|-
|-
|[http://www.mandaliet.com/furcqdb/ Mandaliet.com/furcqdb/]
|[http://freqbase.com/qdb/ FreqBase.com/qdb]
|Yes
|Yes
|Auguste
|Auguste
|The Furcadia quote database
|Generic QdbS QDB
|-
|-
|[http://qdb.mit.edu QDB.MIT.edu]
|[http://frostfall-guild.com/ff/qdb/ Frostfall-Guild.com/ff/qdb]
|Yes
|Yes
|Auguste
|Auguste
|The MIT quote database
|Generic QdbS QDB
|-
|[http://german-bash.org german-bash.org]
|Yes ([http://cornelia.regengedanken.de/~mdrueing/files/german-bash.org%20Quote%20Collection%202011-04-08.7z here])
|Darkstar
|German version of bash.org
|-
|-
|[http://www.qdb.us QDB.us]
|[http://www.i-rox.com I-Rox.com]
|Yes
|Yes
|Auguste
|Auguste
|
|
|-
|-
|[http://www.quoteirc.com QuoteIRC.com]
|[http://ibash.de ibash.de]
|Yes
|Yes ([http://cornelia.regengedanken.de/~mdrueing/files/ibash.de%20Quote%20Collection%202011-04-09.7z here])
|Auguste
|Darkstar
|
|Another german quotes DB
|-
|-
|[http://quotes.burntelectrons.org Quotes.BurntElectrons.org]
|[http://jdl.host.hk-diy.net/quote/ JDL.Host.HK-DIY.net/quote]
|Yes
|Yes
|Auguste
|Auguste
|The IRC.Mozilla.org quote database
|Generic QdbS QDB
|-
|-
|[http://www.warpdrive.se WarpDrive.se]
|[http://linuxcult.org/quotes/ LinuxCult.org/quotes]
|Yes
|Yes
|Auguste
|Auguste
|Quotes are in Swedish
|Generic QdbS QDB
|-
|-
|[http://www.wqdb.org WQDB.org]
|[http://www.lolimbanned.com LolImBanned.com]
|Yes
|Yes
|Auguste
|Auguste
|The Worms quote database
|Generic QdbS QDB
|-
|-
|[http://www.xkcdb.com xkcdb.com]
|[http://www.mandaliet.com/furcqdb/ Mandaliet.com/furcqdb/]
|Yes
|Yes
|Auguste
|Auguste
|The xkcd quote database
|The Furcadia quote database
|-
|[http://german-bash.org german-bash.org]
|Yes ([http://cornelia.regengedanken.de/~mdrueing/files/german-bash.org%20Quote%20Collection%202011-04-08.7z here])
|Darkstar
|German version of bash.org
|-
|-
|[http://ibash.de ibash.de]
|[http://moarpupr.com/quotes MoarPupr.com/quotes]
|Yes ([http://cornelia.regengedanken.de/~mdrueing/files/ibash.de%20Quote%20Collection%202011-04-09.7z here])
|Darkstar
|Another german quotes DB
|-
|JDL.Host.HK-DIY.net/Quote
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Generic QdbS QDB
|-
|-
|MoarPupr.com/Quotes
|[http://notsafeforsanity.com/quotes/ NotSafeForSanity.com/quotes]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Generic QdbS QDB
|-
|-
|Frostfall-Guild.com/FF/QDB
|[http://pilkipedia.co.uk/qdb Pilkipedia.co.uk/qdb]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Generic QdbS QDB
|-
|-
|FreqBase.com/QDB
|[http://qdb.honk-honk.org/ QDB.Honk-Honk.org]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Generic QdbS QDB
|-
|-
|LolImBanned.com
|[http://qdb.mit.edu QDB.MIT.edu]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|The MIT quote database
|-
|-
|BombLol.net/OrNot
|[http://qdb.pesterchum.net/ QDB.PesterChum.net]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Generic QdbS QDB
|-
|-
|NotSafeForSanity.com/Quotes
|[http://www.qdb.us QDB.us]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|
|-
|-
|QDB.PesterChum.net
|[http://qdbs.chanops.org/ QDBS.ChanOps.org]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Generic QdbS QDB
|-
|-
|DeanyDerkheiser.net/QDB
|[http://www.quoteirc.com QuoteIRC.com]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|
|-
|-
|Pilkipedia.co.uk/QDB
|[http://quotes.burntelectrons.org Quotes.BurntElectrons.org]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|The IRC.Mozilla.org quote database
|-
|-
|LinuxCult.org/Quotes
|[http://www.warpdrive.se WarpDrive.se]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|Quotes are in Swedish
|-
|-
|QDBS.ChanOps.org
|[http://www.wqdb.org WQDB.org]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|The Worms quote database
|-
|-
|QDB.Honk-Honk.org
|[http://www.xkcdb.com xkcdb.com]
|Yes
|Yes
|Auguste
|Auguste
|Generic QdbS QDB
|The xkcd quote database
|}
|}
[[Category:Archive Team]]
[[Category:Archive Team]]

Revision as of 10:14, 10 April 2011

What's this, then?

Auguste, BlueMax and Dr-Spangle are currently scraping IRC quote databases (e.g. Bash.org). If you can help out or suggest other quote databases to scrape, please join them in #bashup.

Auguste is currently writing a generic Perl script to grab any QdbS-powered QDB (e.g. Auguste's ArchiveTeam QDB). As long as a website is using the default QdbS template and no major modifications, it works flawlessly. He'll release the script publicly once he gets it working with the other common QdbS templates.

Project Hosting

Auguste is currently hosting scrapes here. Everybody is encouraged to help mirror.

Helping Out

Scraping doesn't take a lot of work; the QDBs are all more or less the same. You only need to write one script, then make a few changes to adapt it to any other QDB you want to scrape. The actual scraping process should easily take under 10 minutes.

If you do want to help with the scraping, please follow the existing scrape format:

  • Each quote has its own file
  • Each file is named 'n.txt', where 'n' is the quote's ID number
  • All quotes should be compressed into an archive
  • The archive name should identify the original location (URL) and date of scraping (e.g. 'QuoteIRC.com Quote Collection 2011-04-04.7z', or 'DOMAIN.TLD Quote Collection YYYY-MM-DD.EXT').
    • If the original location (URL) has subdirectories (e.g. 'Foobar.com/baz'), replace forward slashes with hyphens: 'Foobar.com-baz'.

Tips

  • Scrape from the browse page (e.g. http://bash.org/?browse). This way you can scrape 10-50 quotes per page request, rather than cycling through thousands of individual quote pages.

Project Status

Database Has been scraped Scraper Notes
Bash.org Yes Dr-Spangle The quote database that pretty much created all others.
BombLol.net/ornot Yes Auguste Generic QdbS QDB
DeadDyingDamned.com/qdb/ No The unofficial ArchiveTeam QDB. I'll have the server automatically save these somewhere. --Auguste 13:36, 7 April 2011 (UTC)
DeanyDerkheiser.net/qdb Yes Auguste Generic QdbS QDB
FreqBase.com/qdb Yes Auguste Generic QdbS QDB
Frostfall-Guild.com/ff/qdb Yes Auguste Generic QdbS QDB
german-bash.org Yes (here) Darkstar German version of bash.org
I-Rox.com Yes Auguste
ibash.de Yes (here) Darkstar Another german quotes DB
JDL.Host.HK-DIY.net/quote Yes Auguste Generic QdbS QDB
LinuxCult.org/quotes Yes Auguste Generic QdbS QDB
LolImBanned.com Yes Auguste Generic QdbS QDB
Mandaliet.com/furcqdb/ Yes Auguste The Furcadia quote database
MoarPupr.com/quotes Yes Auguste Generic QdbS QDB
NotSafeForSanity.com/quotes Yes Auguste Generic QdbS QDB
Pilkipedia.co.uk/qdb Yes Auguste Generic QdbS QDB
QDB.Honk-Honk.org Yes Auguste Generic QdbS QDB
QDB.MIT.edu Yes Auguste The MIT quote database
QDB.PesterChum.net Yes Auguste Generic QdbS QDB
QDB.us Yes Auguste
QDBS.ChanOps.org Yes Auguste Generic QdbS QDB
QuoteIRC.com Yes Auguste
Quotes.BurntElectrons.org Yes Auguste The IRC.Mozilla.org quote database
WarpDrive.se Yes Auguste Quotes are in Swedish
WQDB.org Yes Auguste The Worms quote database
xkcdb.com Yes Auguste The xkcd quote database