Difference between revisions of "Usenet"

From Archiveteam
Jump to navigation Jump to search
(Draft)
 
m (→‎top: typos fixed: from it's → from its, supressed → suppressed, propriatery → proprietary)
 
(9 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 +
{{Infobox project
 +
| title = Usenet
 +
| URL = none
 +
| project_status = {{online}}
 +
| archiving_status = {{inprogress}}
 +
| irc = archiveteam
 +
}}
  
USENET is a mailing list based collection of assorted forum groups accessed via the NNTP protocol.
+
'''Usenet''' is a mailing list based collection of assorted forum groups accessed via the NNTP protocol.
  
Currently the major archive of this important forum is Google Groups...
+
Currently the major archive of this important forum is Google Groups, which absorbed DejaNews. However,  there are some concerns raised by this.
 +
#  Google could pull Groups at its whim, with no clear donation policy to other archives.
 +
#  Despite Google's claims not to routinely monitor the service,  postings are removed or suppressed for various reasons.
 +
#  Google currently offers little to distinguish USENET from its own proprietary groups.
 +
#  The interface used for Groups has issues with some browsers, and accessing text versions of postings is an involved process.
  
However, there are some concerns raised by this.
+
[[Google Video]] proves that Google might be the longest-serving goldmine of the internet, but that doesn't make it a reliable long-term host. Copies must be retained by others in safe locations.
  
1. Google could pull Groups at it's whim, with no clear donation policy to other archives.
+
Usenet was and is a distributed system, hence no single company or server will have the whole history of it. Some big players, however, have a good portion. Moreover, everyone with access to a news server should just download everything (non-binary) there is on it and publish it on archive.org; especially the local hierarchies not mirrored in many places. To do this, it's probably enough to put a free software newsreader at work, which saves on a standard open format? Suggestions needed.
2. Despite Google's claims not to routinely monitor the service, postings are removed or supressed for various reasons.
 
3.  Google currently offers little to distinguish USENET from it's own propriatery groups.
 
4.  The interface used for Groups has issues with some browsers, and accessing text versions of postings is an involved process.
 
  
 +
In the meanwhile, Kahle and the Internet Archive are not resting!
 +
* https://archive.org/details/usenet
 +
* https://archive.org/details/giganews (since January 2014)
 +
It's hard to say how complete those archives will be.
  
Therefore there should be an alternative.
+
Tools for sorting USENET and public mailing list archives into big Katamari style archives are available at https://github.com/ZoeB/arcmesg  Improvements are greatly appreciated!
  
'''It is suggested that Archiveteam members form an effort to begin a parallel archive to groups'''
+
{{Navigation box}}
 
 
Such an alternative should offer a credible search facility, indexing by header fields and over date ranges, broadly similar to those offered by the Groups UI. Such a search could also extend beyond that offered by the current Google offering in enabling grep style expressions to be used ( subject to appropriate limitations on resource uages)
 
 
 
Technical issues :
 
* How big is the current USENET colloquia?
 
* By how much is this likely to grow?
 
* How should postings be stored?  ( Ideally text postings should be stored as plain text+headers as they would typically be on a newserver)
 
* Should NNTP style direct access be allowed, or should posting only be accessible via a neogtiated read only API?
 
* Binaries -  Leaves as encoded or translate?
 
 
 
Logistic issues:
 
* How to recover pre 2014 material from alternative sources?
 
* How to upload and index?
 
 
 
 
 
Non-Technical Issues:
 
* Spam - Some less used groups are in effect mostly spam.. is it worth acrchiving the spam along with genuine postings?
 
* Cancelmsg -  Google Groups doesn't respond to them, but some newservers will respond to genuine cancelmsg, as well as issuing their own in respect of material found to be in breach of applicable laws.
 
 
 
* Impersonation of headers-  Mis attribution of sources is an issue because of the potential for legal consquences.
 
* Legally questionable material -  Should an archive of USENET respect archival principles (and challange legal threats) or
 
have takedown procedure?
 
* The New York 22 Banned list -  No responsible archive would support the deliberate inclusion of clearly illegal 'child abuse' images but these are not always easy to identify such, and should an archive be the one to report previously unknown crimes?
 
* Libel(i.e Defamation) - In some countries the 'publisher' of a libel (ie an archive) can be held liable for it as well as the original source. Some postings which would be libel are nonetheless retained in the archive as they form part of the public debate. (This is especailly true of high-profile cases). However , libel of course has to be proven in court.
 
 
 
 
 
*Infringement of copyright -  Whilst the DMCA has a takedown procedure, it's sometimes overreaching, meaning materials posted in good faith are removed unfairly.  Precusors to the DMCA takewdown have also been used for SLAPP purposes and to supress
 

Latest revision as of 23:23, 4 December 2017

Usenet
Usenet logo
Employee captured tearing page.png
URL none
Project status Online!
Archiving status In progress...
Project source Unknown
Project tracker Unknown
IRC channel #archiveteam (on EFnet)
Project lead Unknown

Usenet is a mailing list based collection of assorted forum groups accessed via the NNTP protocol.

Currently the major archive of this important forum is Google Groups, which absorbed DejaNews. However, there are some concerns raised by this.

  1. Google could pull Groups at its whim, with no clear donation policy to other archives.
  2. Despite Google's claims not to routinely monitor the service, postings are removed or suppressed for various reasons.
  3. Google currently offers little to distinguish USENET from its own proprietary groups.
  4. The interface used for Groups has issues with some browsers, and accessing text versions of postings is an involved process.

Google Video proves that Google might be the longest-serving goldmine of the internet, but that doesn't make it a reliable long-term host. Copies must be retained by others in safe locations.

Usenet was and is a distributed system, hence no single company or server will have the whole history of it. Some big players, however, have a good portion. Moreover, everyone with access to a news server should just download everything (non-binary) there is on it and publish it on archive.org; especially the local hierarchies not mirrored in many places. To do this, it's probably enough to put a free software newsreader at work, which saves on a standard open format? Suggestions needed.

In the meanwhile, Kahle and the Internet Archive are not resting!

It's hard to say how complete those archives will be.

Tools for sorting USENET and public mailing list archives into big Katamari style archives are available at https://github.com/ZoeB/arcmesg Improvements are greatly appreciated!


v · t · e         Archive Team
Current events

Alive... OR ARE THEY · Deathwatch · Projects

Archiveteam.jpg
Archiving projects

APKMirror · Archive.is · BetaArchive · Government Backup (#datarefuge · ftp-gov· Gmane · Internet Archive · It Died · Megalodon.jp · OldApps.com · OldVersion.com · OSBetaArchive · TEXTFILES.COM · The Dead, the Dying & The Damned · The Mail Archive · UK Web Archive · WebCite · Vaporwave.me

Blogging

Blog.pl · Blogger · Blogster · Blogter.hu · Freeblog.hu · Fuelmyblog · Jux · LiveJournal · My Opera · Nolblog.hu · Open Diary · ownlog.com · Posterous · Powerblogs · Proust · Roon · Splinder · Tumblr · Vox · Weblog.nl · Windows Live Spaces · Wordpress.com · Xanga · Yahoo! Blog · Zapd

Cloud hosting/file sharing

aDrive · AnyHub · Box · Dropbox · Docstoc · Fast.io · Google Drive · Google Groups Files · iCloud · Fileplanet · LayerVault · MediaCrush · MediaFire · Mega · MegaUpload · MobileMe · OneDrive · Pomf.se · RapidShare · Ubuntu One · Yahoo! Briefcase

Corporations

Apple · IBM · Google · Loblaw · Lycos Europe · Microsoft · Yahoo!

Events

Arab Spring · Great Ape-Snake War · Spanish Revolution

Font Repos

DaFont · Google Web Fonts · GNU FreeFont · Fontspace

Forums/Message boards

4chan · Captain Luffy Forums · College Confidential · DSLReports · ESPN Forums · Facepunch Forums · forums.starwars.com · HeavenGames · JamiiForums · Invisionfree · NeoGAF · Textream · The Classic Horror Film Board · Yahoo! Messages · Yahoo! Neighbors · Yuku.com · Zetaboards

Gaming

Atomicgamer · Bazaar.tf · City of Heroes · Club Nintendo · Clutch · Counter-Strike: Global Offensive · CS:GO Lounge · Desura · Dota 2 · Dota 2 Lounge · Emulation Zone · ESEA · GameBanana · GameMaker Sandbox · GameTrailers · Halo · HLTV.org · HQ Trivia · Infinite Crisis · joinDOTA · League of Legends · Liquipedia · Minecraft.net · Player.me · Playfire · Raptr · SingStar · Steam · SteamDB · SteamGridDB · Team Fortress 2 · TF2 Outpost · Warhammer · Xfire

Image hosting

500px · AOL Pictures · Blipfoto · Blingee · Canv.as · Camera+ · Cameroid · DailyBooth · Degree Confluence Project · DeviantART · Demotivalo.net · Flickr · Fotoalbum.hu · Fotolog.com · Fotopedia · Frontback · Geograph Britain and Ireland · Giphy · GTF Képhost · ImageShack · Imgh.us · Imgur · Inkblazers · Instagram · Kepfeltoltes.hu · Kephost.com · Kephost.hu · Kepkezelo.com · Keptarad.hu · Madden GIFERATOR · MLKSHK · Microsoft Clip Art · Microsoft Photosynth · Nokia Memories · noob.hu · Odysee · Panoramio · Photobucket · Picasa · Picplz · Pixiv · Portalgraphics.net · PSharing · Ptch · puu.sh · Rawporter · Relay.im · ScreenshotsDatabase.com · Sketch · Smack Jeeves · Snapjoy · Streetfiles · Tabblo · Tinypic · Trovebox · TwitPic · Wallbase · Wallhaven · Webshots · Wikimedia Commons

Knowledge/Wikis

arXiv · Citizendium · Clipboard.com · Deletionpedia · EditThis · Encyclopedia Dramatica · Etherpad · Everything2 · infoAnarchy · GeoNames · GNUPedia · Google Books (Google Books Ngram· Horror Movie Database · Insurgency Wiki · Knol · Lost Media Wiki · Neoseeker.com · Notepad.cc · Nupedia · OpenCourseWare · OpenStreetMap · Orain · Pastebin · Patch.com · Project Gutenberg · Puella Magi · Referata · Resedagboken · SongMeanings · ShoutWiki · The Internet Movie Database · TropicalWikis · Uncyclopedia · Urban Dictionary · Urban Exploration Resource · Webmonkey · Wikia · Wikidot · WikiHow · Wikkii · WikiLeaks · Wikipedia (Simple English Wikipedia· Wikispaces · Wikispot · Wik.is · Wiki-Site · WikiTravel · Word Count Journal

Magazines/Blogs/News

Cyberpunkreview.com · Game Developer Magazine · Gigaom · Hardware Canucks · Helium · JPG Magazine · Make Magazine · The Escapist · Polygamia.pl · San Fransisco Bay Guardian · Scoop · Regretsy · Yahoo! Voices

Microblogging

Heello · Identi.ca · Jaiku · Mommo.hu · Plurk · Sina Weibo · Tencent Weibo · Twitter · TwitLonger

Music/Audio

8tracks · AOL Music · Audimated.com · Cinch · digCCmixter · Dogmazic.net · Earbits · exfm · Free Music Archive · Gogoyoko · Indaba Music · Instacast · Instaudio · Jamendo · Last.fm · Music Unlimited · MOG · PureVolume · Reverbnation · ShareTheMusic · SoundCloud · Soundpedia · Spotify · This Is My Jam · TuneWiki · Twaud.io · WinAmp

People

Aaron Swartz · Michael S. Hart · Steve Jobs · Mark Pilgrim · Dennis Ritchie · Len Sassaman Project

Protocols/Infrastructure

FTP · Gopher · IRC · Usenet · World Wide Web
BitTorrent DHT

Q&A

Askville · Answerbag · Answers.com · Ask.com · Askalo · Baidu Knows · Blurtit · ChaCha · Experts Exchange · Formspring · GirlsAskGuys · Google Answers · Google Baraza · JustAnswer · MetaFilter · Quora · Retrospring · StackExchange · The AnswerBank · The Internet Oracle · Uclue · WikiAnswers · Yahoo! Answers

Recipes/Food

Allrecipes · Epicurious · Food.com · Foodily · Food Network · Punchfork · ZipList

Social bookmarking

Addinto · Backflip · Balatarin · BibSonomy · Bkmrx · Blinklist · BlogMarks · BookmarkSync · CiteULike · Connotea · Delicious · Designer News · Digg · Diigo · Dir.eccion.es · Evernote · Excite Bookmark · Faves · Favilous · folkd · Freelish · Getboo · GiveALink.org · Gnolia · Google Bookmarks · Hacker News · HeyStaks · IndianPad · Kippt · Knowledge Plaza · Licorize · Linkwad · Menéame · Microsoft Developer Network · myVIP · Mister Wong · My Web · Mylink Vault · Newsvine · Oneview · Pearltrees · Pinboard · Pocket · Propeller.com · Reddit · sabros.us · Scloog · Scuttle · Simpy · SiteBar · Slashdot · Squidoo · StumbleUpon · Twine · Voat · Vizited · Yummymarks · Xmarks · Yahoo! Buzz · Zootool · Zotero

Social networks

Bebo · BlackPlanet · Classmates.com · Cyworld · Dogster · Dopplr · douban · Ello · Facebook · Flixster · FriendFeed · Friendster · Friends Reunited · Gaia Online · Google+ · Habbo · hi5 · Hyves · iWiW · LinkedIn · Miiverse · mixi · MyHeritage · MyLife · Myspace · myVIP · Netlog · Odnoklassniki · Orkut · Plaxo · Qzone · Renren · Skyrock · Sonico.com · Storylane · Tagged · tvtag · Upcoming · Viadeo · Vine · Vkontakte · WeeWorld · Weibo · Wretch · Yahoo! Groups · Yahoo! Stars India · Yahoo! Upcoming · more sites...

Shopping/Retail

Alibaba · AliExpress · Amazon · Apple Store · Barnes & Noble · DirectCanada · eBay · Kmart · NCIX · Printfection · RadioShack · Sears · Sears Canada · Target · The Book Depository · ThinkGeek · Toys "R" Us · Walmart

Software/code hosting

Android Development · Alioth · Assembla · BerliOS · Betavine · Bitbucket · BountySource · Codecademy · CodePlex · Freepository · Free Software Foundation · GNU Savannah · GitHost  · GitHub · GitHub Downloads · Gitorious · Gna! · Google Code · ibiblio · java.net · JavaForge · KnowledgeForge · Launchpad · LuaForge · Maemo · mozdev · OSOR.eu · OW2 Consortium · Openmoko · OpenSolaris · Ourproject.org · Ovi Store · Project Kenai · RubyForge · SEUL.org · SourceForge · Stypi · TestFlight · tigris.org · Transifex · TuxFamily · Yahoo! Downloads

Television/Radio

ABC · Austin City Limits · BBC · CBC · CBS · Computer Chronicles · CTV · Fox · G4 · Global TV · Jeopardy! · NBC · NHK · PBS · Penn & Teller: Bullshit! · The Howard Stern Show · TV News Archive (Understanding 9/11)

Torrenting/Piracy

ExtraTorrent · EZTV · isoHunt · KickassTorrents · The Pirate Bay · Torrentz · Library Genesis

Video hosting

Academic Earth · Bambuser · Blip.tv · Epic · Freshlive · Google Video · Justin.tv · Mixer · Niconico · Nokia Trailers · Oddshot.tv · Periscope · Plays.tv · Qwiki · Skillfeed · Stickam · TED Talks · Ticker.tv · Twitch.tv · Ustream · Videoplayer.hu · Viddler · Viddy · Vidme · Vimeo · Vine · Vstreamers · Yahoo! Video · YouTube · Famous Internet videos (Me at the zoo)

Web hosting

Angelfire · Brace.io · BT Internet · CableAmerica Personal Web Space · Claranet Netherlands Personal Web Pages · Comcast Personal Web Pages · Extra.hu · FortuneCity · Free ProHosting · GeoCities (patch· Google Business Sitebuilder · Google Sites · Internet Centrum · MBinternet · MSN TV · Nifty · Nwnyet · Parodius Networking · Prodigy.net · Saunalahti Iso G · Swipnet · Telenor · Tripod · University of Michigan personal webpages · Verizon Mysite · Verizon Personal Web Space · Webs · Webzdarma · Virgin Media

Web applications

Mailman · MediaWiki · phpBB · Simple Machines Forum · vBulletin

Information

A Million Ways to Die on the Web · Backup Tips · Cheap storage · Collecting items randomly · Data compression algorithms and tools · Dev · Discovery Data · DOS Floppies · Fortress of Solitude · Keywords · Naughty List · Nightmare Projects · Rescuing floppy disks · Rescuing optical media · Site exploration · The WARC Ecosystem · Working with ARCHIVE.ORG

Projects

ArchiveCorps · Audit2014 · Emularity · Faceoff · FlickrFckr · Froogle · INTERNETARCHIVE.BAK (Internet Archive Census· IRC Quotes · JSMESS · JSVLC · Just Solve the Problem · NewsGrabber · Project Newsletter · Valhalla · Web Roasting (ISP Hosting · University Web Hosting· Woohoo

Tools

ArchiveBot · ArchiveTeam Warrior (Tracker· Google Takeout · HTTrack · Video downloaders · Wget (Lua · WARC)

Teams

Bibliotheca Anonoma · LibreTeam · URLTeam · Yahoo Video Warroom · WikiTeam

Other

800notes · AOL · Akoha · Ancestry.com · April Fools' Day · Amplicate · AutoAdmit · Bre.ad · Circavie · Cobook · Co.mments · Countdown · Discourse · Distill · Dmoz · Easel · Eircode · Electronic Frontier Foundation · FanFiction.Net · Feedly · Ficlets · Forrst · FunnyExam.com · FurAffinity · Google Helpouts · Google Moderator · Google Poly · Google Reader · ICQmail · IFTTT · Jajah · JuniorNet · Lulu Poetry · Mobile Phone Applications · Mochi Media · Mozilla Firefox · MyBlogLog · NBII · Newgrounds · Neopets · Quantcast · Quizilla · Salon Table Talk · Shutdownify · Slidecast · Stack Overflow · SOPA blackout pages · starwars.yahoo.com · TechNet · Toshiba Support · USA-Gov · Volán · Widgetbox · Windows Technical Preview · Wunderlist · YTMND · Zoocasa

About Archive Team

Introduction · Philosophy · Who We Are · Our stance on robots.txt · Why Back Up? · Software · Formats · Storage Media · Recommended Reading · Films and documentaries about archiving · Talks · In The Media · FAQ