Difference between revisions of "Forums.starwars.com"

From Archiveteam
Jump to navigation Jump to search
(→‎Forums.StarWars.Com Shutdown: Added #archivewars info)
Line 21: Line 21:
</blockquote>
</blockquote>


== ArchiveWars on IRC ==
Please use channel [irc://irc.efnet.org/archivewars #archivewars ]  for project coordination.  ([http://efnet.org EFNet] network.)
== Preliminary Project Scope ==


{| class="wikitable"
{| class="wikitable"
Line 50: Line 57:
|colspan="3"|Note: these are the main categories found from a quick scrape.<br />Possible repetition : File type 'messages' might be the same as 'thread message'
|colspan="3"|Note: these are the main categories found from a quick scrape.<br />Possible repetition : File type 'messages' might be the same as 'thread message'
|}
|}


== '''Profile''' Range Signup Sheet ==
== '''Profile''' Range Signup Sheet ==

Revision as of 04:33, 11 May 2011

FORUMS.STARWARS.COM
ForumsStarWarsCom.jpg
URL http://forums.starwars.com
Status Closing on 2011-06-03 announcement tf.n report
Archiving status Not saved yet
Archiving type Unknown
IRC channel #archiveteam-bs (on hackint)

Forums.StarWars.Com Shutdown

StarWars.Com announced the closure of their forums on 03 June 2011. (Forum will lock on 3rd May 2011) tf.n report

"The StarWars.com forums have been online since October 2001 and have featured conversations with various Star Wars VIPs. Lucas Licensing's Sue Rostoni had an ongoing dialog with Del Rey customers where she responded to fan questions and concerns. The forums received a facelift in July 2010 that gave users several new features."

Closure Update :

"Update: Due to a forums outage over the weekend of April 22-24th, we've extended the time before the forums will be locked into read-only mode. The new date for that is Tuesday, May 3rd."


ArchiveWars on IRC

Please use channel #archivewars for project coordination. (EFNet network.)


Preliminary Project Scope

Main forums.starwars.com Page Types and Preliminary Counts
Page Type File Name Preliminary Count Estimate
Announcements : ann.jspa?annID=# Uncertain (probably under 10)
Category : category.jspa?categoryID=#
category.jspa?categoryID=#&start=##
(## = 0, 15, 30, 45, etc.)
1 thru 20 each with multiple start values
Forum : forum.jspa?forumID=#
forum.jspa?forumID=#&start=##
1 thru ~193 (don't seem to be sequential)
3,500 estimate (One page for every 15 threads)
[Highly used forumID=61 has at least 1102 pages start=16515]
Messages : message.jspa?messageID=# ~2 million : according to stats on main forum page
[quick scrap found a high value of 17965717]
Profiles : profile.jspa?userID=# ???? : quick scrape found a high value of 9782310
[Seem to be sequential. Earlier # have earlier creation date]
[Random numbers do find blank error-500.jsp pages]
RSS : rss.jspa?feed=rss%2Frssmessages.jspa?forumID=# [Please confirm if these are scrape worthy]
Tag : tag.jspa?tagName=__NAME__ ???? : Quantity Unknown
[Main Star Wars terms each have their own __NAME__ tag]
Thread Message : thread.jspa?messageID=# ???? : quick scrape found a high value of 17966647
[Similar to 'message'. Maybe redundant.]
Thread Thread : thread.jspa?threadID=# 50,574 according to stats of main forum page
???? : quick scrape found a high value of 275287
Other : Folder 'dwf', 'resources' & 'scripts' have JavaScript (.js)
Folders 'images' & 'share' have .gifs
File types 'index' and a few other misc. types
Note: these are the main categories found from a quick scrape.
Possible repetition : File type 'messages' might be the same as 'thread message'

Profile Range Signup Sheet

We're going to break up the Profile ids into ranges and let individuals claim a range to download. Use this table to mark your territory:

Start End Status Size (Uncompressed) Claimant
0000001 0009999 Downloaded 102.3MB none295
0010000 0019999 Downloaded 24.8MB none295
0020000 0099999 Downloaded 412.9MB none295
0100000 0199999 Downloaded 939.2MB none295
0200000 0299999 Downloaded 1.01GB none295
0300000 0399999 Downloaded 1.09GB none295
0400000 0499999 Downloaded __MB underscor & none295
0500000 0599999 Pool
0600000 0699999 Pool
0700000 0799999 Pool
0800000 0899999 Pool
0900000 0999999 Pool
1000000 1999999 Claimed none295
2000000 2099999 Pool
2100000 2199999 Pool
2200000 2299999 Pool
2300000 2399999 Pool
2400000 2499999 Pool
2500000 2599999 Pool
2600000 2699999 Pool
2700000 2799999 Pool
2800000 2899999 Pool
2900000 2999999 Pool
3000000 9999999 Pool Please split as required.

Please try and claim 100,000 id blocks at this time, or more if your system has adequate space.

Example Profile List Generator:

perl -le 'print "http://forums.starwars.com/profile.jspa?userID=$_" for 2000000..2099999' > forums.starwars.com-profile_2000000-2099999