Difference between revisions of "AOL"
(Getting started with AOL)
(→Tokens: fix broken link)
|Line 71:||Line 71:|
== Downloading a file ==
== Downloading a file ==
Revision as of 03:01, 8 January 2014
|Status||Online! on January 28, 2013|
|IRC channel||(on hackint)|
This is about archiving the original AOL, not AOL's current website. The AOL system is currently in major disrepair. It is as if they left the machines sitting in the datacenter, and as they die, they do not fix any issues. There is much broken infrastructure.
You'll need to sign up for a user account here: https://new.aol.com. Not every field is required- phone definitely isn't. Make sure there are no special chars in the username or password, and the password is short (8 chars or less).
If you're running the software through Wine, your choices are very simple: AOL 4.0 or AOL 5.0. Everything else is in various states of not-workingness.
If you're not running through Wine, you can use any version you want, but there's good reasons for not using AOL 6 or higher:
- Fairly rich legacy of third-party tools up to 5.0
- Documentation is probably applicable to every later version, but there's not as many or any ways to make good use of it in later versions
Why you might want to use a later version:
- More things on the site will certainly work (there may be less stuff accessible, but more of it should work because it's actively supported by AOL)
AOL has copies of the 9.x series on their site, and Oldversion has copies of all the other versions.
In general (these instructions assume you create the account beforehand on AOL's site):
Run aol50.exe (or your chosen version's installer) Choose Current member, and then add my account to this computer. When the installer finishes, and AOL launches, it will bring up the AOL setup screen. You'll want Expert Setup, so you can tell AOL right away that you want to use your TCP/IP connection to connect to AOL (instead of dial-up). Once you select TCP/IP connection, AOL asks if you want to sign on right away- unless you have proxy settings or can't use the defaults, just hit next. If you need to change any of the settings before connecting, read the message box carefully- it explains where to go to change the settings. You want to pick You already have a screen name and password - fill in the account name and password you created earlier on AOL's site. Press Next, and you are done.
Biggest difference is that you can't create an account inside of AOL 5 or less, so you must create the account beforehand.
The trunk version of Wireshark includes a dissassembler for the AOL protocol that breaks out the basic header information, such as the packet type and the token. It doesn't go into any detail about the contents of the packet, but this is a good start. This isn't available for download yet, so you'll have to build it yourself, from the svn trunk; once built wireshark reports itself as 1.9.0.
- INIT (x’23’)
- Client sends this to the server to begin comminucation.
- ACK (x’24’)
- Acknowledge a packet as recieved, for instance an INIT or heartbeat.
- SS (x’21’)
- An SS requests the other end of the connection to send an SSR.
- SSR (x’22’)
- An SSR is a response to an SS.
- NAK (x’25’)
- Negative acknowlegement of a packet, when the packet was recieved incorrectly.
- DATA (x’20’)
- A packet containing data, identified by a token.
- HEARTBEAT (x’26’)
- The other side suspects that the line has dropped; respond with an ACK
Each packet has a token that determines what is in the data field of the packet. Documentation for these tokens is very sparse; it's likely that AOL never had a comprehensive document listing all of them. Instead, the documentation merely tells the reader to view the list of tokens while logged into the server.
Downloading a file
- ← mD – client requests a file (by id?)
- → uJ – unicode file name
- → tf – start of a download; includes file name (non-unicode?); requests immediate xG ack
- ← xG – client acks download
- → FF – packet containing file data, no ack requested
- → F7 – packet containing file data, no ack requested
- ← xG – periodic acks
- → F9 – packet containing file data, last in sequence
- ← eX – mail download complete (unrelated?)
[21:24:10] <db48x> there's a packet coming from the server with a token tf [21:24:16] <db48x> the data has a filename in it [21:24:59] <db48x> the data is in a series of packets with token FF and F7 (no explanation of the difference is available) [21:25:24] <balrog_> but like when you view a file library [21:25:34] <balrog_> how does it tell the server which library to display? [21:25:36] <db48x> the last packet of the file has token F9 21:25:42] <db48x> haven't figured that out yet [21:25:56] <balrog_> ah [21:26:01] <db48x> before this file in the capture there are packets with tokens EB and uJ going from the client to the server [21:26:03] <balrog_> none of the documentation covers this? [21:26:09] <balrog_> aaah [21:26:44] <db48x> and mD [21:26:51] <db48x> and tokens AT and tD coming back [21:29:29] <db48x> looks like the tD coming back has the metadata in it [21:30:50] <balrog_> http://sicexcels.tripod.com/~SicExcels/rm-vpd_info/TokenTypes_Basic.txt [21:31:12] <balrog_> http://sicexcels.tripod.com/~SicExcels/rm-vpd_info/TokenTypes_Plus.txt [21:31:16] <balrog_> quite incomplete [21:33:26] <db48x> mD = download now, then [21:34:31] <db48x> and an mF, file description [21:34:41] <db48x> followed by an AT with a bunch of data [21:35:35] <db48x> looks like labels for buttons like 'download now', 'download later', 'ask the staff', 'related files' [21:35:56] <db48x> packet 538 [21:37:19] <db48x> continues in the next AT packet, 540, which looks like it has the description in it [21:37:29] <db48x> talks about using ShrinkIt to unpack the file
In normal transmission, messages are being passed in both directions. Each message sent carries the number of the last message correctly received, which is an implicit acknowledgement of all messages up to and including that one. When a message is received correctly, it is passed up to the application level. Then the response number of the message is examined. If it acknowledges any messages currently in the buffer, they are dropped from the buffer. If the receiver of the message has received a certain number of messages without acknowledging, it will send an ACK to keep the sender’s window from closing. (A window is closed when the buffer of sent messages is full, preventing any more transmissions.)
If a single message gets mangled, the receiver will get a bad checksum, and send a NAK (assuming its window is open) requesting re-transmission of all messages starting at the mangled one. It will then ignore out of sequence messages until it gets the mangled message correctly. If its window is closed, and there is no NAK queued, it will queue the NAK for transmission when the window opens. If there is a NAK queued already, it will ignore the new one.
The same NAK logic would apply to messages received out of sequence, assuming that a NAK had not already been sent.
In all cases, where a numbered message is sent, the window is checked. If it is closed, an SS is sent to try to re-open the wondow. When an SS has been sent, and no SSR has been received, all NAKs are accepted, but they are ignored, instead of being processed.
When a SSR is received, any messages that were not received are queued for transmission. When there is a message to send, and the window is open, it is sent and put into buffer. If the window is closed, the message is queued for transmission. This is separate from NAK queue.
- 1722: Keywords
- 2719: Chatrooms
- 3548: User profiles
- 4344: Interactive page
- 4400: File libraries
- 4401: Files
- 586x: ???
- aol://4344:226.llll.2755674.520114429 (Access code: 3675)
Lots of sources: Documents covering how to make AOL forms and various such things:
- aol://4344:1087.navbar.8505088.517972198 Navigation bar specifications- AOL resource with some forms and examples
Samples of custom forms:
- http://www.mattmazur.com/archive/aol-files/downloads/tools/win/star/index.html - Open up the links under More Info
Some FDO lessons:
About the class names:
Here is an early version of aol-files.com:
<balrog_> yes, but aol://4344:nnnn doesn't work without the extra [19:52] <balrog_> aol://4344:1264.a2main.10029531 also works <balrog_> simply aol://4344:1264.a2main does not work.
[20:17] <DrainLbry> so to summarize we've got aol://4400:ID (from spreadsheet), for file libraries, and aol://4344:uniqueidentifier for interactive content [20:18] <balrog_> aol://4344:uniqueidentifier:ID <balrog_> as per http://web.archive.org/web/20060207004722/http://daol.aol.com/aolatoz keywords used to be aol://1722:keyword <balrog_> but that's no longer working
- aol://4344:1344.bpsfront.10164598.526964474 BPS SoftWare - addons for AOL
AOL has a large number of forums on every topic, file libraries containing art, shareware, game mods, etc, etc. These should be fairly easy to enumerate, and once found it should be fairly easy to download all of the forum messages and files. Archives of these would be worth saving.
Every window that you can click on in AOL was created by a 'producer' at AOL. Many of them are essentially identical, file libraries for instance, but many are also one-offs created for a specific purpose. We ought to save these as well. Going this route will take a more thorough understanding of both the AOL protocol and the FDO scripts.
There are several ways to go about this, with tradeoffs that are only lightly explored.
Write a scraper in python that understands the AOL protocol and FDO scripts, and writes everything to warc files. Warc save us much of the trouble of figuring out how to organize everything on disk. they also make it much easier to create a server than can pretend to be the AOL server, or that can translate into http/html to allow anyone with a web browser to see what AOL was like.
Modify wget to support the AOL protocol. Very ambitious, but it would let us reuse wget's infrastructure, which may make the task easier; we'd be able to concentrate on just implementing the protocol and FDO parsing and leave the rest to wget. Would that reuse save us time, or would dealing with wget's internals drive us mad? Hard to say. This method would also allow us to create warc files.
script the client
Drive the real AOL client, perhaps with debugging tools installed, in order to capture both the FDO sources and screenshots of the rendering. Probably more fragile, but we wouldn't have to understand the actual protocol. Wouldn't be able to create warc files.