Talk:ESPN Forums

Backup Notes

Capturing the webpage with the usual tools won't work well, as loading comments requires Javascript. So the best thing to do is to download the back-end JSON (or use PhantomJS, but that's too slow IMO.)

The API URL for the comments/topics is http://api.echoenabled.com/v1/search?q=...&appkey=dev.espn.go.com and the documentation for the '...' part is here

I wrote a small Python script that loops through the topics list, downloads the title/description and comments for each topic, and stores it a file. It's a bit buggy right now though, so I won't post it just yet.

They do have some rather restrictive limits on how many requests "the customer" can make per minute and "The rate limits are per API endpoint and per API key." --Mithrandir 16:15, 4 July 2013 (EDT)

Looks like they've locked the forums, so it could go offline at anytime. That script is still buggy, but I've managed to get >100 forums so far with it. Here they are on Dropbox. It's just tarred and lzipped directories of JSON files (each file being one thread.) They should be pretty complete, maybe 99% up-to-date or so. --Mithrandir 19:56, 20 July 2013 (EDT)

I've uploaded all the files onto archive.org. This should probably be moved to the Just In Time collection, but I'm not sure how to do that. Thoughts? --Mithrandir 04:22, 24 August 2013 (EDT)