Difference between revisions of "Talk:ESPN Forums"
Mithrandir (talk | contribs) (added some info about backup) |
(No difference)
|
Revision as of 20:15, 4 July 2013
Backup Notes
Capturing the webpage with the usual tools won't work well, as loading comments requires Javascript. So the best thing to do is to download the back-end JSON (or use PhantomJS, but that's too slow IMO.)
The API URL for the comments/topics is
http://api.echoenabled.com/v1/search?q=...&appkey=dev.espn.go.com
and the documentation for the '...' part is here
I wrote a small Python script that loops through the topics list, downloads the title/description and comments for each topic, and stores it a file. It's a bit buggy right now though, so I won't post it just yet.
They do have some rather restrictive limits on how many requests "the customer" can make per minute and "The rate limits are per API endpoint and per API key." --Mithrandir 16:15, 4 July 2013 (EDT)