Bintray

From Archiveteam
Revision as of 04:09, 5 June 2021 by OrIdow6 (talk | contribs) (Remarks that may be useful for learning in the future)
Jump to navigation Jump to search
Bintray
Bintray logo
Employee captured tearing page.png
URL https://bintray.com/
Project status Closing
Archiving status In progress...
Project source bintray-grab
Project tracker bintray
IRC channel #binnedtray (on hackint)
Project lead Unknown

Bintray is a software distribution platform by JFrog. It closed on 2021-05-01 but is still serving (some) content for the time being.

Account of grab

OrIdow6 was the main author of the Bintray grab script, and procrastinated in the process of writing it. This led to less data being captured than might have otherwise happened. Items were at first users, which included normal user pages, all their repositories etc., the index of the downloads for that user, and non-CDN hosted download files for that user; and CDN-hosted download files.

When the site shut down, normal user pages/metadata stayed up, and some downloads stayed up (perhaps it was cached); so download listings and files were split off into their own items, using a confusing hack to effectively requeue failed downloads through backfeed. This enabled downloads to cycle around until they hit a machine that was able to grab them. After a few days, these too stopped being served, and only some official/paid downloads remain(ed).

Things to avoid in the future:

  • Waiting too long to finish the grab script.
  • Putting huge chunks of the download section, which had a very simple structure, in single items, instead of splitting it up.