Talk:Glitch

CDN archiving and current project status

The state of archiving is still ongoing, as of 2025-08-07 but the progress is more of stop-and-go, with certain assets needing to be retrieved. As of 2025-08-15, git repositories cannot be obtained due to legal issues. See IRC log below for details:

 anonymoususer852: Do we have CDN assets for Glitch saved? https://wiki.archiveteam.org/index.php?title=Glitch&curid=9768&diff=56723&oldid=56491
 arkiver: anonymoususer852: some of them yes
 nyakase: from what i recall the cdn urls came from the project repos, which were partially saved, so the cdn would be too
 arkiver: kpfleming: sorry for getting back late on this
 arkiver: was looking into finishing this
 arkiver: i wonder if i'm missing some logs, i do see several days missing at 21 22 july, but remembered there was something about cdn.glitch.me working when cdn.glitch.global does not
 arkiver: but, that does not seem to be the case
 arkiver: for example both
 arkiver: http://cdn.glitch.me/53e6984a-8533-47bd-9d86-4d0b3014e298/solar_system_model_orrery.glb
 arkiver: http://cdn.glitch.global/53e6984a-8533-47bd-9d86-4d0b3014e298/solar_system_model_orrery.glb
 arkiver: give the 503 error
 arkiver: or was there a different solution?
 arkiver: for progress you can always check https://tracker.archiveteam.org/glitch but it was been pretty silent the last few days
 kpfleming: in the past few days cdn.glitch.global and cdn.glitch.me became the same service, so they now have identical behavior
 kpfleming: i can look into enabling large-object support for the service
 kpfleming: ok, large-object support has been enabled so you should be able to retrieve your final batch of objects
 kpfleming: however, there is very aggressive rate limiting in place now, so you'll need to keep your downloads to something like 5 RPS or less in order to avoid it... or just handle the 429s that come back and wait until the rate limiting window has expired
 arkiver: kpfleming: i'll keep the rate low
 arkiver: but we'll also handle the 429 appropriately
 arkiver: kpfleming: archiving now!
 arkiver: 44k at 5/sec is not too bad
 arkiver: yeah first 10+ MB items coming in
 kpfleming: excellent!
 arkiver: allowing 403s and fixed various bugs, to finish this
 arkiver: all requeued
 arkiver: interesting, status code 905
 kpfleming: looks like it is stuck with 105 items left to go
 pabs: what was the plan with the remaining git repos?
 arkiver: pabs: kpfleming is going to give us dumps of them
 arkiver: or at least, make them available from some online location, after which we'll support them in this project and archive them into the Wayback Machine
 arkiver: kpfleming: done!
 arkiver: kpfleming: do you have an idea when we might be able to finish the last step with the project data?
 arkiver: (finished the last few items myself after fixes)
 anonymoususer852: I take it that CDN (assets) are now complete(ly archived), and we're now waiting on project data?
 arkiver: yes, the discovered CDN stuff is done
 anonymoususer852: Ah, thanks, will update wiki now.
 arkiver: but we skipped the git repos, as kpfleming is going to get us dumps later, that is still coming
 anonymoususer852: How about glitch.me? on the wiki that's grouped with source code, which I believe source code == git repos and git repos are partial/in progress.
 arkiver: yes, the repos are in progress i'd say
 arkiver: the non-repo stuff is done
 pabs: anonymoususer852 arkiver - CDN assets can't be complete, because we don't discover those from git, and we haven't got all of git yet :)
 pabs: er we do discover those from git
 arkiver: right!
 arkiver: we did discovery them from the git repos
 arkiver: but indeed, we have not gotten them yet for what kpfleming is going to get us
 kpfleming: @arkiver @pabs I've got some unwelcome news - we've been told by our leadership team that we can't extract and provide the project Git repositories as we had planned to do. Now that they are not publicly accessible, they have become 'private' and cannot be shared. There wasn't any way to know in advance that this would happen, and even if we had known and you kept crawling the site to pull what you could pull we would have had to shut down the site early because it was broken for regular users while the repository-crawling was happening.
 pabs: ugh :/
 pabs: any chance they will reconsider?
 arkiver: kpfleming: thank you for letting forwarding the bad news. i am not entirely unsurprised, but it is still not great
 arkiver: if there is any possibility for reconsideration, that would be very welcome, as pabs notes
 kpfleming: i don't think there is any chance, honestly. realistically the terms of service only gave us permission to publish the code uploaded by users as part of the Glitch service, but the Glitch service is no longer operating.
 arkiver: it's legal stuff
 arkiver: kpfleming: thanks anyway for getting back to us here!
 kpfleming: yes, it's legal stuff. the git repositories generally do not contain licenses at all, so the only reason we had permission to make them available is because of the Glitch ToS which gave us explicit permission to publish them as part of the service.
 kpfleming: with the service no longer operating, we don't have permission to publish them, but even if we did nobody else does either because of the lack of licenses.

Anonymoususer852 (talk) 14:33, 15 August 2025 (UTC)