Difference between revisions of "Google Drive"

From Archiveteam
Jump to navigation Jump to search
(On the slowly-progressing access tool)
(A tepid retrospect on a grab that was almost 2 years ago(???))
Line 13: Line 13:
'''Google Drive''' is a filehosting service, a la Dropbox, run by Google (not to be confused with [https://cloud.google.com/ Google Cloud Storage] and similar more technical storage solutions). It is popular both for personal storage and for sharing of files.
'''Google Drive''' is a filehosting service, a la Dropbox, run by Google (not to be confused with [https://cloud.google.com/ Google Cloud Storage] and similar more technical storage solutions). It is popular both for personal storage and for sharing of files.


Google Drive IDs are not random (anecdotally, IDs of folders in the same tree often share long parts), which makes them predictable, a problem which Google has been trying to rectify across its products (others of which have similar issues) throughout 2021<ref>https://www.theverge.com/google/2021/6/23/22548007/youtube-google-drive-security-link-sharing</ref>. As such, on September 13, 2021, Google will be requiring that, in order to access files and folders, users either have permissions tied to their signed-in Google Accounts, or access the item through a URL with a random per-item parameter called resourceKey, apparently introduced in 2021.<ref>https://support.google.com/drive/answer/10729743</ref> The result of this will be that at least millions of links across the Web will effectively break. It should be noted that Docs, Sheets and Slides will be exempted from this update <ref>https://support.google.com/a/answer/10685032</ref> Apart from the longening links, the main threat is the users deleting files. Files are usually deleted to fit the users 15 GB limit.
== 2021 grab ==


== Grab ==
Google Drive IDs are not random (anecdotally, IDs of folders in the same tree often share long parts), which makes them predictable, a problem which Google had been trying to rectify across its products (others of which have similar issues) throughout 2021<ref>https://www.theverge.com/google/2021/6/23/22548007/youtube-google-drive-security-link-sharing</ref>. As such, on September 13, 2021, Google required that, in order to access files and folders, users either have permissions tied to their signed-in Google Accounts, or access the item through a URL with a random per-item parameter called resourceKey, apparently introduced in 2021.<ref>https://support.google.com/drive/answer/10729743</ref> The result of this will be that at least millions of links across the Web will effectively break. Docs, Sheets and Slides will be exempted from this update <ref>https://support.google.com/a/answer/10685032</ref> Apart from the longening links, the main threat was the users deleting files. Files are usually deleted to fit the users 15 GB limit.
The upcoming grab script has 3 item types, folder:, file:, and user:. It is intended that all folder: items be run first, to get a pool (through backfeed) of file: items, that can be randomly sampled to determine a size threshold that the Internet Archive will accept; then files will be run. Users contain some user metadata but not links to files or folders.
 
=== Grab ===
The grab script had 3 item types, folder:, file:, and user:. It was intended that all folder: items be run first, to get a pool (through backfeed) of file: items, that can be randomly sampled to determine a size threshold that the Internet Archive will accept; then files will be run. Users contain some user metadata but not links to files or folders.


Playback is theoretically possible with a flexible, POST-capable Wayback Machine, but this does not yet exist. In the meantime, it may be possible to get files with vanilla wget or similar from the WBM. A tool is currently in development, with the Google Drive parts finished, to download from the Google Drive archives; contact [[User:OrIdow6|OrIdow6]] if you want a copy of the current state.
Playback is theoretically possible with a flexible, POST-capable Wayback Machine, but this does not yet exist. In the meantime, it may be possible to get files with vanilla wget or similar from the WBM. A tool is currently in development, with the Google Drive parts finished, to download from the Google Drive archives; contact [[User:OrIdow6|OrIdow6]] if you want a copy of the current state.


== How to help ==
=== Results ===
Tech234a has written a small bit for the unfamiliar on how to help [https://datahorde.org/help-archive-team-archive-public-google-drive-files-before-september-13/ here].
(This is based on OrIdow6's very vague memories)
It appeared that there were 2 types of Google Drive items, those that automatically got a redirect to a version with a resourceKey, and those that didn't. There was speculation that the latter, which had more random-looking IDs, would not suffer in the quasi-removal.

Revision as of 13:26, 22 July 2023

Google Drive
Google drive logo.png
URL http://drive.google.com/[IAWcite.todayMemWeb]
Status Special case (Technically files are only becoming inaccessible)
Archiving status In progress...
Archiving type Unknown
Project source google-drive-grab, google-drive-items
Project tracker google-drive
IRC channel #googlecrash (on hackint)
Data[how to use] archiveteam_googledrive

Google Drive is a filehosting service, a la Dropbox, run by Google (not to be confused with Google Cloud Storage and similar more technical storage solutions). It is popular both for personal storage and for sharing of files.

2021 grab

Google Drive IDs are not random (anecdotally, IDs of folders in the same tree often share long parts), which makes them predictable, a problem which Google had been trying to rectify across its products (others of which have similar issues) throughout 2021[1]. As such, on September 13, 2021, Google required that, in order to access files and folders, users either have permissions tied to their signed-in Google Accounts, or access the item through a URL with a random per-item parameter called resourceKey, apparently introduced in 2021.[2] The result of this will be that at least millions of links across the Web will effectively break. Docs, Sheets and Slides will be exempted from this update [3] Apart from the longening links, the main threat was the users deleting files. Files are usually deleted to fit the users 15 GB limit.

Grab

The grab script had 3 item types, folder:, file:, and user:. It was intended that all folder: items be run first, to get a pool (through backfeed) of file: items, that can be randomly sampled to determine a size threshold that the Internet Archive will accept; then files will be run. Users contain some user metadata but not links to files or folders.

Playback is theoretically possible with a flexible, POST-capable Wayback Machine, but this does not yet exist. In the meantime, it may be possible to get files with vanilla wget or similar from the WBM. A tool is currently in development, with the Google Drive parts finished, to download from the Google Drive archives; contact OrIdow6 if you want a copy of the current state.

Results

(This is based on OrIdow6's very vague memories) It appeared that there were 2 types of Google Drive items, those that automatically got a redirect to a version with a resourceKey, and those that didn't. There was speculation that the latter, which had more random-looking IDs, would not suffer in the quasi-removal.