From Archiveteam
Revision as of 17:18, 15 November 2021 by Jake (talk | contribs) (Add link to collection.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Status Special case
Archiving status In progress...
Archiving type Unknown
Project source https://github.com/ArchiveTeam/ftp-nab
Project tracker https://tracker.archiveteam.org/ftp/
IRC channel #effteepee (on hackint)
Data[how to use] archiveteam_ftp

The File Transfer Protocol, FTP, is a protocol for file transfer published as RFC 114 on 16 April 1971. In the older days of the internet the protocol was frequently used to upload and share files. Today the protocol is not used so much anymore. This made Archive Team decide to grab all the FTP servers.

The FTP grab started 30 November 2015.

How can I help?

Running the script manually

If you use Linux and you're a bit familiar with it, you can try running the script directly.

The instructions can be found at https://github.com/ArchiveTeam/ftp-grab.

Some additional information
Don't forget to replace YOURNICKHERE with your nickname.

The number after --concurrent determines how many threads run at the same time. You can increase this number if your resources (RAM, CPU, bandwidth) are sufficient. However, if you constantly see messages about rate limiting, there is no need to increase the concurrency.

If you want to stop the script, please do it gracefully if possible. To do so, create an empty file named STOP in the folder of the script (terminal command: touch STOP). The script finishes the current item(s) and stops only after that. (If you kill the script immediately, the items get broken, and they will need to be reassigned to another user.) – Before starting the script again, don't forget to remove the STOP file.

If you see "Project code is out of date", kill the script, go to its folder (cd ftp-grab) and issue git pull https://github.com/ArchiveTeam/ftp-grab. After the updating has finished, re-launch the script.

Discovery items

The project needs to have items to be able to run. You can help discovering these items.

Scripts for creating items for the grab can be found at https://github.com/ArchiveTeam/ftp-queue. Instructions on how to run the grab can be found in the README. A list of FTPs that need to be scanned can be found at FTP/List.

Some additional information
User:Squidboy says:

It's worth noting that as of June 2019 ftp-queue has several issues that may make it hard to use.

Donating to the Internet Archive

Content downloaded by the ArchiveTeam will be uploaded to the Internet Archive, where it will be stored and be available – hopefully – forever. However, storing it costs thousands of dollars in the long run. So, if you can afford, please consider donating to the Internet Archive, so that this piece of history can be kept for us all. https://archive.org/donate/

External Links