Difference between revisions of "Distributed recursive crawls"
Jump to navigation
Jump to search
(Created page with "{{Infobox project | project_status = {{specialcase}} | archiving_status = {{in progress}} | archiving_type = DPoS | source = [https://github.com/ArchiveTeam/grab-grab grab-grab] | tracker = [https://tracker.archiveteam.org/grab/ grab] | irc = Y }} This is a project to recursively crawl large websites that have no clear structure that can easily be split into work items the way we usually do on DPoS projects. It is somewhat comparable to ArchiveBot in that crawls...") |
(No difference)
|
Revision as of 20:07, 22 March 2022
| Distributed recursive crawls | |
| Status | Special case |
| Archiving status | In progress... |
| Archiving type | DPoS |
| Project source | grab-grab |
| Project tracker | grab |
| IRC channel | #Y (on hackint) |
This is a project to recursively crawl large websites that have no clear structure that can easily be split into work items the way we usually do on DPoS projects. It is somewhat comparable to ArchiveBot in that crawls are started manually for specific sites of interest.