Difference between revisions of "Tracker"

From Archiveteam
Jump to navigation Jump to search
(add a screenshot of a leaderboard)
m (→‎General Overview: formatting)
Line 1: Line 1:
= General Overview =
== General Overview ==


[[File:Tracker_test_project_overview_screenshot.png|right|thumb|Project admin overview]]
[[File:Tracker_test_project_overview_screenshot.png|right|thumb|Project admin overview]]
Line 5: Line 5:
The [https://github.com/ArchiveTeam/universal-tracker Tracker] software is the center-pivot of Archiveteam's distributed archiving efforts. It hands out items to be downloaded and keeps track of what is completed. Items can be usernames, subdomains, full urls, basically any unit we can use to break the site into manageable chunks. The progress of each project can be viewed via the leader board interface on http://tracker.archiveteam.org .
The [https://github.com/ArchiveTeam/universal-tracker Tracker] software is the center-pivot of Archiveteam's distributed archiving efforts. It hands out items to be downloaded and keeps track of what is completed. Items can be usernames, subdomains, full urls, basically any unit we can use to break the site into manageable chunks. The progress of each project can be viewed via the leader board interface on http://tracker.archiveteam.org .


[[File:Xanga_leaderboard.png|left|thumb|A leaderboard]]
[[File:Xanga_leaderboard.png|right|thumb|A leaderboard]]


The [[ArchiveTeam Warrior|Warrior]] is the yang to the Tracker's yin. The warriors get the list of current projects from the project file on http://warriorhq.archiveteam.org/ .
The [[ArchiveTeam Warrior|Warrior]] is the yang to the Tracker's yin. The warriors get the list of current projects from the project file on http://warriorhq.archiveteam.org/ .


This is a sample from the project file:
__TOC__
 
== API ==
 
This is a sample from the project file (line breaks included for readability):


<pre>
<pre>
Line 18: Line 22:
     "repository": "https://github.com/ArchiveTeam/streetfiles-grab.git",
     "repository": "https://github.com/ArchiveTeam/streetfiles-grab.git",
     "logo": "http://archiveteam.org/images/7/7b/Streetfiles-logo.png",
     "logo": "http://archiveteam.org/images/7/7b/Streetfiles-logo.png",
     "marker_html": "<a href='http://tracker.archiveteam.org/streetfiles/'><img src='http://archiveteam.org/images/7/7b/Streetfiles-logo.png' alt='Streetfiles' width='235' height='50' /></a>",
     "marker_html":  
        "<a href='http://tracker.archiveteam.org/streetfiles/'>
        <img src='http://archiveteam.org/images/7/7b/Streetfiles-logo.png'
        alt='Streetfiles' width='235' height='50' /></a>",
     "deadline": "2013-04-30T23:59:59Z",
     "deadline": "2013-04-30T23:59:59Z",
     "host": "streetfiles.org",
     "host": "streetfiles.org",

Revision as of 07:02, 2 August 2013

General Overview

Project admin overview

The Tracker software is the center-pivot of Archiveteam's distributed archiving efforts. It hands out items to be downloaded and keeps track of what is completed. Items can be usernames, subdomains, full urls, basically any unit we can use to break the site into manageable chunks. The progress of each project can be viewed via the leader board interface on http://tracker.archiveteam.org .

A leaderboard

The Warrior is the yang to the Tracker's yin. The warriors get the list of current projects from the project file on http://warriorhq.archiveteam.org/ .

API

This is a sample from the project file (line breaks included for readability):

{
    "name": "streetfiles",
    "title": "Streetfiles",
    "description": "Streetfiles is closing April, 30th, 2013.",
    "repository": "https://github.com/ArchiveTeam/streetfiles-grab.git",
    "logo": "http://archiveteam.org/images/7/7b/Streetfiles-logo.png",
    "marker_html": 
        "<a href='http://tracker.archiveteam.org/streetfiles/'>
        <img src='http://archiveteam.org/images/7/7b/Streetfiles-logo.png'
        alt='Streetfiles' width='235' height='50' /></a>",
    "deadline": "2013-04-30T23:59:59Z",
    "host": "streetfiles.org",
    "leaderboard": "http://tracker.archiveteam.org/streetfiles/",
    "lat_lng": [
        51,
        9
    ]
},

It shows where to get the grab code and other project information.

Hardware

The tracker runs on a Linode 1 GB instance operated by chronomex. The system is tracked by Munin.

Software used:

  • Universal Tracker is a Ruby HTTP application that sends and receives JSON payloads and uses Redis for the data store.
  • Redis A memory based key value store
  • Debian is the Linux distribution the stack is built upon.
  • warrior-hq a small Sinatra web app to manage the Warriors and display the geo-location world map.

People

These are the volunteers who take care of the different services that form Archive Team and URLTeam.

  • Wiki Admins - SketchCow, winr4r
  • URLTeam Tracker - ersi, xmc, (formerly soultcer)
  • Warrior Tracker ssh - alard, underscor, Smiley
  • Warrior Tracker web - alard, underscor, omf_, Smiley
  • Anarchive server - GLaDOS, omf_, Smiley
  • Github Organization Admins - GLaDOS, omf_, ivan
  • IRC twitter bot - swebb
  • pad.archivingyoursh.it & paste.archivingyoursh.it - GLaDOS
  • Domain registration (archiveteam.org urlte.am) - SketchCow