Difference between revisions of "Friendster dataset"

From Archiveteam
Jump to navigation Jump to search
(Created page with 'It would be nice to collect a dataset of the Friendster social graph, linking friends to friends. This could be of interest to social computing scientists and other graph lovers.…')
 
Line 16: Line 16:
html pages it downloads are fairly large, but the resulting list of friends is
html pages it downloads are fairly large, but the resulting list of friends is
much smaller.
much smaller.
To run the script you need [http://ruby-lang.org/ Ruby] and the [http://rubygems.org/gems/typhoeus typhoeus gem]. Download the script from the [https://github.com/alard/friendster-graph git repository] and run <code>ruby bff-graph-client.rb</code>.


== Installing on Debian / Ubuntu ==
== Installing on Debian / Ubuntu ==

Revision as of 22:26, 21 June 2011

It would be nice to collect a dataset of the Friendster social graph, linking friends to friends. This could be of interest to social computing scientists and other graph lovers.

There already exists a small dataset, but that only contains 100,000 users. We can do better.

The friends list of Friendster remain after the change to the new version at the end of June, so there should be enough time to download everything. If you want to help, here's how you can:

Script

Download Friendster friend lists, to investigate the Friendster social network.

This script will contact a central tracker to get an id range (of 10,000 Friendster ids at a time). It will then download the friends lists for these users and parse them to extract the user ids of the friends. The list of connections for each user will be saved to a local file and submitted back to the tracker. This process takes between 200 to 300 seconds for each range of 10,000 ids. The html pages it downloads are fairly large, but the resulting list of friends is much smaller.

To run the script you need Ruby and the typhoeus gem. Download the script from the git repository and run ruby bff-graph-client.rb.


Installing on Debian / Ubuntu

sudo apt-get install ruby rubygems libcurl4-gnutls-dev
sudo gem install typhoeus

git clone git://github.com/alard/friendster-graph.git
ruby bff-graph-client.rb