Difference between revisions of "Twitter"

From Archiveteam
Jump to navigation Jump to search
(TweetScan has died, sometime around 2014)
(3 intermediate revisions by 3 users not shown)
Line 3: Line 3:
| image = Twitter_account_timeline.png
| image = Twitter_account_timeline.png
| description =  
| description =  
| URL = http://twitter.com
| URL = https://twitter.com/
| project_status = {{online}}
| project_status = {{online}}
| archiving_status = {{nosavedyet}}
| archiving_status = {{notsavedyet}}
}}
}}
'''Twitter''' is a microblogging service. With each "entry" being 280 (formerly 140) characters or less, the ease with which you can track the tiniest details of your life is amazing. The site has become very popular as a result.
'''Twitter''' is a microblogging service. With each "entry" being 280 (formerly 140) characters or less, the ease with which you can track the tiniest details of your life is amazing. The site has become very popular as a result.
Line 13: Line 13:
== Archives ==
== Archives ==
There are currently a few archives (but only partially):
There are currently a few archives (but only partially):
* [http://www.archive.org/details/twitter_cikm_2010 Cheng-Caverlee-Lee September 2009 - January 2010 Twitter Scrape]: almost 10 millon tweets
* [https://archive.org/details/twitter_cikm_2010 Cheng-Caverlee-Lee September 2009 - January 2010 Twitter Scrape]: almost 10 millon tweets
* [http://www.archive.org/details/2011-05-calufa-twitter-sql The May 2011 Calufa Twitter Scrape]: 90+ million tweets from more than 6 million users
* [https://archive.org/details/2011-05-calufa-twitter-sql The May 2011 Calufa Twitter Scrape]: 90+ million tweets from more than 6 million users
* https://archive.org/search.php?query=twitterstream
* {{IA item|twitterstream}}


The Twitter search API seemingly returns only the latest 7 days worth of tweets.
The Twitter search API seemingly returns only the latest 7 days worth of tweets.
Line 21: Line 21:
== Backup Tools ==
== Backup Tools ==


* Twitter enables you to [https://twitter.com/settings/account request an archive of all of your tweets from the main settings page], which includes every tweet of yours (therefore bypassing the normal 3200 tweet API limit). This is then emailed to the account linked with the account.
* Twitter enables you to [https://twitter.com/settings/your_twitter_data request an archive of all of your tweets from the main settings page], which includes every tweet of yours (therefore bypassing the normal 3200 tweet API limit). This is then emailed to the address linked with the account.
* [https://web.archive.org/web/20140105210020/http://www.tweetscan.com:80/data.php Tweetscan Data] (died sometime after 2014) downloaded your Twitter archive from 12/2007 onward in CSV format (requires Twitter account login/password)
 
* [https://github.com/JustAnotherArchivist/snscrape snscrape]
 
* Tweets can be archived easily through socialbot (an IRC bot that utilizes snscrape, [[ArchiveBot]], and [[ChromeBot]]), individual pages can be archived through ChromeBot. Alternatively, run snscrape locally, upload the list of URLs to https://transfer.notkiska.pw/, and feed them into ArchiveBot using the <code>!ao < <i>file</i></code> command.


* [https://github.com/sferik/t t by sferik] is a command-line interface for Twitter using the API via an application you create on your account. Not only does it allow easy CSV/JSON export of your own data, but it allows you to scrape others tweets. API limits apply but this tool is <b>very</b> powerful
* [https://github.com/sferik/t t by sferik] is a command-line interface for Twitter using the API via an application you create on your account. Not only does it allow easy CSV/JSON export of your own data, but it allows you to scrape others tweets. API limits apply but this tool is <b>very</b> powerful


Twitter automatically resizes uploaded images. To get image in its original resolution, append :orig after the url, e.g.:
Twitter automatically resizes uploaded images. To get image in its original resolution, append :orig after the url, e.g.:
     https://pbs.twimg.com/media/CBAoaU1UwAIUPIc.jpg:orig
     https://pbs.twimg.com/media/CBAoaU1UwAIUPIc.jpg:orig
When using [[ArchiveBot]], the following arguments are helpful:
--phantomjs --ignore-sets twitter
It is also '''important to add a trailing slash to the URL''', so it gets each tweet individually, rather than only trying to download the whole timeline.
A simple way to archive many Tweets from accounts and searches is [[ChromeBot]].
A common way to archive entire Twitter accounts is to put the tweets into an URL list using [[snscrape]], uploading them to https://transfer.sh/ or https://transfer.notkiska.pw/ and feeding them into [[ArchiveBot]] using the <code>!ao < <i>file</i></code> command.


* [https://github.com/sixohsix/twitter The Python Twitter API by sixohsix] has some pretty easy to use scripts for archiving Twitter accounts to a TXT file for people who aren't as technically inclined. It can only save the last 3K or so tweets due to inbuilt Twitter limits, though. (Note: the "-o" flag is pretty much required to archive accounts.)
* [https://github.com/sixohsix/twitter The Python Twitter API by sixohsix] has some pretty easy to use scripts for archiving Twitter accounts to a TXT file for people who aren't as technically inclined. It can only save the last 3K or so tweets due to inbuilt Twitter limits, though. (Note: the "-o" flag is pretty much required to archive accounts.)


* '''[https://github.com/DocNow/twarc/ twarc]'''
* [https://github.com/DocNow/twarc twarc]


* [https://gist.github.com/Asparagirl/e3ee274e4df49230875c880255819d95 Here's a Gist with a step-by-step guide] to getting a long list of a user's tweet status URL's, using a Python program called Tweep.
* [https://gist.github.com/Asparagirl/e3ee274e4df49230875c880255819d95 Here's a Gist with a step-by-step guide] to getting a long list of a user's tweet status URL's, using a Python program called Tweep.
* [https://web.archive.org/web/20140105210020/http://www.tweetscan.com:80/data.php Tweetscan Data] (died sometime after 2014) downloaded your Twitter archive from 12/2007 onward in CSV format (requires Twitter account login/password)


=== Scraping ===
=== Scraping ===
Line 57: Line 54:
== Library of Congress ==
== Library of Congress ==


The U.S. Library of Congress announced in April 2010, via its official Twitter account that it will be acquiring the entire archive of Twitter messages back through March 2006.[http://www.readwriteweb.com/archives/twitters_entire_archive_headed_to_the_library_of_c.php] As of 2016-02-23, this archive is still not available, and when/if it does become accessible it will likely be restricted to researchers, rather than the general public.[http://www.politico.com/story/2015/07/library-of-congress-twitter-archive-119698.html]  In January 2017, it was announced that the Library of Congress will no longer archive all tweets, just ones from major news stories.
The U.S. Library of Congress announced in April 2010, via its official Twitter account that it will be acquiring the entire archive of Twitter messages back through March 2006.[http://www.readwriteweb.com/archives/twitters_entire_archive_headed_to_the_library_of_c.php] As of 2016-02-23, this archive is still not available, and when/if it does become accessible it will likely be restricted to researchers, rather than the general public.[https://www.politico.com/story/2015/07/library-of-congress-twitter-archive-119698.html]  In January 2017, it was announced that the Library of Congress will no longer archive all tweets, just ones from major news stories.[https://www.npr.org/sections/thetwo-way/2017/12/26/573609499/library-of-congress-will-no-longer-archive-every-tweet]
 
== Lists ==
 
* [[List of Twitter hashtags by country and territory]]
* [[List of Twitter hashtags by language]]


== External links ==
== External links ==
* http://twitter.com
* https://twitter.com/


{{Navigation box}}
{{Navigation box}}


[[Category:Twitter| ]]
[[Category:Microblogging services]]
[[Category:Microblogging services]]

Revision as of 09:57, 10 June 2020

Twitter
Twitter account timeline.png
URL https://twitter.com/
Status Online!
Archiving status Not saved yet
Archiving type Unknown
IRC channel #archiveteam-bs (on hackint)

Twitter is a microblogging service. With each "entry" being 280 (formerly 140) characters or less, the ease with which you can track the tiniest details of your life is amazing. The site has become very popular as a result.

The site is becoming so popular, in fact, that many people are deserting or cutting back on their weblogs to just use the Twitter service for what their weblogging used to fulfill; and with that comes rampant centralization, and with that, greater risk. Back up your tweets!

Archives

There are currently a few archives (but only partially):

The Twitter search API seemingly returns only the latest 7 days worth of tweets.

Backup Tools

  • Tweets can be archived easily through socialbot (an IRC bot that utilizes snscrape, ArchiveBot, and ChromeBot), individual pages can be archived through ChromeBot. Alternatively, run snscrape locally, upload the list of URLs to https://transfer.notkiska.pw/, and feed them into ArchiveBot using the !ao < file command.
  • t by sferik is a command-line interface for Twitter using the API via an application you create on your account. Not only does it allow easy CSV/JSON export of your own data, but it allows you to scrape others tweets. API limits apply but this tool is very powerful

Twitter automatically resizes uploaded images. To get image in its original resolution, append :orig after the url, e.g.:

   https://pbs.twimg.com/media/CBAoaU1UwAIUPIc.jpg:orig
  • The Python Twitter API by sixohsix has some pretty easy to use scripts for archiving Twitter accounts to a TXT file for people who aren't as technically inclined. It can only save the last 3K or so tweets due to inbuilt Twitter limits, though. (Note: the "-o" flag is pretty much required to archive accounts.)
  • Tweetscan Data (died sometime after 2014) downloaded your Twitter archive from 12/2007 onward in CSV format (requires Twitter account login/password)

Scraping

See Site exploration for details.

Vital Signs

Very stable, probably not going anywhere too soon without warning.

Also see

Library of Congress

The U.S. Library of Congress announced in April 2010, via its official Twitter account that it will be acquiring the entire archive of Twitter messages back through March 2006.[1] As of 2016-02-23, this archive is still not available, and when/if it does become accessible it will likely be restricted to researchers, rather than the general public.[2] In January 2017, it was announced that the Library of Congress will no longer archive all tweets, just ones from major news stories.[3]

Lists

External links