Difference between revisions of "TwitPic"

From Archiveteam
Jump to navigation Jump to search
m
Line 56: Line 56:
=== Phase 2: content grab ===
=== Phase 2: content grab ===


After some testing, actual content grab has begun on September 14. Its progress can be followed on the [http://tracker.archiveteam.org/twitpic tracker]. (One item contains 36 images and other elements of the image pages.)
After some testing, actual content grab began on September 14. Its progress can be followed on the [http://tracker.archiveteam.org/twitpic tracker]. (One item contains 36 images and other elements of the image pages.)


== How can I help? ==
== How can I help? ==
Line 88: Line 88:
=== Joining us on IRC ===
=== Joining us on IRC ===


Either you run the Warrior or the script, you should join our IRC channel '''#quitpic''' to catch the latest news about the project and its progress, and there you can also put questions if something doesn't work. You can use the web interface at http://chat.efnet.org:9090, or if you use a standalone IRC client, connect to irc://irc.efnet.org/quitpic.
Either you run the Warrior or the script, you should join our IRC channel '''#quitpic''' to catch the latest news about the project and its progress, and there you can also put questions if something doesn't work. You can use the web interface at http://chat.efnet.org:9090, or if you use a standalone IRC client, connect to irc://irc.efnet.org.


== Archives ==
== Archives ==

Revision as of 18:10, 15 September 2014

TwitPic
TwitPic logo
TwitPic mainpage in 2011-01-12
TwitPic mainpage in 2011-01-12
URL http://twitpic.com
Status Closing
Archiving status In progress...
Archiving type Unknown
Project source twitpic-discovery, twitpic-grab, twitpic-items
Project tracker twitpicdisco, twitpic
IRC channel #quitpic (on hackint)

TwitPic is an image hosting service. The service is designed mainly for Twitter users - the images uploaded on the service are given short URLs for usage in Twitter posts. Twitter carries a 140-character post limit, the average Twitpic URL is 25/26 characters long.

On September 4, 2014 TwitPic announced they were shutting down on September 25.

Shutdown

Posted on September 4, 2014 by Noah Everett on blog.twitpic.com:

Twitpic will be shutting down September 25th. You will be able to export all your photos and videos. We’ll let everyone know when this feature is live in the next few days.

This is an unexpected and hard announcement for us to make and we want to lay out what led us to this decision.

A few weeks ago Twitter contacted our legal demanding that we abandon our trademark application or risk losing access to their API. This came as a shock to us since Twitpic has been around since early 2008, and our trademark application has been in the USPTO since 2009.

Here is some backstory on the history of our trademark:

We originally filed for our trademark in 2009 and our first use in commerce dates back to February 2008 when we launched. We encountered several hurdles and difficulties in getting our trademark approved even though our first use in commerce predated other applications, but we worked through each challenge and in fact had just recently finished the last one. During the “published for opposition” phase of the trademark is when Twitter reached out to our counsel and implied we could be denied access to their API if we did not give up our mark.

Unfortunately we do not have the resources to fend off a large company like Twitter to maintain our mark which we believe whole heartedly is rightfully ours. Therefore, we have decided to shut down Twitpic.

On a personal note I (@noaheverett) want to thank you for letting us be a part of your life and helping you share your experiences over the past 6 years, it’s truly been an honor. I have learned so much through running Twitpic over the years. Through the many mistakes I’ve made and lessons learned, to the bad days and the great days. Thank you again everyone…I will miss and cherish the days of Twitpic we had together.

Site structure

Image page urls:

http://twitpic.com/****** http://twitpic.com/***** http://twitpic.com/**** http://twitpic.com/*** http://twitpic.com/** http://twitpic.com/* where * = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z

where ****** consists of up to 6 alphanumeric characters. Leading zeros are irrelevant, e.g.: /000joe = /0joe = /joe. Like incremental numbers in base-36 numeral system.

Progress

Phase 1: content discovery

From September 5 to 6, until ArchiveTeam got banned, ~41 million of the possible ~900 million urls were discovered. The discovery was suspended.

Phase 2: content grab

After some testing, actual content grab began on September 14. Its progress can be followed on the tracker. (One item contains 36 images and other elements of the image pages.)

How can I help?

Important notice: TwitPic staff may ban ArchiveTeam members' access to their site through AT tools, or completely (IP address ban), and for a long time. If you want to use TwitPic outside ArchiveTeam tools (e.g. if you have an account there and you want to access it), consider running the Warrior/script with low concurrency, or, if you're paranoid, not running it at all.

Running a Warrior

You can start up a Warrior and there select TwitPic Phase 2. (If you don't really care what you are archiving, select ArchiveTeam's Choice instead, as at some points ArchiveTeam may priorize another project.)

If you see "Project code is out of date", simply restart the warrior.

Running the script manually

If you use Linux and you're a bit familiar with it, you can try running the script directly.

The instructions can be found at twitpic-grab. Please replace occurences of "twitpic-discovery" with "twitpic-grab".

Don't forget to replace YOURNICKHERE with your nickname.

The number after --concurrent determines how many threads run at the same time. You can increase this number if your resources (RAM, CPU, bandwidth) are sufficient. However, if you see messages about rate limiting, there is no need to increase the concurrency. Note: the higher the concurrency is, the more the chance is to be banned by TwitPic staff.

If you want to stop the script, please do it softly if possible. To do so, create an empty file named STOP in the folder of the script (twitpic-grab). That way you don't interrupt an active download/upload process.

If you see "Project code is out of date", kill the script, go to its folder (cd twitpic-grab) and issue git pull https://github.com/ArchiveTeam/twitpic-grab. After the updating has finished, re-launch the script.

For both Warrior and script

If you see 403 error codes in the output of the script about files not on twitpic.com (e.g. on twimg.com or else), don't worry. That is normal and the script handles the problem. However, if the script receives 403s from twitpic.com, your script (or even your IP address) has possibly got blocked. You can retry later, but you may be blocked for a long time.

Joining us on IRC

Either you run the Warrior or the script, you should join our IRC channel #quitpic to catch the latest news about the project and its progress, and there you can also put questions if something doesn't work. You can use the web interface at http://chat.efnet.org:9090, or if you use a standalone IRC client, connect to irc://irc.efnet.org.

Archives

Coming soon.

Download Your Data

Noah Everett (TwitPic's owner) promised they would create a tool with which users can export their data. Follow TwitPic's news to get informed about the ways of saving your own data. (When it's about your content, don't rely on ArchiveTeam's archives, as they may not be complete, and are not made in a way that a single user's content can be extracted from them.)

Downloaders

  • Downloader by tag (it saves the full resolution image and metadata: uploader, date and description)

External links