Difference between revisions of "URLTeam"
(→Non-warrior projects: snipurl count in old dump)
(→Warrior projects: note trap-it re-scrape starting)
|Line 575:||Line 575:|
Revision as of 05:35, 5 December 2015
url shortening was a fucking awful idea
|Archiving status||In progress...|
|Project source||Old: urlteam-stuff tinyback tinyarchive|
|Project tracker||http://tracker.archiveteam.org:1337/ (HTTPS)|
|IRC channel||(on hackint)|
TinyURL, bit.ly and other similar services allow long URLs to be converted to smaller ones on their specific service; the small URL is visited by a consumer and their web browser is redirected to the long URL.
Such services are a ticking timebomb. If they go away, get hacked or sell out millions of links will be lost (see Wikipedia: Link Rot). Archive.org/301Works is acting as an escrow for URL shortener databases, but they rely on URL shorteners to actually give them their databases. Even 301Works founding member bit.ly does not actually share their databases and most other big shorteners don't share theirs either.
The fine folks at archive.org have provided us with upload permissions to the 301Works archive: http://www.archive.org/details/301utm. They unfortunately do not want to make them downloadable, but the same data is in our torrents too, just in a different format (we use pipe-delimited, xz-compressed files while 301works uses comma-delimited uncompressed files).
- fetcher.pl: Perl-based scraper by User:Chronomex
- TinyBack: Python 2.x-based, distributed scraper (formerly used)
- Terror of Tiny Town: currently used by ArchiveTeam
Terror of Tiny Town
The easiest way to help with scraping is to run the Warrior and select the URLTeam 2 project. You can also run ToTT outside the warrior; to do so, follow the instructions at https://github.com/ArchiveTeam/terroroftinytown-client-grab.
|Name||Est. number of shorturls||Scraping done by||Status||Comments||# in dump|
|http://goo.gl||?||User:Scumola||started (2011-03-04)||goo.gl throttles pulls ; (this apparently didn't make it in the last old-style dump)||0|
|http://ff.im||?||User:Chronomex||only used by FriendFeed, no interface to shorten new URLs||1,189,782|
|http://4url.cc||1279 (2009-08-14)||User:Chronomex||dead (2011-02-15)||1,279|
|http://litturl.com||17096 (2010-04-15)||User:Chronomex||dead (2010-11-18)||17,096|
|http://xs.md||3084 (2009-08-15)||User:Chronomex||done||dead (2010-11-18)||3,084|
|http://url.0daymeme.com||14867 (2009-08-14)||User:Chronomex||done||dead (2010-11-18)||14,867|
|http://tr.im (old)||1990425||?||got what we could||dead (2011-12-31)||1,990,425|
|visibli (hex)||16777216||User:Chfoo Warrior||In progress
Done. 15104865 301MB
|Using links.sharedby.co/links/ as URL prefix.
|http://post.ly (Posterous)||?||Warrior/EC2||done||dead (divided up in 11 files in the last old-style dump)||153,281,595|
|http://zapd.co Zapd||326592||User:Chfoo||Done. 144093 1.7M||xxxx.zapd.co. Uploaded to IA||0|
|http://bre.ad Bre.ad||120932351||User:Chfoo||Incomplete (59771889 examined). 54506 1.2MB||de.ad (2013-11-18). Uploaded to IA
Got what I can without overloading their EC2 instance.
|http://arseh.at||?||?||?||also worked on as a Warrior job, see below.||23,655|
|http://bit.ly||?||?||?||divided up into 3,835 files in the last old-style dump, totaling 39 GB (compressed!); also worked on as a Warrior job, see below.||1,507,816,439|
|http://is.gd||?||?||?||divided up into 125 files (totally 8.5 GB) in the last old-style dump; also worked on as a Warrior job, see below.||302,434,257|
|http://kl.am||?||?||?||Dead (as of 03:00, 3 December 2015 (EST))||1,870,335|
|http://links.sharedby.co||?||?||?||in a file called links.sharedby.co-links.txt.xz in the last old-style dump||9,298,101|
|http://ow.ly||?||?||?||divided up into 23 files (totaling 4.5 GB) in the last old-style dump; also worked on as a Warrior job, see below.||?|
|http://surl.ws||?||?||?||Yes, there really are only 48 listed.||48|
|http://snipurl.com||?||?||?||divided up into 8 files in the last old-style dump (for a total of 6.8 GB compressed, though); also worked on as a Warrior job, see below.||368,750,225|
|http://tinyurl.com||?||?||?||divided up into 60 files (totaling 9.4 GB) in the last old-style dump; also worked on as a Warrior job, see below.||470,413,092|
|http://tr.im (new)||?||?||?||in the last old-style dump as a file called tr.im-relaunched.txt.xz||1,266,068|
|http://ur1.ca||?||?||?||(Note this is named "ur" followed by the digit one, not L.)||8,432,282|
|http://vbly.us||?||?||?||also worked on as a Warrior job, see below.||114,924|
|twitter-unrolled-urls-spritzer-stream||?||?||?||in a 4.6GB file named: twitter-unrolled-urls-spritzer-stream-20111015-20130318.txt.xz ; there's a note about it in the README file||257,993,902|
|Name||Number of shorturls||Scraping done by||Status||Comments||# in dump|
# in dump refers to the number of short URLs included in the last old-style dump, URLTeamTorrentRelease2013July.
For the latest TinyTown updates, please see chfoo's spreadsheet.
|Warrior project name||Est. # shorturls||Last scraped date||Initially scraped date||# checked||Example URL||Incr||Comments|
|10,000,000,000||2023-01-28||2014-11-22||2,737,170,000||http://tinyurl.com/mxzufis||N||done: sequential to zzzzzz; current: non-sequential, 7 characters; uses custom code|
|?||2015-11-01||2015-08-13||20,308,700||http://tinyurl.hu/4q22||?||uses custom code|
|50,000,000,000||2023-01-28||2014-11-22||4,219,363,400||https://bit.ly/1Zmfo8z||N||done: non-sequential 6 characters; current: non-sequential, 6 characters; uses custom code|
|934,134,706 (as of 2013-05-20)||2023-01-28||2014-11-22||2,243,213,300||http://is.gd/mBNPCM||?||done: sequential up to ZZZZZ ; new shorturls: non-sequential, 6 characters; uses custom code|
|?||2014-12-12||2014-12-01||42,813,150||?||?||done: sequential to 42pzz ; dead (2014-JUL-17) ; Appears incremental - Ex: http://tr.im/44tn2 http://tr.im/44tn4|
|?||2023-01-28||2015-01-29||45,484,500||?||?||(Also see http://vsb.li. Double redirects via USERNAME.sharedby.co/share/XXXXXX ) (and http://shrd.by ); uses custom code|
|?||2014-12-20||2014-12-17||10,874,150||?||?||new shorturls: sequential ; FOSS, run by StatusNet; claims to offer a download of their database, but it just contains garbage|
|?||2015-03-06||2015-01-24||181,015,750||?||Y||new shorturls: sequential ; snipr.com / snipurl.com / snurl.com - Appears incremental - Ex: http://snipr.com/27nvst http://snipr.com/27nvtt. snipr.com and snipurl.com work but appear infected with malware. ; uses custom code|
|?||2015-04-20||2015-03-06||293,372,600||?||?||see snipurl entry; uses custom code|
|?||2015-11-09||2015-01-11||628,650||http://vbly.us/2mwv||Y||new shorturls: sequential; 2015-11-09 scape from 2jp6 up to 2mwv (seq # 123000)|
|?||2015-01-15||2015-01-11||1,842,350||?||Y||Appears down; new shorturls: sequential|
|?||2015-1-24||2014-12-13||8,615,100||?||?||Appears down; uses custom code|
|?||2015-05-27||2015-03-06||39,367,900||http://alturl.com/wqok||?||Appears to redirect to http://shorturl.com ; Probably sequential/loweralpha; uses custom code|
|?||2014-12-18||2014-12-17||3,303,100||?||?||Appears down; Argyle Social, main page 404s, existing urls still work|
|?||2015-04-04||2014-12-24||967,591,000||?||?||main page redirects, doesn't allow for new urls to be publicly shortened, existing urls still work; uses custom code|
|50,000||2015-11-15||2015-11-15||20,555||http://bull.hn/l/19JQE/||?||Vanity URL shortner for a recruiting company, "Bullhorn Reach"|
|1,500||2015-11-08||2014-11-06||3,250||http://burl.se/428||Y||200 re-checked on 2015-11-08|
|2023-01-28||2015-11-16||41,150||http://dwurl.hu/gMEtiA ( created 01:36, 2 November 2015 (EST))||N||Allows public shortening; appears to give 6 character, mixed case alphabetic (no digits), non-incremental URLs|
|?||2015-01-03||2015-01-03||129,762,450||?||?||Banned; uses custom code|
|?||2023-01-28||2015-11-16||8,613,131||http://fwdurl.net/uDRZ4TGAj6V (created Nov 16, 2015)
http://fwdurl.net/aYwa (created Dec 17, 2013)
http://fwdurl.net/bjkDtwYAoWSF (created Dec 4, 2015)
|?||responds very slowly; stats (including creation date, & hit count) available by suffixing "!"|
|?||2015-11-22||2014-11-06||5,239,050||http://kcy.me/28trr||?||Shortening service for http://karmacracy.com ; requires free account to create short URLs ; ran initially for one day, re-checked a year later|
|?||2015-11-22||2015-04-11||12,767,850||http://korta.nu/9ob||?||Initial run ended on 2015-05-27; doing a 2nd pass through the 3-character values starting 2015-11-22|
|420,000,000||2023-01-28||2015-11-08||51,626,015||http://migre.me/sd9AB||Y||up to 5 characters, mixed case alphanumeric, currently around rZIfF (as of 02:00, 2 November 2015 (EST))|
|?||2015-04-04||2015-02-09||16,245,049||?||?||uses custom code|
|?||2015-12-04||2014-11-06||279,800||?||?||Home page is a blank black screen. (as of 01:38, 9 November 2015 (EST)); re-scan of 2-character range (nothing new found); and 4 character range on 2015-12-04|
|?||2015-03-10||2015-01-20||367,074,900||http://ow.ly/UVFi7||Y||owned by Hootsuite; new shorturls: sequential ; (aliases: http://htl.li , http://ht.ly , http://owl.li ); uses custom code|
|?||2015-04-04||2014-12-13||989,290,500||?||?||Related to the pond called Philadelphia, where links are born and raised, doesn't allow for new urls to be publicly shortened, existing urls still work|
|?||2015-12-01||2015-04-11||255,950||http://piciurl.hu/7wr||Y||Initial scrape done on 2015-04-11; new scrape done on 2015-12-01 (up to seq# 11043)|
|?||2014-11-22||2014-11-16||15,067,550||?||?||Now part of Oracle; uses custom code|
|?||2015-04-08||2014-12-13||1,031,784,400||?||?||Still resolves URLs, but the homepage is 404; related to http://sharethis.com ; uses custom code|
|?||2014-11-06||2014-11-06||738,600||http://shrt.st/vpz||Y||Appears down; doesn't allow new urls to be shortened, existing urls still work.|
|?||2015-12-01||2014-11-06||55,800||http://srtn.us/10zd||Y||still resolves URLs, but site just shows blank page; first scrape on 2014-11-06; re-checked from seq# 47929 through 48441 on 2015-12-01|
|?||2014-12-13||2014-12-13||585,800||?||?||Doesn't make any more shorturls|
|Y||(alias: 2tu.us); appears to do case folding; uses custom code|
|?||2015-02-20||2015-01-01||3,130,300||?||?||first scrape (only covering 5-character codes) from 2015-01-01 through 2015-02-20; next scrape (starting at 0c4uk) began on 2015-12-04|
|?||2023-01-28||2015-05-02||52,891,199||?||?||Doesn't allow new shortened URLs, as of 01:57, 19 November 2015 (EST)|
|?||2023-01-28||2015-02-14||1,875,123,250||?||?||uses custom code|
|?||2014-12-18||2014-11-16||832,351,700||?||?||dead; viddy; partially saved|
|?||2014-11-16||2014-11-16||2,151,400||?||?||Requires free login to create shorturls|
|Y||Appears incremental, but custom ones also exist (up to 10 characters); requires (free) GoDaddy account to create short URLs; uses custom code|
|?||2015-01-10||2014-12-12||161,601,700||?||?||self-saved; Thank you Metamark for the database dump!|
|?||2015-01-14||2015-01-10||28,675,900||?||?||see xrl-us entry|
y-ahoo-it_5: 982,090,300 checked between 2014-11-06 and 2015-02-25
y-ahoo-it_6: 1,670,279,150 checked between 2014-11-06 and 2015-04-03
y-ahoo-it_8: 1,952,022,300 checked between 2014-11-06 and 2015-04-04 . Now dead.
|?||2014-12-13||2014-12-13||597,150||?||?||Not accepting new urls; uses custom code|
|?||2015-11-08||2014-11-16||333,450||http://yoolink.to/1dwa||Y||Up to 3 characters scraped on 2014-11-16 (a small 2 character segment re-scraped on 2014-11-22); 4 characters (up to 1dwa) scraped starting 2015-11-08|
|Warrior project name||Est. # shorturls||Last scraped date||Initially scraped date||# checked||Example URL||Incr||Comments|
(please keep list alphabetized, and list verification dates inline)
Sources include: http://blog.go2.me/2009/01/exhausting-review-of-link-shorteners.html (last updated 2009-08-14) and http://code.google.com/p/shortenurl/wiki/URLShorteningServices (Updated May 19, 2011; not fully integrated yet)
- 2.gp -- http://2.gp/GDYZ (created 21:34, 21 November 2015 (EST)) ; 4-character alphanumeric (excluding "similar looking letters"), incremental; seems to check that URL resolves
- 2.ly -- looks very similar to 2.gp (but not an alias)
- adf.ly - incremental, but displays interstitial ads, so requires custom code; Ex: http://adf.ly/bnpYL ; http://adf.ly/1RP4DP (current as of 01:38, 10 November 2015 (EST))
- adfoc.us - displays interstitial ads; appears to block curl; so requires custom code
- ask.fm - Ex: http://ask.fm/a/40k05kgp
- bc.vc - displays interstitial ads
- budurl.com - Appears non-incremental
- buff.ly - Buffer App
- cf.ly (CashFly.com)
- chilp.it -- seems to allow public creation of shorturls; as of 21:52, 21 November 2015 (EST)
- cl.ly - CloudApp
- cmt.com - Country Music Television
- cur.lv (CoinURL.com)
- decenturl.com - Not at all easy to scrape.
- del.ly - sprinklr
- df4.us - daringfireball.net
- dld.bz - "private URL shortening service"
- dlvr.it - Requires free login; then requires connecting to another service; URLs are shortened when sent through. ( as of 01:36, 2 November 2015 (EST))
- doiop.com - Appears non-incremental
- durl.me -- appears to allow public creation of shorturls; as of 22:00, 21 November 2015 (EST)
- easyurl.net - Appears non-incremental. Ex: http://easyurl.net/afd2f
- fav.me - Used by DeviantArt. Ex: http://fav.me/d31sfml
- flip.it - Flipboard
- flpbd.it - Flipboard
- fnd.us (See offical shorteners)
- fos.hu – incremental alphanumeric, but shares pattern with an image sharing service
- gkurl.us -- appears to allow public creation of shorturls; as of 22:00, 21 November 2015 (EST)
- none that start with h or i, yet.
- jdem.cz - Incremental with random (?) last digit - Ex: http://jdem.cz/bw388
- kics.it – Restricted access to shourturl creation
- lien2.com - http://lien2.com/go/a6cx ; seems-non-incremental, only 583 shortened urls (as of 02:27, 29 November 2015 (EST) )
- ln.is - linkis.com
- lnq.me - http://lnq.me/DnXtHO ; 6-character, alphanumeric, seems-non-incremental ; http://lnq.me/preview/en/ -- which seems to some kind of bizarre guessing if you put in less than 6 characters (as of 20:33, 21 November 2015 (EST) )
- mgnet.me - for torrent magnet URIs.
- moourl.com – Random
- my.dot.tk/tweak - Appears non-incremental
- nblo.gs -- no obvious way to create URLs from the home page as of 20:08, 7 November 2015 (EST)
- news.me -- no obvious way to create URLs from the home page as of 20:08, 7 November 2015 (EST)
- nohref.hu – Allows custom shorturl & deletes links after a specified time period (or 1 year without use)
- notlong.com - Appears to be alpha-only - Ex: http://yeitoo.notlong.com/ ; doesn't seem to be allow creating new shorturls, as of 20:08, 7 November 2015 (EST)
- nutshellurl.com - Appears incremental. 301s to a redirector script, which then 301s you to the destination.
- none that start with o yet.
- p.pw -- sells interstitial ads before showing the full URL; likely to be harder to scrape (as of 20:08, 7 November 2015 (EST))
- pear.ly - Used by pearltrees.com. Ex: http://pear.ly/6J1H
- pnut.co - see nutshellurl.com Ex: http://pnut.co/3a
- po.st -- "social sharing platform"; no obvious way to create URLs from the home page (as of 20:08, 7 November 2015 (EST))
- prsm.tc - getprismatic.com
- none that start with q yet.
- rod.gs - up to 3 characters, alphanumeric, creating new ones appears to hang (as of 02:14, 2 November 2015 (EST))
- sdai.ly – Allows custom shorturl
- shorl.com - Doesn't appear guessable - Ex: http://shorl.com/tisikestibahu
- shorte.st - sells interstitial ads before showing the full URL; likely to be harder to scrape
- shrinkurl.us - Still resolves, but does not allow creating new URLs ("The URL you entered was not valid or did not exist.")
- smarturl.eu / joturl.com - Doesn't appear guessable, HTML redirect.
- smarturl.it - smartURL
- soa.li - Gigya inc.
- soc.li - Gigya inc.
- spne.ws - Silicon Prairie News
- spnsr.tw - sponsoredtweets.com
- surl.co.uk - Many shortening options.
- techme.me - Techmeme
- tinyarrows.com / ta.gd / ri.ms / ➡.ws / ➨.ws / ➯.ws / ➔.ws / ➞.ws / ➽.ws / ➹.ws / ✩.ws / ✿.ws / ❥.ws / ›.ws / ⌘.ws / ‽.ws / ☁.ws - Appears non-incremental: uses user-defined words for URLs (e.g. http://➡.ws/URLTEAM)
- tiny.cc - Appears non-incremental
- totesz.hu/x – Allows custom shorturl
- trib.al -- Does not appear to allow public creation of new short-URLs; owned by SocialFlow
- twitthis.com -- requires a Twitter account to create shortURLs (as of 20:08, 7 November 2015 (EST))
- urlcut.com - "We are not currently accepting new redirects at this time." ; existing ones seem to still work, e.g. http://urlcut.com/1xvha (as of 02:09, 2 November 2015 (EST))
- usite.hu/link.php – Numeric incremental, public database
- vk.cc -- no obvious way to create URLs from the home page (as of 20:08, 7 November 2015 (EST))
- none that start with w or x yet.
- y2u.be - meant for YouTube videos
- yep.it -- allows custom shortcodes; validates provided URL; example: http://yep.it/bgnhpu ; seems non-incremental, only lowercase letters; appears to make the whole database available via: http://yep.it/stat.php?page=5719 (as of 20:08, 7 November 2015 (EST))
- none that start with z yet.
- bln.gs - Blingee (format: bln.gs/b/28fss0 and bln.gs/b/1)
- CokeURL.com - Coca-Cola (examples: CokeURL.com/3yuz9 ; CokeURL.com/vs5s ; Cokeurl.com/theaterseat )
- db.tt - Dropbox
- di.sn - Disney
- fb.me - Facebook
- flic.kr - Flickr
- fnd.us - Fundrazr.com
- fxn.ws - Fox News
- g.co - Google (used for Google products and services)
- getpocket.com/s/ - Pocket
- goo.gl - Google
- go.usa.gov - USA Government (and since they control the Internets, it doesn't get much more official than this)
- git.io - GitHub only URLs
- gty.im - Getty Images (format: gty.im/488068439; links by editorial number)
- gu.com - The Guardian (weird format - https://gu.com/p/3f7ca )
- hub.me - HubPages
- ift.tt - IFTTT
- igg.me - Indiegogo
- lnkd.in - LinkedIn
- mfi.re - MediaFire
- msft.it - Microsoft (or maybe something called "Sprinklr"?)
- mysp.ac - Myspace
- nydn.us - New York Daily News
- off365.ms - Office 365
- pocket.co - Pocket
- post.ly - Posterous
- redd.it - Reddit
- reut.rs - Reuters
- rsg.ms - Rockstar Games
- skfb.ly - Sketchfab
- spoti.fi - Spotify
- stanford.io - Stanford University
- su.pr - StumbleUpon
- sx3.se - swedishstartupspace.se
- t.co - Twitter
- ti.me - Time Magazine
- tmblr.co - Tumblr
- tw.appstore.com - Apple App Store
- uoft.me - University of Toronto
- upl.nu - Ung Pirat (Youth Pirate Party, Sweden)
- vstphl.ly - Visit Philly
- wapo.st - Washington Post
- wh.gov - White House (format: wh.gov/i3lXR)
- wp.me - Wordpress.com
- youtu.be - YouTube
- hrts.me - University of Hertfordshire. Seems to be 5 characters long. a-z with usage of capitals and non capitals. Includes numbers. Mainly used on https://twitter.com/UniofHerts
A bit.ly alias works just like a bit.ly URL. The shortcode is the same, it sets the same bit.ly cookie, and DNS resolving the address shows the IP addresses are the same as bit.ly. The homepage may be different however.
- abcn.ws - ABC News (examples: abcn.ws/1aOoijH ; abcn.ws/okiWbi )
- 1.usa.gov - USA Government
- 4sq.com - Foursquare
- aje.me - Aljazeera
- amzn.to - Amazon
- atfp.co - Foreign Policy
- bbc.in - BBC
- binged.it - Bing (bonus points for being longer than bing.com)
- bnkrpt.am - Bankrupting America
- bzfd.it - Buzzfeed
- carrot.cr - Carrot Creative
- cb.com - Career Builder
- chzb.gr - Cheezeburger
- cmplx.it - Complex Magazine
- cnet.co - CNET
- cnnmon.ie - CNN Money
- conta.cc - Constant Contact Inc.
- corb.is - Corbis Images
- cpurl.net - Current Photographer.com
- curbed.cc - Curbed.com
- dennysd.in - Denny's Restaurants
- dtoid.it - Destructoid
- econ.st - The Economist
- emarketee.rs - Emarketeers
- engri.sh - Engrish.com
- eonli.ne - E! Online
- es.pn - ESPN
- fakes.pn - The Fake ESPN (at lockerdome.com)
- fanpa.ge - Fanpage.it
- feedly.com/k/ - redirect, see below for their own
- gaw.kr - Gawker
- geekiss.im - Geekismo
- grd.to - The Grid TO
- grn.bz - GreenBiz
- gtg.lu - GetGlue
- hoblu.es - House of Blues
- hub.am - HubSpot
- huff.to - Huffington Post
- ift.tt - IFTTT
- j.mp - bit.ly
- jrnl.to - thejournal.ie
- kck.st - Kickstarter
- marsdd.it - MaRS Discovery District
- mbist.ro - MediaBistro
- mojo.ly - Mother Jones
- muo.fm - MakeUseOf
- mwne.ws - MarketWired News
- nie.mn - Neiman Journalism Lab
- nokia.ly - Nokia
- nyti.ms - New York Times
- onforb.es - Forbes
- onion.com - The Onion
- pops.ci - Popular Science
- popu.pe - Pop-Up Pantry
- propub.ca - ProPublica
- read.bi - Business Insider
- rseo.co - realseo
- s831.us - Studio831 - whatever that is
- sbn.to - sbnation
- skygrid.me - SkyGrid
- slackers.co - slackers.com
- squid.us - Laughing Squid
- s.shr.lc - shareaholic - Naive, redirects any shortcode to bit.ly
- stjo.es - St. Joseph Media
- tcrn.ch - Techcrunch
- theatln.tc - The Atlantic
- tnw.co - The Next Web
- tom.hn - Tom Hillenbrand
- toms.sh - TOMS Shoes
- tvt.ag - tvtag.com
- txpr.de - TexasStore
- unr.ly - Unruly media
- usat.ly - USA Today Newspaper
- vrge.co - The Verge
- yhoo.it - Yahoo! (not to be confused with y.ahoo.it, their non-bitly public url shortener)
- zite.to - Zite
Dead or Broken
(please keep list alphabetized)
- 1link.in - Website dead
- 6url.com - HTML redirect, Error 500
- abra.me - server down as of 21:52, 21 November 2015 (EST)
- ad.vu - mirror of adjix.com, application not found
- arm.in - domain for sale as of 21:44, 21 November 2015 (EST)
- biglnk.com - dead, replaced with unrelated blog
- bwtm.co - DNS fails to resolve.
- calyp.co - Server error. 403 - Forbidden: Access is denied.
- canurl.com - Website dead
- chod.sk - Appears non-incremental, not resolving
- come.to - (wayback of homepage) Related to various .to shorteners. Started in 1997, killed in 2013 after parent company died.
- cli.gs - server timeout as of 21:52, 21 November 2015 (EST); Appears non-incremental
- clop.in -- domain parked as of 21:52, 21 November 2015 (EST)
- coge.la -- just a logo as of 22:00, 21 November 2015 (EST)
- da.co - Parked.
- dft.ba - Server gone as of 01:11, 17 November 2015 (EST) ; site had shutdown message (claiming links would continue to work), from July 2015 through Sept 2015.
- digg.com - discontinued - 
- dwarfurl.com - Website dead/Numeric, appears incremental: http://dwarfurl.com/08041
- easy.tc - DNS not resolving.
- easyuri.com - Website dead/Appears hex incremental with last digit random/checksum: http://easyuri.com/1339f , http://easyuri.com/133a3
- eqent.me - Improper redirect to bitly.
- feedzil.la - Domain parked.
- fon.gs - server down as of 22:00, 21 November 2015 (EST)
- fwd4.me - redirects to a site about traffic cameras as of 22:00, 21 November 2015 (EST)
- go2cut.com - Website dead
- gob.li - Golbin Ridge Limited. Timed out
- gonext.org - not resolving
- go.to - sold its domains on Sedo apparently.
- go2.me - everything 404s
- gyar.eu - Server gone as of 01:07, 17 November 2015 (EST) (examples: http://gyar.eu/c8 ; http:// gyar.eu/agB)
- hashonomy.com - Timed out
- hj.to -- server requests HTTP Basic auth as of 22:05, 21 November 2015 (EST)~
- htcdev.net - DNS not resolving.
- hurl.no - Shut down sometime between March 5, 2011 and May 29, 2011 due to an influx of spam. As of JesseW 02:24, 3 December 2015 (EST), the domain appears to be owned by someone else.
- iawtp.me - DNS not resolving
- icymi.me - DNS not resolving
- iKr.me - Asian-script spam site as of 02:24, 3 December 2015 (EST)
- ilix.in - domain parked
- imfy.us - requires a recaptcha to get to the linked site, and avast goes nuts. DNS fails to resolve.
- inspr.in - Inspired Beta. Can't find server
- irt.me - DNS not resolving as of 02:24, 3 December 2015 (EST)
- ix.it - Not resolving
- jijr.com - Doesn't appear to be a shortener, now parked
- joomlagyar.hu/usb - DNS not resolving
- jump.to - dead as of February 1, 2013
- kissa.be - "Kissa.be url shortener service is shutdown"
- kl.am - "kl.am Closes its Shell"
- krz.ch - redirects to idealizer.ch (SEO company) as of 02:24, 3 December 2015 (EST)
- kuijt.nu - replaced with unrelated site
- kurl.us - Parked.
- lnkurl.com - Website dead
- marv.ly - DNS fails to resolve.
- mash.to - Cannot connect.
- memurl.com - Pronounceable. Broken.
- me.lt - Connection refused.
- mens.hm - Not responding (timeout)
- miklos.dk - Doesn't appear guessable: http://miklos.dk/!z7bA6a - "Vi arbejder på sagen..."
- minilien.com - Doesn't appear guessable: http://minilien.com/?9nyvwnA0gh - Website dead
- minim.in - Times out
- minurl.org - Presently in ERROR 404
- ms.me - Parked.
- msplinks.com - Used by Myspace
- mtw.tl - everything 403s
- muhlink.com - Not resolving
- mytinyurl.com - redirects to an unrelated image
- myurl.us - cpanel frontend
- myv.bz - Not resolving
- nyturl.com - NY Times (bonus points for being longer than nyt.com, which they own). Taken by squatters
- onvzi.com - DNS fails to resolve.
- otf.me - Empty WordPress site
- ping.fm - Fails to resolve.
- pln.so - Not working.
- plzretwt.me - Fails to resolve.
- pnt.me - Doesn't appear guessable, too big a space to bruteforce: http://pnt.me/FzAblc
- pulsene.ws - Expired. Parked by GoDaddy.
- re.ad - Fails to resolve.
- redirx.com - Lowercase alpha only, appears sequential or guessable - Ex: http://redirx.com/?wyok. Website still online but does not resolve existing URLs nor does it allow creating new ones (responds with the message: blame the spammers)
- see.sc - Fails to resolve.
- s.me - Domain parked.
- say.ly - redirects to unrelated site
- s3nt.com - Probably sequential. http://s3nt.com/aa goes somewhere different from /ab . Domain parked.
- shortlinks.co.uk - Working again. Maybe not.
- short.to - Domain is parked - Probably sequential/loweralpha: http://short.to/msmp
- shrinklink.co.uk - Doesn't appear sequential: http://www.shrinklink.co.uk/45bmx , www.shrinklink.co.uk/npk6xp . Domain parked.
- shrtn.us - myshorturls.appspot.com. 404, does not resolve
- simurl.com - Doesn't appear guessable - Ex: http://simurl.com/panpes. Website is blank; does not resolve URLs ("This SimURL is now inactive")
- smf.is - DNS not resolving.
- sns.mx - SNS Analytics, domain parked
- sq.com - Now redirects to Singapore Airlines.
- tiny.ly - DNS not resolving.
- tm.to - Twtmore has "flown away"
- to.gg - Global Giving, everything 503s
- traceurl.com - DNS fails to resolve.
- tr.im (1st generation) - "Be back soon!"
- tweetburner.com / twurl.nl - Appears incremental, everything 404s
- twixar.com - "Estamos fora do ar por algum tempo, mas estamos trabalhando para voltar a oferecer o serviço para encurtar URLs longa em breve!"
- twthpr.co - DNS not resolving.
- twitpwr.com - Domain parked.
- u.mavrev.com - Stopped accepting new urls. Now times out
- u.nu - "The shortest URLs. period." Website dead since at least 1st of october 2010 (http://web.archive.org/web/20100104023208/http://u.nu/)
- url9.com - Sequential, alphanumeric. Leading 0s are significant. "The site is working correctly."
- urlborg.com - 404 Not Found.
- urlcover.com - Domain parked.
- urlhawk.com - Domain parked.
- url-press.com - Suspended by web host.
- urlsinn.com - DNS not resolving.
- urlsmash.com - DNS not resolving.
- urltea.com - Dreamhost's coming soon page.
- urlvi.be - Domain parked.
- urlx.org - Owner has agreed to share his database
- uxp.in -
still resolves URLs, but site just shows blank page. Domain parked.
- vibemag.co - Vibe Magazine. Times out
- vsb.li / links.visibli.com/links/ - The latter uses truncated md5 hex string. See sharedby.co.
- w3t.org - 403 Forbidden.
- wlink.us - Domain parked.
- wl.tl - DNS not resolving.
- xaddr.com - Domain parked.
- xil.in - Under construction.
- x.se - Cannot resolve, but www.x.se works.
- xym.kr - Gibberish (?) Korean text blog.
- y.ahoo.it - Yahoo
- yweb.com - Suspicious iframe with long url and fake loading gif image.
- zi.ma - DNS not resolving.
- zip.sm - was a redirect to joturl.com. Now times out
- adjix.com -
Still resolves URLs, but site does not work: "The requested application was not found on this server."- Is static host on AWS service.
- feedly.com/e/ - realized that URL shorteners were bad . Non-cooperative.
- metamark.net / xrl.us - no longer allowing new urls to be shortened, existing urls still work (Ex. http://xrl.us/bfabog). Uploaded a database dump to Internet archive.
- urlbrief.com - co-operates with 301Works.org
External lists to integrate
(not yet checked or de-duplicated, please move them (and comment with at least the date) when you do check them)
From:• • •
LinksPreadeR (l.pr) lin.io 1 Linkee.com LNK.by lnk.ly ly.my Moourl.com LNK.sk lt.tl lurl.no mangk.us micURL.com min2.me minilink.org (lnk.nu) MinURL.fr MySp.in MyURL.in nbx.ch ndurl.com 2 3 nm.ly (namely) 1 omf.gd Pendek.in Pic.gd Piko.me PiURL.com Plo.cc 4 pnt.me pt2.me Puke.It qik.li qr.cx Qurl.com qux.in r.im 1 RDE.me re.p.ly (p.ly) retwt.me qik.li redir.ec RI.MS rnk.me RT.nu RubyURL.com Safe.mn 2 Sai.ly SevenRZ (7rz.de) 4 Sexy URL (sl.ly) ShadyURL.com slki dot ru (to get around spam filter) SFU.ca shorl.com 2 Short.ie short.to shorten4charity (s4c.in) 2 shortn.me shrt.ws Shrtn.com Shw.me siteo.us Smallr.net SMFU.in Snipie.com snkr.me song.ly 3 srnk.net SwU.me 4 TimesURL.at tini.us Tiny.cc TinyPl.us 2 tllg.net to.je to.ly to.vg tr.im tra.kz 3 trumpink.lt tsort.us tweet.me Tweetburner (twurl.nl) Twip.us Twirl.at twtr.us (tw6.us) 2 u.nu Uiop.Me ur.ly urizy.com (unfaker.it) URL (un)faker URLCorta.es URL.AG URL.ie URLi.nl URLoo.com urlBorg.com urlG.info URLZ.at urlShort (ooqx.com) urlShort (u.mavrev.com) urlu.ms urlzen vi.ly Virl.com 1 vl.am Voizle.com VTC.es W3T.org 1 wa.la 4 XiY.net XORTR (xrt.me) XR.com xrl.in X.vu xxsURL.de Z.PE Zapt.In Zi.ma Zi.pe Zip.li ZipMyURL.com ZZ.GD 1 using frame/toolbar 2 using preview/delay 3 for MP3 URL only 4 currently only available on dev build 5 the service has been shutdown and will be removed on the next build To be decided ir.pe These services are dead or no longer working 2 Short.Url (2su.de) 2Zeus 3.ly 301.to 307.to 9mp a.gd a.nf abbr aurls.info bloat.me Buk.me clk.my Crum.pl DiggBar Fly2.ws Foxy URL gl.am 1 Good.ly Gurl.es hao.jp hex.io Hop.im Hurl.ws idek.net J2j.de k.vu keTKP.in kissa.be! Kisa.Ch 2 kl.am Kots.Nu ktzros Lincr LinxFix Tiny.pl
From• • •
0rz.tw 1link.in 1url.com 2.gp 2big.at 2tu.us 3.ly 307.to 4ms.me 4sq.com 4url.cc 6url.com 7.ly a.gg a.nf aa.cx abcurl.net ad.vu adf.ly adjix.com afx.cc all.fuseurl.com alturl.com amzn.to ar.gy arst.ch atu.ca azc.cc b23 dot ru b2l.me bacn.me bcool.bz binged.it bit.ly bizj.us bloat.me bravo.ly bsa.ly budurl.com canurl.com chilp.it chzb.gr cl.lk cl.ly clck dot ru cli.gs cliccami.info clickthru.ca clop.in conta.cc cort.as cot.ag crks.me ctvr.us cutt.us dai.ly decenturl.com dfl8.me digbig.com digg.com disq.us dld.bz dlvr.it do.my doiop.com dopen.us easyuri.com easyurl.net eepurl.com eweri.com fa.by fav.me fb.me fbshare.me ff.im fff.to fire.to firsturl.de firsturl.net flic.kr flq.us fly2.ws fon.gs freak.to fuseurl.com fuzzy.to fwd4.me fwib.net g.ro.lt gizmo.do gl.am go.9nl.com go.ign.com go.usa.gov goo.gl goshrink.com gurl.es hex.io hiderefer.com hmm.ph href.in hsblinks.com htxt.it huff.to hulu.com hurl.me hurl.ws icanhaz.com idek.net ilix.in is.gd its.my ix.lt j.mp jijr.com kl.am klck.me krunchd.com l9k.net lat.ms liip.to liltext.com linkbee.com linkbun.ch liurl.cn ln-s.net ln-s dot ru lnk.gd lnk.ms lnkd.in lnkurl.com lru.jp lt.tl lurl.no macte.ch mash.to merky.de migre.me miniurl.com minurl.fr mke.me moby.to moourl.com mrte.ch myloc.me myurl.in n.pr nbc.co nblo.gs nn.nf not.my notlong.com nsfw.in nutshellurl.com nxy.in nyti.ms o-x.fr oc1.us om.ly omf.gd omoikane.net on.cnn.com on.mktw.net onforb.es orz.se ow.ly ping.fm pli.gs pnt.me politi.co post.ly pp.gg profile.to ptiturl.com pub.vitrue.com qlnk.net qte.me qu.tc qy.fi r.im rb6.me read.bi readthis.ca reallytinyurl.com redir.ec redirects.ca redirx.com retwt.me ri.ms rickroll.it riz.gd rt.nu ru.ly rubyurl.com rurl.org rww.tw s4c.in s7y.us safe.mn sameurl.com sdut.us shar.es shink.de shorl.com short.ie short.to shortlinks.co.uk shorturl.com shout.to show.my shrinkify.com shrinkr.com shrt.fr shrt.st shrten.com shrunkin.com simurl.com slate.me smallr.com smsh.me smurl.name sn.im snipr.com snipurl.com snurl.com sp2.ro spedr.com srnk.net srs.li starturl.com su.pr surl.co.uk surl.hu t.cn t.co t.lh.com ta.gd tbd.ly tcrn.ch tgr.me tgr.ph tighturl.com tiniuri.com tiny.cc tiny.ly tiny.pl tinylink.in tinyuri.ca tinyurl.com tk. tl.gd tmi.me tnij.org tnw.to tny.com to. to.ly togoto.us totc.us toysr.us tpm.ly tr.im tra.kz trunc.it twhub.com twirl.at twitclicks.com twitterurl.net twitterurl.org twiturl.de twurl.cc twurl.nl u.mavrev.com u.nu u76.org ub0.cc ulu.lu updating.me ur1.ca url.az url.co.uk url.ie url360.me url4.eu urlborg.com urlbrief.com urlcover.com urlcut.com urlenco.de urli.nl urls.im urlshorteningservicefortwitter.com urlx.ie urlzen.com usat.ly use.my vb.ly vgn.am vl.am vm.lc w55.de wapo.st wapurl.co.uk wipi.es wp.me x.vu xr.com xrl.in xrl.us xurl.es xurl.jp y.ahoo.it yatuc.com ye.pe yep.it yfrog.com yhoo.it yiyd.com youtu.be yuarel.com z0p.de zi.ma zi.mu zipmyurl.com zud.me zurl.ws zz.gd zzang.kr ›.ws ✩.ws ✿.ws ❥.ws ➔.ws ➞.ws ➡.ws ➨.ws ➯.ws ➹.ws ➽.ws
From• • •
Not copied in yet
From• • •
Not copied in yet
Check out Audit2014 and help audit the archives. In particular, the stuff not on Internet Archive needs to be uploaded.
- See the latest torrent release for URLs before Tinytown. A copy is available at URLTeamTorrentRelease2013July
- Tinytown results are uploaded to the Internet Archive. They are incremental, so you will need to download them all to get all URLs.
- They are formatted as follows:
- Each IA item (which can be downloaded in full via BitTorrent or as a zip file) contains multiple zip files, named
PROJECT_NAMEis the warrior project name (as listed in the table below, and on http://tracker.archiveteam.org:1337/status) and
TIMESTAMPis a dash-separated timestamp matching the name of the item.
- Each zip file contains a subdirectory matching the
PROJECT_NAME, which contains a file named
PROJECT_NAME.meta.json.xzand one or more files whose names start with one or more underscores, followed by
xzfiles can be decompressed with the wikipedia:XZ Utils.
meta.jsonfile is a copy of the project settings (as linked from the table below) at the time the dump was made.
txtfiles contain the actual URLs, in BEACON format.
- The simple description of BEACON format is that each line (except for a few header lines) consists of the shortcode, followed by a vertical bar, followed by the original URL.
- Each IA item (which can be downloaded in full via BitTorrent or as a zip file) contains multiple zip files, named
- They are formatted as follows:
Common URL shortening software
Ha-ha! Please don't run a URL shortening service.