DNS History
| DNS History | |
| |
 "Here at DNS History we have been crawling DNS records since 2009, our database currently contains over 500 million domains and over 2 billion DNS records." | |
| URL | dnshistory.org[IA•Wcite•.today] |
| Status | Endangered |
| Archiving status | Not saved yet |
| Archiving type | Unknown |
| Project source | dnshistory-grab |
| Project tracker | dnshistory |
| IRC channel | #archiveteam-bs (on hackint) (formerly #greatlookup (on EFnet)) |
DNS History is a DNS record archive provided by 8086 Consultancy[IA•Wcite•.today].
It was scheduled to close on 2016-07-10 due to "funding issues".
Announcement
According to a Twitter entry[IA•Wcite•.today] dated 2016-06-26:
- "dnshistory.org will be closing down in a couple of weeks - it has been a fun project over the years thanks folks."
Also, on the main page[IA•Wcite•.today], on the right:
- "This site will shut down on the 10th July 2016 due to funding issues."
Final announcement
On the main page, appeared on 2016-08-23:
- "DNS History has now shutdown, any updates are for my personal requirements only - the site may go up/down as I need the resources it uses. The number of servers dedicated to the DNS History project is now ZERO. Due to the ArchiveTeams self-righteous attitude CloudFlares DDoS protection has been enabled. They are quite open about their attitude of ignoring robots.txt files and work around blocks on their user agent - this as abuse."
Discovery results
Out of the 1365 TLDs,
Archiving
After ArchiveTeam started the project on 2016-07-04, dnshistory.org staff shortly activated CloudFlare in order to block our access to the site.
bzc6p contacted them asking for cooperation, but they replied[IA•Wcite•.today] that "there is no way you can archive the >billion pages on the site and trying to do so is causing issues for other users on the site."
Later CloudFlare seemed to have been deactivated, but the site was broken for a month, giving 500 errors. However, as of 2016-08-17, the site is up, with various queries, including search, still functioning, with the closure notice still up.
On 2016-08-20, ArchiveTeam restarted its efforts. On 2016-08-21, although the site seemed to deal with the load, and the site has officially been closed for 42 days, they started to ban our crawlers. On 2016-08-22, they deployed their impenetrable Cloudflare browser checker again.
On 2016-08-23, a final shutdown notice appeared (see above).
ArchiveTeam was hardly able to save anything.
Since at least 2018, the site has had numerous prolonged downtimes returning blank HTTP 500 responses on all pages except the homepage.
Updates
As of late 2025, the website is still operating and still behind "Attack Mode" Cloudflare. Old data is available.
It also seems to have a custom hotlinking/crawling protection: requesting a page not adjacent to one you've already seen (session cookie, probably... IP doesn't matter) redirects you to page 1.
Site header says "Please be patient as the site returns from a long absence." for a couple of years already.
Some owner's statements from Discord:
- Size on 7/12/2025 (M/D/Y): "About 3.8TB (8.8TB uncompressed) for ~3.3 billion documents (~50 billion rows in SQL assuming 1 row = 1 DNS record)."
- API planned for ages but not implemented ever.
- "There are lots of gaps from 2016 to ~2023 as I didn't have the £ to fund it."
- No possibility of bulk DB export "due to the T&C of my data sources"
- "I switched to mongodb many years ago"