Difference between revisions of "Project Sonar"
Jump to navigation
Jump to search
(Add size estimate) |
(Update) |
||
Line 9: | Line 9: | ||
== Datasets == | == Datasets == | ||
As of | As of 2021-06-29, the following data is available through Project Sonar: | ||
{| class="wikitable" | {| class="wikitable" | ||
Line 15: | Line 15: | ||
! Dataset !! Description !! Scan frequency !! Data range !! Available without registration !! Size !! Size growth per month | ! Dataset !! Description !! Scan frequency !! Data range !! Available without registration !! Size !! Size growth per month | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.fdns_v2/ Forward DNS (FDNS)] || "DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" || Weekly || Feb 2017 to present || Third most recent scan || | | [https://opendata.rapid7.com/sonar.fdns_v2/ Forward DNS (FDNS)] || "DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" || Weekly || Feb 2017 to present || Third most recent scan || 11.1 TiB || 393 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.rdns_v2/ Reverse DNS (RDNS)] || "DNS IPv4 PTR responses" || Weekly || Feb 2017 to present || Third most recent scan || | | [https://opendata.rapid7.com/sonar.rdns_v2/ Reverse DNS (RDNS)] || "DNS IPv4 PTR responses" || Weekly || Feb 2017 to present || Third most recent scan || 3.14 TiB || 58 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.http/ HTTP GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTP ports" || Bi-weekly || Oct 2013 to present || Second most recent scan || | | [https://opendata.rapid7.com/sonar.http/ HTTP GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTP ports" || Bi-weekly || Oct 2013 to present || Second most recent scan || 15.3 TiB || 265 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.https/ HTTPS GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTPS ports" || Bi-weekly || July 2017 to present || Second most recent scan || | | [https://opendata.rapid7.com/sonar.https/ HTTPS GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTPS ports" || Bi-weekly || July 2017 to present || Second most recent scan || 11.2 TiB || 299 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.ssl/ SSL Certificates] || "X.509 certificate metadata observed when communicating with HTTPS endpoints" || Bi-weekly || Oct 2013 to present || Second most recent scan || | | [https://opendata.rapid7.com/sonar.ssl/ SSL Certificates] || "X.509 certificate metadata observed when communicating with HTTPS endpoints" || Bi-weekly || Oct 2013 to present || Second most recent scan || 1.24 TiB || 61 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.moressl/ More SSL Certificates (non-443)] || "X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." || Bi-weekly || June 2014 to present || Second most recent scan || | | [https://opendata.rapid7.com/sonar.moressl/ More SSL Certificates (non-443)] || "X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." || Bi-weekly || June 2014 to present || Second most recent scan || 450 GiB || 14.6 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.udp/ UDP Scans] || "UDP scan results for common UDP services across all of IPv4" || Monthly || June 2014 to present || Second most recent scan || | | [https://opendata.rapid7.com/sonar.udp/ UDP Scans] || "UDP scan results for common UDP services across all of IPv4" || Monthly || June 2014 to present || Second most recent scan || 340 GiB || 1 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.tcp/ TCP Scans] || "SYN scan results for common TCP services across all of IPv4" || Weekly || Mar 2017 to present || Second most recent scan || | | [https://opendata.rapid7.com/sonar.tcp/ TCP Scans] || "SYN scan results for common TCP services across all of IPv4" || Weekly || Mar 2017 to present || Second most recent scan || 680 GiB || 19 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.national_exposure/ National Exposure Scans] || "Open port results for Rapid7's National Exposure reports" || N/A || Apr 2016 to Apr 2018 || None || 13 GiB || N/A | | [https://opendata.rapid7.com/sonar.national_exposure/ National Exposure Scans] || "Open port results for Rapid7's National Exposure reports" || N/A || Apr 2016 to Apr 2018 || None || 13 GiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/ | | [https://opendata.rapid7.com/sonar.fdns/ Forward DNS (FDNS) -- ANY 2014-2017] || "DNS 'ANY' responses for known forward DNS names from 2014-2017" || Weekly || Mar 2014 to Jan 2017 || None || 1.1 TiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar. | | [https://opendata.rapid7.com/sonar.rdns/ Reverse DNS (RDNS) -- 2013-2017] || "DNS IPv4 PTR responses from 2013-2017" || Weekly || Sept 2013 to Feb 2017 || None || 650 GiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.cio/ Critical.IO Service Fingerprints] || "The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013." || Monthly || May 2012 to Mar 2013 || None || 355 GiB || N/A | |||
| [https://opendata.rapid7.com/sonar.cio/ Critical.IO Service Fingerprints] || "The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013." || Monthly || May 2012 to Mar 2013 || None || 355 GiB || N/A | |||
|- | |- | ||
! scope="row" colspan="5"|Total | ! scope="row" colspan="5"|Total | ||
| | | 46.3 TiB | ||
| | | 1.08 TiB | ||
|} | |} | ||
(Note: The website uses the wrong units on the files. 'MB' on their list actually means MiB.) | |||
=== No longer available (lost?) === | |||
* {{URL|https://opendata.rapid7.com/heisenberg.cowrie/|Rapid7 Heisenberg Cloud Honeypot cowrie Logs}} from Nov 2016. A single file of 520 MiB was available as of 2018-11-13 and has vanished as of 2021-06-29. The file was publicly accessible but is not available in the Wayback Machine and presumed {{lost}}. |
Revision as of 22:01, 29 June 2021
Project Sonar | |
URL | Description[IA•Wcite•.today•MemWeb] Datasets[IA•Wcite•.today•MemWeb] |
Status | Online! |
Archiving status | Not saved yet |
Archiving type | Unknown |
IRC channel | #archiveteam-bs (on hackint) |
Project Sonar by security company Rapid7 is a collection of internet-wide network scans with the aim of investigating vulnerabilities. The data is available for free, though many files (historical and most current ones) require an account.
Datasets
As of 2021-06-29, the following data is available through Project Sonar:
Dataset | Description | Scan frequency | Data range | Available without registration | Size | Size growth per month |
---|---|---|---|---|---|---|
Forward DNS (FDNS) | "DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" | Weekly | Feb 2017 to present | Third most recent scan | 11.1 TiB | 393 GiB |
Reverse DNS (RDNS) | "DNS IPv4 PTR responses" | Weekly | Feb 2017 to present | Third most recent scan | 3.14 TiB | 58 GiB |
HTTP GET Responses | "Responses to HTTP/1.1 GET requests against various HTTP ports" | Bi-weekly | Oct 2013 to present | Second most recent scan | 15.3 TiB | 265 GiB |
HTTPS GET Responses | "Responses to HTTP/1.1 GET requests against various HTTPS ports" | Bi-weekly | July 2017 to present | Second most recent scan | 11.2 TiB | 299 GiB |
SSL Certificates | "X.509 certificate metadata observed when communicating with HTTPS endpoints" | Bi-weekly | Oct 2013 to present | Second most recent scan | 1.24 TiB | 61 GiB |
More SSL Certificates (non-443) | "X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." | Bi-weekly | June 2014 to present | Second most recent scan | 450 GiB | 14.6 GiB |
UDP Scans | "UDP scan results for common UDP services across all of IPv4" | Monthly | June 2014 to present | Second most recent scan | 340 GiB | 1 GiB |
TCP Scans | "SYN scan results for common TCP services across all of IPv4" | Weekly | Mar 2017 to present | Second most recent scan | 680 GiB | 19 GiB |
National Exposure Scans | "Open port results for Rapid7's National Exposure reports" | N/A | Apr 2016 to Apr 2018 | None | 13 GiB | N/A |
Forward DNS (FDNS) -- ANY 2014-2017 | "DNS 'ANY' responses for known forward DNS names from 2014-2017" | Weekly | Mar 2014 to Jan 2017 | None | 1.1 TiB | N/A |
Reverse DNS (RDNS) -- 2013-2017 | "DNS IPv4 PTR responses from 2013-2017" | Weekly | Sept 2013 to Feb 2017 | None | 650 GiB | N/A |
Critical.IO Service Fingerprints | "The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013." | Monthly | May 2012 to Mar 2013 | None | 355 GiB | N/A |
Total | 46.3 TiB | 1.08 TiB |
(Note: The website uses the wrong units on the files. 'MB' on their list actually means MiB.)
No longer available (lost?)
- Rapid7 Heisenberg Cloud Honeypot cowrie Logs[IA•Wcite•.today•MemWeb] from Nov 2016. A single file of 520 MiB was available as of 2018-11-13 and has vanished as of 2021-06-29. The file was publicly accessible but is not available in the Wayback Machine and presumed Lost.