Difference between revisions of "Project Sonar"
Jump to navigation
Jump to search
(Created page with "{{Infobox project | title = Project Sonar | URL = {{URL|https://www.rapid7.com/research/project-sonar/|Description}}<br />{{URL|https://opendata.rapid7.com/|Datasets}} | proje...") |
(Add size estimate) |
||
Line 9: | Line 9: | ||
== Datasets == | == Datasets == | ||
As of | As of 2018-11-13, the following data is available through Project Sonar: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
! Dataset !! Description !! Scan frequency !! Data range !! Available without registration | ! Dataset !! Description !! Scan frequency !! Data range !! Available without registration !! Size !! Size growth per month | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.fdns_v2/ Forward DNS (FDNS)] || "DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" || Weekly || Feb 2017 to present || Third most recent scan | | [https://opendata.rapid7.com/sonar.fdns_v2/ Forward DNS (FDNS)] || "DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" || Weekly || Feb 2017 to present || Third most recent scan || 2.8 TiB || 220 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.rdns_v2/ Reverse DNS (RDNS)] || "DNS IPv4 PTR responses" || Weekly || Feb 2017 to present || Third most recent scan | | [https://opendata.rapid7.com/sonar.rdns_v2/ Reverse DNS (RDNS)] || "DNS IPv4 PTR responses" || Weekly || Feb 2017 to present || Third most recent scan || 1.6 TiB || 90 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.http/ HTTP GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTP ports" || Bi-weekly || Oct 2013 to present || Second most recent scan | | [https://opendata.rapid7.com/sonar.http/ HTTP GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTP ports" || Bi-weekly || Oct 2013 to present || Second most recent scan || 8.6 TiB || 155 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.https/ HTTPS GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTPS ports" || Bi-weekly || July 2017 to present || Second most recent scan | | [https://opendata.rapid7.com/sonar.https/ HTTPS GET Responses] || "Responses to HTTP/1.1 GET requests against various HTTPS ports" || Bi-weekly || July 2017 to present || Second most recent scan || 4.2 TiB || 125 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.ssl/ SSL Certificates] || "X.509 certificate metadata observed when communicating with HTTPS endpoints" || Bi-weekly || Oct 2013 to present || Second most recent scan | | [https://opendata.rapid7.com/sonar.ssl/ SSL Certificates] || "X.509 certificate metadata observed when communicating with HTTPS endpoints" || Bi-weekly || Oct 2013 to present || Second most recent scan || 380 GiB || 9 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.moressl/ More SSL Certificates (non-443)] || "X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." || Bi-weekly || June 2014 to present || Second most recent scan | | [https://opendata.rapid7.com/sonar.moressl/ More SSL Certificates (non-443)] || "X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." || Bi-weekly || June 2014 to present || Second most recent scan || 130 GiB || 5 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.udp/ UDP Scans] || "UDP scan results for common UDP services across all of IPv4" || Monthly || June 2014 to present || Second most recent scan | | [https://opendata.rapid7.com/sonar.udp/ UDP Scans] || "UDP scan results for common UDP services across all of IPv4" || Monthly || June 2014 to present || Second most recent scan || 295 GiB || 2 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.tcp/ TCP Scans] || "SYN scan results for common TCP services across all of IPv4" || Weekly || Mar 2017 to present || Second most recent scan | | [https://opendata.rapid7.com/sonar.tcp/ TCP Scans] || "SYN scan results for common TCP services across all of IPv4" || Weekly || Mar 2017 to present || Second most recent scan || 140 GiB || 11 GiB | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.national_exposure/ National Exposure Scans] || "Open port results for Rapid7's National Exposure reports" || N/A || Apr 2016 to Apr 2018 || None | | [https://opendata.rapid7.com/sonar.national_exposure/ National Exposure Scans] || "Open port results for Rapid7's National Exposure reports" || N/A || Apr 2016 to Apr 2018 || None || 13 GiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/heisenberg.cowrie/ Rapid7 Heisenberg Cloud Honeypot cowrie Logs] || "Rapid7 Heisenberg Cloud Honeypot cowrie Logs" || N/A || Nov 2016 || None | | [https://opendata.rapid7.com/heisenberg.cowrie/ Rapid7 Heisenberg Cloud Honeypot cowrie Logs] || "Rapid7 Heisenberg Cloud Honeypot cowrie Logs" || N/A || Nov 2016 || None || 520 MiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.fdns/ Forward DNS (FDNS) -- ANY 2014-2017] || "DNS 'ANY' responses for known forward DNS names from 2014-2017" || Weekly || Mar 2014 to Jan 2017 || None | | [https://opendata.rapid7.com/sonar.fdns/ Forward DNS (FDNS) -- ANY 2014-2017] || "DNS 'ANY' responses for known forward DNS names from 2014-2017" || Weekly || Mar 2014 to Jan 2017 || None || 1.1 TiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.rdns/ Reverse DNS (RDNS) -- 2013-2017] || "DNS IPv4 PTR responses from 2013-2017" || Weekly || Sept 2013 to Feb 2017 || None | | [https://opendata.rapid7.com/sonar.rdns/ Reverse DNS (RDNS) -- 2013-2017] || "DNS IPv4 PTR responses from 2013-2017" || Weekly || Sept 2013 to Feb 2017 || None || 650 GiB || N/A | ||
|- | |- | ||
| [https://opendata.rapid7.com/sonar.cio/ Critical.IO Service Fingerprints] || "The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013." || Monthly || May 2012 to Mar 2013 || None | | [https://opendata.rapid7.com/sonar.cio/ Critical.IO Service Fingerprints] || "The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013." || Monthly || May 2012 to Mar 2013 || None || 355 GiB || N/A | ||
|- | |||
! scope="row" colspan="5"|Total | |||
| 20.2 TiB | |||
| 620 GiB | |||
|} | |} |
Revision as of 19:16, 13 November 2018
Project Sonar | |
URL | Description[IA•Wcite•.today•MemWeb] Datasets[IA•Wcite•.today•MemWeb] |
Status | Online! |
Archiving status | Not saved yet |
Archiving type | Unknown |
IRC channel | #archiveteam-bs (on hackint) |
Project Sonar by security company Rapid7 is a collection of internet-wide network scans with the aim of investigating vulnerabilities. The data is available for free, though many files (historical and most current ones) require an account.
Datasets
As of 2018-11-13, the following data is available through Project Sonar:
Dataset | Description | Scan frequency | Data range | Available without registration | Size | Size growth per month |
---|---|---|---|---|---|---|
Forward DNS (FDNS) | "DNS 'ANY', 'A', 'AAAA', 'TXT', 'MX', and 'CNAME' responses for known forward DNS names" | Weekly | Feb 2017 to present | Third most recent scan | 2.8 TiB | 220 GiB |
Reverse DNS (RDNS) | "DNS IPv4 PTR responses" | Weekly | Feb 2017 to present | Third most recent scan | 1.6 TiB | 90 GiB |
HTTP GET Responses | "Responses to HTTP/1.1 GET requests against various HTTP ports" | Bi-weekly | Oct 2013 to present | Second most recent scan | 8.6 TiB | 155 GiB |
HTTPS GET Responses | "Responses to HTTP/1.1 GET requests against various HTTPS ports" | Bi-weekly | July 2017 to present | Second most recent scan | 4.2 TiB | 125 GiB |
SSL Certificates | "X.509 certificate metadata observed when communicating with HTTPS endpoints" | Bi-weekly | Oct 2013 to present | Second most recent scan | 380 GiB | 9 GiB |
More SSL Certificates (non-443) | "X.509 certificate metadata observed when communicating with miscellaneous non-HTTPS endpoints, such as IMAPS, POP3S, etc." | Bi-weekly | June 2014 to present | Second most recent scan | 130 GiB | 5 GiB |
UDP Scans | "UDP scan results for common UDP services across all of IPv4" | Monthly | June 2014 to present | Second most recent scan | 295 GiB | 2 GiB |
TCP Scans | "SYN scan results for common TCP services across all of IPv4" | Weekly | Mar 2017 to present | Second most recent scan | 140 GiB | 11 GiB |
National Exposure Scans | "Open port results for Rapid7's National Exposure reports" | N/A | Apr 2016 to Apr 2018 | None | 13 GiB | N/A |
Rapid7 Heisenberg Cloud Honeypot cowrie Logs | "Rapid7 Heisenberg Cloud Honeypot cowrie Logs" | N/A | Nov 2016 | None | 520 MiB | N/A |
Forward DNS (FDNS) -- ANY 2014-2017 | "DNS 'ANY' responses for known forward DNS names from 2014-2017" | Weekly | Mar 2014 to Jan 2017 | None | 1.1 TiB | N/A |
Reverse DNS (RDNS) -- 2013-2017 | "DNS IPv4 PTR responses from 2013-2017" | Weekly | Sept 2013 to Feb 2017 | None | 650 GiB | N/A |
Critical.IO Service Fingerprints | "The Critical.IO project was designed to uncover large-scale vulnerabilities on the global IPv4 internet and scanned a number of ports across between May 2012 and March 2013." | Monthly | May 2012 to Mar 2013 | None | 355 GiB | N/A |
Total | 20.2 TiB | 620 GiB |