1. Vulnerability Information

https://cvedb.shodan.io

The CVEDB website lets you explore known vulnerabilities and provides a free API to quickly get vulnerability information. The API returns all the usual information (CVSS, references, summary) as well as the EPSS score and whether it's in the CISA KEV (Known Exploited Vulnerabilities) catalog.

Here's a sample JSON response with some fields truncated:

{
    cve_id: "CVE-2019-1653",
    summary: "A vulnerability in the web-based management interface of...",
    cvss: 7.5,
    cvss_version: 3,
    cvss_v2: 5,
    cvss_v3: 7.5,
    epss: 0.97564,
    ranking_epss: 1,
    kev: true,
    propose_action: "Cisco Small Business RV320 and RV325 Dual Gigabit WAN ...",
    ransomware_campaign: "Unknown",
    references: [
        "http://packetstormsecurity.com/files/152260/Cisco-RV320-Unauthenticated-Configuration-Export.html",
        "http://packetstormsecurity.com/files/152261/Cisco-RV320-Unauthenticated-Diagnosti..."
    ],
    published_time: "2019-01-24T16:29:00",
    cpes: [
    "cpe:2.3:o:cisco:rv320_firmware:1.4.2.15",
        "cpe:2.3:o:cisco:rv320_firmware:1.4.2.17",
        "cpe:2.3:o:cisco:rv325_firmware:1.4.2.15",
        "cpe:2.3:o:cisco:rv325_firmware:1.4.2.17"
    ]
}

2. IP Enrichment to see Open Ports

https://internetdb.shodan.io

The InternetDB API lets you do basic IP enrichment for free. It returns information about open ports, tags, hostnames, cpes and potential vulnerabilities. We also offer a companion tool called nrich that uses the InternetDB to let you enrich IPs within a file:

Here's a sample JSON response from the API:

{
    "ip": "51.83.59.99",
    "ports": [
        22,
        80,
        443,
        500
    ],
    "cpes": [
        "cpe:/a:f5:nginx",
        "cpe:/a:openbsd:openssh:7.4"
    ],
    "hostnames": [
        "www.sampleresponse.fr"
    ],
    "tags": [
        "vpn"
    ],
    "vulns": [
        "CVE-2017-15906"
    ]
}

3. Geographic Network Tools

https://geonet.shodan.io

Geonet lets you ping an IP or do DNS lookups from multiple places around the world. It's helpful to find all IPs for websites that do geographic loadbalancing or check for potential connectivity issues depending on location.

We provide the geoping and geodns tools that make it easy to use Geonet without having to write any code.

Here's a sample JSON response for a ping request to "twitter.com":

{
    ip: "104.244.42.1",
    is_alive: true,
    min_rtt: 38.108,
    avg_rtt: 38.514,
    max_rtt: 38.961,
    rtts: [
        38.96141052246094,
        38.47217559814453,
        38.107872009277344
    ],
    packets_sent: 3,
    packets_received: 3,
    packet_loss: 0,
    from_loc: {
        city: "Santa Clara",
        country: "US",
        latlon: "37.3924,-121.9623"
    }
}

4. Information about Public Companies

https://entitydb.shodan.io

The EntityDB website and API lets you browse financial information for public companies based on SEC filings. We also associate them with known domains and hostnames which can be used to tie the company to IP/ DNS data.

5. Browser Plugins

Firefox: https://addons.mozilla.org/en-US/firefox/addon/shodan-addon/

Chrome: https://chromewebstore.google.com/detail/shodan/jjalcfnidlmpjhdfepjhjbhnhkbgleap

The browser plugins for Chrome and Firefox automatically lookup the open ports for the website you're visiting using the InternetDB API. You can click through to see the full information that Shodan has for the IP or learn more about the known subdomains (ex. mozilla.org).

Bonus: Sh_d_n

https://shdn.io

Sh_d_n is a free, lightweight website for IP and domain enrichment. It's optimized for performance and size to focus on just doing one thing: fast lookups for specific resources (IPs and domains). The website is powered by Rust, Axum and the SQLite datasets provided by Shodan Enterprise. We're excited to have reduced the size for most of the pages on this website to less than 10kb, including the CSS stylesheet. If you have the stylesheet cached on the browser then most pages are less than 1 kb!

Waiting for the Membership Sale?

Every year we also run a special lifetime Membership promotion where we offer it for $5. If you're on a tight budget or are looking to get started with Shodan then you can wait until that sale happens. Follow us on Mastodon to get notified when the Membership sale goes live:

https://mastodon.shodan.io

They typically contain the logo of the company which gives them 2 functions:

• An easy way to find the tab of a website when you have multiple open tabs.
• A sense of authenticity that the website you're visiting belongs to the right company.

Shodan collects the favicons for websites and stores the information in the http.favicon property:

{
        "data": "AAABAAIAEBAAAAAAIABoBAAAJgAAACAgAAAAACAAqBAAAI4EAAAoAAAAEAAAACAAAAABACAAAAAA\nAEAEAAAAAAAAAAAAAAAAAAAAAAAA////Af///wH///8B////Af///wH///8B////ASWX/RUne/MX\n////Af///wH///8B////Af///wH///8B////Af///wH///8B////Af///wH///8B////ASan/EEn\nbvThJm/05yai/EP///8B////Af///wH///8B////Af///wH///8B////Af///wH///8BJqP8Ayaj\n/I0mi/v5J1Tt/SdT7Psmi/z5JqL8j////wH///8B////Af///wH///8B////...",
        "hash": 516963061,
        "location": "https://about.gitlab.com:443/ico/favicon.ico"
}

• data contains the image as a base64-encoded string.

• hash is the MurmurHash3 of the data property. The Shodan API has a search filter called http.favicon.hash to search based on this value.

• location lets you know where the favicon was found. Historically, the favicon.ico file was located at the root of the web server but it can be put in any arbitrary location by referencing it in the HTML. For example:

  html <link rel="icon" type="image/png" href="/assets/favicon-yellow-018213ceb87b472388095d0264be5b4319ef47471dacea03c83ecc233ced2fd5.png" />

At Shodan, we developed the technique of hashing the favicon to make it possible to search across the Internet for identical favicon images. We developed it nearly a decade ago to help with 2 use cases:

• Identify Phishing Websites: bad actors will commonly use the same favicon as the website they're imitating. By searching for the favicon of a company you can identify potential phishing websites.
• Origin IP Disclosure: websites that are hosted behind a CDN (ex. Cloudflare) should restrict access to their web server to only accept connections from the CDN. By searching for the favicon of a website you can confirm that a website has been correctly configured and isn't responding to requests from its origin IP.

The favicon hash is calculated by applying the MurmurHash3 algorithm to the http.favicon.data property on the banner.

Why MMH3? The key considerations when we developed the technique were speed of the hashing algorithm and size of the resulting hash. We didn't need the cryptographic guarantees of MD5/ etc.

favscan

We provide a simple tool called favscan that calculates the favicon hash given a URL, hostname or local file path.

$ favscan -h
Calculate the favicon hash of a local file, hostname or URL

Usage: favscan [OPTIONS] <LOCATION>

Arguments:  
  <LOCATION>  

Options:  
  -v, --verbose  
  -h, --help     Print help
  -V, --version  Print version

favscan will first look for the favicon in the common /favicon.ico path and if that fails it will check the frontpage for a shortcut icon link. The tool is available for download across many platforms:

• Linux
• Windows
• DEB package
• RPM package
• Mac (arm64)

For example, to get the favicon hash for google.com you would run:

favscan google.com  

You can also specify ports as part of the URL:

favscan https://test.shodan.io:6993  

Or calculate it for a local file:

favscan favicon.ico  

Example

Lets say we want to find public instances of Gitlab using favicons. We start off by grabbing the favicon hash of a known Gitlab instance:

$ favscan gitlab.com
1265477436  

We then take that hash and use it in a search query of:

http.favicon.hash:1265477436  

The search query can be used on the website, CLI or API. For now, lets just see how many instances there are based on the favicon:

$ shodan count http.favicon.hash:1265477436
29558  

And this is what it looks like on the website:

https://www.shodan.io/search/report?query=http.favicon.hash%3A1265477436

Note: Shodan already fingerprints Gitlab services so you can search for product:gitlab instead of using favicons.

References

• Wikipedia article on the favicon: https://en.wikipedia.org/wiki/Favicon
• Map of Favicons on the Internet: https://faviconmap.shodan.io
• Datapedia entry for http.favicon: https://datapedia.shodan.io/property/http.html#HttpFavicon

Web Technologies

Web technologies are now grouped by categories and we show version information (if available). The information was always grouped in the underlying JSON and we now also show it that way on the website. Learn more about the http property

Hostnames

They're now sorted and the top-level domain is highlighted. This should make it easier to read the different subdomains that an IP is responsible for. Note that for forward DNS information you would want to use the DNSDB - the IP information page relies on reverse DNS or certificate information for the hostnames property.

CVSS

The list of vulnerabilities is now sorted and shows the CVSS score with a color code that ranges from black (low score) to red (high score). Learn more about the vulns property.

Icons in the Raw Data

The raw data page lets you browse the JSON as it's returned by the API and we now show icons to make navigating it easier:

• If the service has SSL/ TLS information then there is a lock icon. This includes services that allow STARTTLS to upgrade a connection to TLS.
• If it's a web service then we add a globe icon and a red arrow to open the website in a new browser tab.

To learn more about the data that is available on the banners check out the Datapedia.

Up until now, the only option to download the information was to click on the Export button in the table:

We're happy to announce that Shodan Trends now has a developer API that you can access with your Shodan API key. The API is documented in a Postman collection which you can check out here:

https://www.postman.com/shodanhq/workspace/shodan/documentation/27930959-002d7e02-f109-4e83-9a3e-74d2d7996687

The Shodan CLI has also been updated with a new shodan trends command so you can take advantage of the API without having to write any code. For example, the following command shows the total number of industrial control systems over time:

shodan trends tag:ics  

The optional facet lets you get a breakdown over time for a specific property. For example, a breakdown of the top countries that have exposed industrial control systems:

shodan trends --facets country tag:ics  

By default, the API will return the top 5 values for the property. However, you can tell the API to return more than 5. For example, the following gives a breakdown of the top 20 ports that have been seen on the 198.20.69.0/24 network over the past few years:

shodan trends --facets port:20 net:198.20.69.0/24  

The results can be saved to a file using the -S option (or -O <filename>) which will generate a gzip-compressed JSON file where each line contains a JSON object representing 1 month of information. For example, the following is the JSON object for July 2023 for the command shodan trends --facets country:10 tag:ics:

{"month": "2023-07", "count": 89333, "facets": {"country": [{"count": 25835, "value": "US"}, {"count": 5504, "value": "IT"}, {"count": 5305, "value": "ES"}, {"count": 4105, "value": "CA"}, {"count": 3697, "value": "FR"}, {"count": 3510, "value": "KR"}, {"count": 3163, "value": "DE"}, {"count": 3131, "value": "TR"}, {"count": 2506, "value": "RU"}, {"count": 2352, "value": "SE"}]}}

There are a few technical considerations:

1. You can only request information about 1 facet at a time.
2. The API calls can take a while to complete (at least several seconds).
3. Data is only indexed going back to 2017. We have historical data that goes back farther but it's not yet indexed.

That being said, the API is available to all members so check it out if you're looking for a way to programmatically see how the Internet has changed over the past few years.

Background

Shodan is a website aimed at technical users and organizations. We have nearly 5 million registered users which makes Shodan one of the larger security-related websites. Around 8 years ago we started getting emails asking us to accept Bitcoin or other cryptocurrencies:

Is there any other way to make a payment other then Paypal.... I have btc or credit card ,but no paypal

Back then we only accepted PayPal so this wasn't an uncommon request. Most of them asked for credit card support but a handful also kept asking us to accept Bitcoin as a form of payment. Eventually we decided to use Stripe to accept credit card payments for the Membership. And shortly thereafter Stripe started experimenting with Bitcoin support. We jumped on that opportunity to now accept both credit cards and Bitcoin. The excitement was short-lived though as we would soon realize that nobody actually paid in crypto. We followed-up with the users that had asked for this and we'd sometimes get a "thanks" but no actual payment. There was interest in having Shodan accept cryptocurrencies as payment but there wasn't any intent on actually making a purchase with crypto. We ended up removing support for Bitcoin after leaving it running for a few months and only getting 1-2 payments.

Second Round

Flash forward a few years and we decided to re-enable cryptocurrency support as there were more options available now to accept crypto as a vendor and the ecosystem had been around for longer. It was time to see whether more people were using cryptocurrency as a form of payment. The real test for it though would happen during our once-a-year sales event for our Membership. It's a big 24 hours for us and this year was no different. We had 220 cryptocurrency transactions which made up <1% of our total transactions. The assets used for payment are broken down as follows:

This means that based on our experience Bitcoin is the most popular choice, followed by Ethereum and Dogecoin. I didn't expect Dogecoin to be be that popular and for it to be so close to Ethereum. Of those 220 transactions, 6 of them were underpayments due to volatility in the underlying asset but they were close enough that we accepted them as a payment anyways.

• Scams Everywhere: every time somebody tweeted a question at our account there would inevitably be fake Coinbase accounts pretending to be customer service. We don't see this for payments made with Stripe and we never saw it for PayPal transactions either. We've also seen cryptocurrency scams posted as comments in our social media posts.
• Poor Developer Experience: Coinbase didn't let us specify a common secret or pass a variable to them to pre-fill a field (ex. email address). This resulted in many users not getting upgraded automatically. They had to email us and then we could manually match up their Coinbase payment with their Shodan account.
• Sending Money to the Wrong Address: we had several users that sent a payment to the wrong location. They thought the money was sent to the address provided by Coinbase but they made an error somewhere and the money never arrived in our account.

As a result of the above, we received a disproportionate amount of support emails for cryptocurrency transactions which put undue burden on our support team. It's clear that from our perspective crypto is still not widely used as a form of payment. We once again disabled it on the website and maybe we'll try it again in another 10 years.

There are 2 parts of the Shodan API: REST and Streaming.

• The REST API lets you do IP lookups, run search queries and setup network monitoring among other things. Most of Shodan's capabilities are accessible via the REST API. You send a request to the REST API and you get a response.
• The Streaming API (aka Firehose) is much smaller and provides methods to get real-time streams of data. You connect to a streaming method and then you get a never-ending stream of data until you disconnect from the Streaming API.

Shodan actually uses the Streaming API to build the REST API and send out notifications in Monitor. Here is an overview of our architecture:

The Shodan crawlers collect data, publish it to the Streaming API, the REST API consumes the Streaming API to store/ index the data and the websites use both APIs to access the data. Shodan is an API-first company and every website is built on-top of the same public API that you have access to.

We've now added the ability to create custom data feeds from search queries using the new /shodan/custom method of the Streaming API:

There is a difference though between the search query syntax of the REST API/ website and of this new custom Streaming API method: streaming queries are case-sensitive. Otherwise you should be able to take your existing search query, plug it into the Streaming API and get a real-time data feed.

The Shodan CLI has been updated with a new --custom-filters option to run streaming queries. Here are a few examples:

• Compromised services in the US:

shodan stream --custom-filters "tag:compromised country:US"  

• Elastic or Kubernetes services deployed on Google Cloud:

shodan stream --custom-filters 'org:"Google LLC" product:Elastic,Kubernetes'  

• Industrial control systems in Germany, Switzerland or France:

shodan stream --custom-filters "tag:ics country:CH,DE,FR"  

Streaming queries are an efficient way to keep track of new results and lets you focus on the data that is relevant for your use case.

https://internetdb.shodan.io

The following properties are currently provided by InternetDB:

• Open ports
• Vulnerabilities
• Hostnames
• CPEs
• Tags

And this is how a sample response looks like:

{
    "cpes": [
        "cpe:/a:varnish-cache:varnish"
    ],
    "hostnames": [],
    "ip": "151.101.41.140",
    "ports": [
        80,
        443
    ],
    "tags": [
        "cdn"
    ],
    "vulns": []
}

The major differences between the InternetDB API and the main Shodan API are:

• No API key required
• Much higher rate limit
• Weekly updates
• Minimal port/ service information
• Free for non-commercial use: you can use it at a company but you can't use it to build commercial products that you charge money for

If you'd like to get started doing fast IP lookups without writing any code then there are a few tools that can help you out:

• nrich: official Shodan tool to quickly lookup open ports/ vulnerabilities for a list of IPs. Stay tuned for a separate article on nrich.
• sdlookup: similar to nrich but written in Go and uses a different output format
• ShodanBot: a Discord bot to help you lookup IP information
• iOS/ Mac shortcut: easy way to check the open ports/ vulnerabilities from a Mac shortcut/ sheet

Internally, we're using InternetDB to quickly enrich port information for domains (ex. Twitter DNS information) among other things. If you decide to integrate InternetDB API let us know as we plan on having a page highlighting the available integrations.

Over the past few years, we've increasingly seen websites behave differently depending on where Shodan connects from (Shodan scans a few hundred million hostnames each month) and internally we've had some tooling to help debug location-based issues. We're now making some of that available to you as a free API and accompanying CLI tools:

GeoNet API: https://geonet.shodan.io

The API currently provides 2 main commands:

• geodns
• geoping

The geodns method will perform a DNS lookup and geoping does an ICMP ping request. Both of those methods are run from multiple locations around the world and the API aggregates the results into a single API response to you.

Alongside the API, we're releasing accompanying CLI tools if you want to use the geoping and geodns methods directly from your terminal without having to write any code. We provide pre-built releases for Linux. For example, this is how the geoping command works on the CLI:

The API is free to use but subject to a rate limit of 1 request per second. Talk to us if you need to do a higher volume of requests.

• CDNs
• Honeypots
• SSH servers
• Webcams
• A ton of services running on non-standard ports

You could start modifying the search query to remove all the other services but then you're still faced with the issue that:

a) If Elastic changes its banner then your query won't work anymore.
b) You will miss out on Elastic clusters that are running on non-standard ports.

The better solution is to recognize that Shodan is able to fingerprint Elastic and sets the product property to Elastic in those cases. As a result, you can simply search for product:Elastic to get all public clusters across all the ports. And now that we have a good search query we can see how those results have changed over time to identify whether the exposure of Elastic clusters has improved or not:

In general, if you only look at the number of open ports across the Internet you often get wildly inaccurate numbers. For example, here is how the results for port:102 have looked over time:

Port 102 is commonly used by the Siemens S7 industrial protocol and most likely what you're interested in when searching on that port. Based on the above image you might think there was a surge in Internet-connected industrial control systems. However, if you add the tag:ics term to filter for services that were identified as industrial protocols then you'll see a very different picture:

Shodan is good about identifying services on unexpected ports so whenever possible you should leverage the product, os, tag and other protocol-specific properties (http, ssl, etc.) to narrow down results. This will help you get a better understanding of how something is used across the Internet regardless of whether it was left with its default settings or whether the end-user configured it to listen on a different port.

https://trends.shodan.io

At Shodan, we've always kept a full history for every IP that we've ever seen on the Internet. You can look at the history for individual IPs using the new website or the API/ CLI. However, we never indexed that historical data in the search engine so you couldn't see how results have changed over time unless you wrote your own scripts to periodically query Shodan. With Shodan Trends you can now search the historical data to answer large-scale questions about the Internet. And we're making this new feature available to all members at no additional cost (see FAQ below).

Technology Trends

Shodan Trends shines when looking up the history of more complex queries but you can just as easily also get a breakdown of web server software. It turns out that in 2021, nginx overtook Apache as the most popular web server software on the Internet.

And of course we can do the same for other software as well. For example, here is a breakdown of FTP software:

The sharp increase in 2018 for Pure-FTPd is due to GoDaddy's use of the software.

We can also detect protocol-level trends such as the decline in Telnet use across the Internet:

Telnet has seen a 33% decline over the past year and 63% over the past 2 years. And at the same time there's been a significant increase in services that are using Let's Encrypt:

Both of those are trends are positive and show that encrypted services are becoming the norm.

However, we're also seeing that some users are increasingly putting services on non-standard ports instead of properly securing them. The following is a chart of Modbus services running on port 503 (the standard port is 502). Note that the Modbus protocol doesn't support authentication or encryption and should never be directly exposed to the Internet:

Mysteries of the Internet

Sometimes weird things happen on the Internet and it's not exactly clear why. For example, below is a trend chart for VPN services (tag:vpn):

Why was there such a huge spike in 2018? Lets break it down by country and see if that provides any insights:

It looks like an ISP in China for a short time was responding to all VPN handshake requests. The practice stopped after a few months and it's unclear why they did so.

FAQ

1. How much does it cost?
   A trend search uses 1 query credit if that search isn't yet cached.
   Any Shodan account that has query credits is able to use Shodan
   Trends - there aren't any additional costs.

2. Can I download the trend data?
   Yes, you can export the trend information as a CSV.

3. How far back does it go?
   We've indexed data going back to 2017. Technically, Shodan has data from as old as 2015 but we haven't yet indexed it in Shodan Trends. We will be adding older data over time.