Introducing Shodan Trends
Shodan was originally designed as a tool to understand how technology use is changing on the Internet. The information collected from the Shodan crawlers would be able to provide users with a data-driven view of what the Internet looks like; i.e. not based on surveys or sampling of popular websites. Which FTP software is most popular? How quickly is my hosting provider patching services? Which countries are running most of the VPNs? There were others that provided the information for web servers but I wanted to offer it for everything that's online - not just the web. I'm happy to announce that we now have a website that does that and more:
At Shodan, we've always kept a full history for every IP that we've ever seen on the Internet. You can look at the history for individual IPs using the new website or the API/ CLI. However, we never indexed that historical data in the search engine so you couldn't see how results have changed over time unless you wrote your own scripts to periodically query Shodan. With Shodan Trends you can now search the historical data to answer large-scale questions about the Internet. And we're making this new feature available to all members at no additional cost (see FAQ below).
Technology Trends
Shodan Trends shines when looking up the history of more complex queries but you can just as easily also get a breakdown of web server software. It turns out that in 2021, nginx overtook Apache as the most popular web server software on the Internet.
And of course we can do the same for other software as well. For example, here is a breakdown of FTP software:
The sharp increase in 2018 for Pure-FTPd is due to GoDaddy's use of the software.
We can also detect protocol-level trends such as the decline in Telnet use across the Internet:
Telnet has seen a 33% decline over the past year and 63% over the past 2 years. And at the same time there's been a significant increase in services that are using Let's Encrypt:
Both of those are trends are positive and show that encrypted services are becoming the norm.
However, we're also seeing that some users are increasingly putting services on non-standard ports instead of properly securing them. The following is a chart of Modbus services running on port 503 (the standard port is 502). Note that the Modbus protocol doesn't support authentication or encryption and should never be directly exposed to the Internet:
Mysteries of the Internet
Sometimes weird things happen on the Internet and it's not exactly clear why. For example, below is a trend chart for VPN services (tag:vpn):
Why was there such a huge spike in 2018? Lets break it down by country and see if that provides any insights:
It looks like an ISP in China for a short time was responding to all VPN handshake requests. The practice stopped after a few months and it's unclear why they did so.
FAQ
How much does it cost?
A trend search uses 1 query credit if that search isn't yet cached.
Any Shodan account that has query credits is able to use Shodan
Trends - there aren't any additional costs.Can I download the trend data?
Yes, you can export the trend information as a CSV.How far back does it go?
We've indexed data going back to 2017. Technically, Shodan has data from as old as 2015 but we haven't yet indexed it in Shodan Trends. We will be adding older data over time.