Edit: The original data for RDP in March, 2020 included IPv6 results whereas the historical analysis only looked at IPv4. I've changed the numbers to reflect the new counts. Tl;dr: we're still seeing growth but significantly less than before.

More companies are going remote due to COVID-19 and as a result there's been a lot of speculation around how this impacts the exposure of companies and the Internet as a whole (in terms of publicly-accessible services). I was actually already working on creating trends for various services due to a presentation I gave late last year so let me share with you some updated charts on how the Internet has evolved over the past few years (up to March 29, 2020).

Methodology

Just quickly I'll mention a bit about how the data itself is generated:

1. Shodan infrastructure is globally distributed to prevent being geographically biased
2. Crawlers run 24/7 and don't do sweeps of IP ranges the same way a network scanner would
3. Crawlers attempt full protocol-specific handshakes to validate that a port is responding. Depending on the protocol Shodan also performs additional steps to validate the response. For example, in the case of RDP the crawlers grab a screenshot, perform OCR on that screenshot and do a variety of basic security checks.

Timeframe

Shodan keeps a full history of every IP in the Internet that it's ever seen. We store that archive in a variety of formats and for this purpose I reprocessed our data going back to the beginning of 2017. You can also access that historical data via the API, CLI, or the new beta website.

Aggregation

I binned the results by unique IPs per month for each port/ tag. This means that data is not based on point-in-time scans but rather an aggregate view of the active IPs during a month.

Remote Desktop

The Remote Desktop Protocol (RDP) is a common way for Windows users to remotely manage their workstation or server. However, it has a history of security issues and generally shouldn't be publicly accessible without any other protections (ex. firewall whitelist, 2FA).

The number of devices exposing RDP to the Internet has grown over the past month which makes sense given how many organizations are moving to remote work.

It's surprising how the number of RDP instances actually went up after the initial Microsoft bulletin on Bluekeep in May 2019. And then it dropped sharply in August once a series of issues were revealed (DejaBlue) that impacted newer versions of RDP.

A common tactic we've seen in the past by IT departments is to put an insecure service on a non-standard port (aka security by obscurity). To that point, this is how the exposure for RDP looks like on an alternate port (3388) that we've seen organizations use:

It follows very similar growth as seen for the standard port (3389). The last thing I wanted to point out is that 8% of the results remain vulnerable to BlueKeep (CVE-2019-0708).

VPNs

The above chart encompasses a few different VPN protocols and ports (IKE, PPTP etc.). VPNs are a secure way to allow remote workers access to your network and it's not surprising to see that number grow as well the past month.

Industrial Control Systems

We've observed significant growth in other protocols (HTTPS) but one of the important areas where we've seen a worrying increase in exposure is for industrial control systems (ICS). The growth is not as large as for other protocols but these are ICS protocols that don't have any authentication or security measures. We had actually seen a stagnation in the ICS exposure up until now. And there have been significant advancements in OT security so there are plenty of secure options to choose from.

We're also keeping our country-wide exposure dashboards up-to-date if you'd like to see breakdowns by country.

Conclusion

I hope the above data provides a more data-driven view of how the exposure of those ports has changed the past few years. There aren't any earth-shattering surprises in the data but it's good to validate what many already assumed. If you're an organization that is concerned with your Internet exposure and wants to keep track of what you have connected to the Internet then please check out our Shodan Monitor service.

There's been much focus on MongoDB, Elastic and Redis in terms of data exposure on the Internet due to their general popularity in the developer community. However, in terms of data volume it turns out that HDFS is the real juggernaut. To give you a better idea here's a quick comparison between MongoDB and HDFS:

Even though there are more MongoDB databases connected to the Internet without authentication in terms of data exposure it is dwarfed by HDFS clusters (25 TB vs 5 PB). Where are all these instances located?

Most of the HDFS NameNodes are located in the US (1,900) and China (1,426). And nearly all of the HDFS instances are hosted on the cloud with Amazon leading the charge (1,059) followed by Alibaba (507).

The ransomware attacks on databases that were widely publicized earlier in the year are still happening. And they're impacting both MongoDB and HDFS deployments. For HDFS, Shodan has discovered roughly 207 clusters that have a message warning of the public exposure. And a quick glance at search results in Shodan reveals that most of the public MongoDB instances seem to be compromised. I've previously written on the reason behind these exposures but note that both products nowadays have extensive documentation on secure deployment.

Technical Details

If you'd like to replicate the above findings or perform your own investigations into data exposure, this is how I measured the above.

1. Download data using the Shodan command-line interface:

   shodan download --limit -1 hdfs-servers product:namenode
   

2. Write a Python script to measure the amount of exposed data (hdfs-exposure.py):

   from shodan.helpers import iterate_files, humanize_bytes
   from sys import argv, exit
   
   
   if len(argv) <=1 :
       print('Usage: {} <file1.json.gz> ...'.format(argv[0]))
       exit(1)
   
   
   datasize = 0
   clusters = {}
   
   
   # Loop over all the banners in the provided files
   for banner in iterate_files(argv[1:]):
       try:
           # Grab the HDFS information that Shodan gathers
           info = banner['opts']['hdfs-namenode']
           cid = info['ClusterId']
           # Skip clusters we've already counted
           if cid in clusters:
               continue
           datasize += info['Used']
           clusters[cid] = True
       except:
           pass
   
   
   print(humanize_bytes(datasize))
   

3. Run the Python script to get the amount of data exposed:

   $ python hdfs-exposure.py hdfs-data.json.gz
   5.0 PB
   

With Shodan it's easy to get an overview of the security for a country. Real-world borders don't necessarily translate to the Internet but it can still reveal useful information as shown by OECD. I will show how I use Shodan to get a big picture view of a country; in this case I'm looking at the USA.

First, lets have a look at how SSL is deployed in the USA. I will start off by getting a breakdown of the SSL versions that are supported by web servers:

shodan stats --facets ssl.version country:US has_ssl:true HTTP

To do this I'm faceting on the ssl.version property which contains a list of SSL versions that the web server supports. This is possible because Shodan crawlers explicitly test for SSLv2 through TLSv1.2.

Unsurprisingly, the majority of the HTTPS servers are hosted by Akamai and Amazon. However, there's still a sizable chunk (600,000+) devices that support SSLv2 so lets look at those briefly:

shodan stats --facets org country:US ssl.version:sslv2 HTTP

Here I'm faceting on the org (organization) property and filtering for web servers that support SSLv2. This doesn't mean that they only accept SSLv2 connections but it is one of the versions the service supports.

Around 25% of the services that support SSLv2 are operating on CenturyLink's network. Just looking at the results it seems like some of CenturyLink's modems are the reason for their #1 spot on the list. Their numbers are significantly higher than the next provider but I'm hoping these numbers will decline as CenturyLink phases out their older equipment.

The Shodan crawls also check for the various SSL vulnerabilities such as Heartbleed and FREAK so lets see how the US fares for those. For Heartbleed there are at least ~30,000 devices in the US still vulnerable to it.

Interestingly, Verizon Wireless is the network with the most services vulnerable to Heartbleed. The runner-up, Amazon, is less surprising since it's not unusual for people to deploy old images that haven't yet been patched (or lack protection). There are 2 types of devices operated by Verizon Wireless that are affected:

1. Wireless routers that run on the alternate HTTPS port 8443 and are made by CradlePoint Technology.

2. Digital billboards made by Watchfire Signs that run a web server on port 9443.

I have not heard of these products before but this explains why Verizon Wireless has the most devices affected by Heartbleed - I wouldn't have expected many regular web servers to operate on their network. The same analysis can be performed by looking at services that support export ciphers (CVE-2015-0204) which I will leave as an exercise.

Finally, lets look at the distribution of SSL certificates. It usually isn't a good sign if the same SSL certificate is deployed across a large number of devices. To see the usage of duplicate SSL certificates we can facet on the ssl.cert.fingerprint property:

shodan stats --facets ssl.cert.fingerprint country:us has_ssl:true http

The results of the command will give us the 10 most common SSL certificate fingerprints:

If you want to get more than 10 you can also provide a number to the facet. For example, this is how to get the top 100 SSL fingerprints:

shodan stats --facets ssl.cert.fingerprint:100 country:us has_ssl:true http

The most common SSL certificate is for what looks like Google's CDN on IPv6. However, the 2nd most often seen SSL certificate is for Ecommerce Corporation which is a familiar company if you've read some of my earlier articles on defaced websites. While we're looking at duplicate fingerprints, what about SSH fingerprints?

shodan stats --facets ssh.fingerprint country:us

The most common duplicate SSH fingerprint in the US belongs to GoDaddy. Looking at those results will require another blog post but the above is how I usually get started when trying to identify systemic problems.

Here is a short video that shows how I've done a similar analysis for Germany:

SSL is only one of many aspects that should be looked at and I will be discussing some other angles in future posts. I hope I've given you a better idea of how I use Shodan to breakdown SSL issues on a national level.

I wanted to revisit the results of a few posts last year on how to track website defacements and see how things have changed since then. In case you're wondering how this data is collected, I've created a video that shows in real-time the commands I used to generate the data:

Here's the Top 10 Website Defacers as of January 2016:

1. GHoST61: 51
2. Kadimoun: 39
3. AnonCoders: 35
4. r00t-x: 31
5. Shor7cut: 28
6. Owner Dzz: 27
7. Toxic Phantom FROM BANGLADESH BLACK HAT HACKERS: 27
8. TechnicaL: 21
9. virus3033: 21
10. Yuba: 17

GHoST61 also topped the ranking last year and remains at the top at the moment. Other familiar names are: r00t-x (moved down 1 rank), TechnicaL (moved down 2 ranks) and virus3033 (moved down 2 ranks). This means that 4 of out of the previous top 10 are still around, while the other 6 weren't listed before.

In terms of organizations containing defaced websites, the Ecommerce Corporation remains the most affected by far. At this point it seems a given that Ecommerce will have the worst ranking so lets look at the other organisations on the list. The full ranking is:

1. Ecommerce Corporation
2. Unified Layer (+1)
3. GoDaddy (-1)
4. CyrusOne
5. iServer Hosting
6. SoftLayer Technologies
7. Media Temple (-1)
8. Peer1 Dedicated Hosting (-4)
9. New Dream Network
10. Digital Ocean

The top 3 have remained the same, though GoDaddy and Unified Layer switched spots. New entries on the list are: CyrusOne, iServer Hosting, SoftLayer, New Dream Network and Digital Ocean. At this point it's clear that there are a few hosting providers with on-going problems and it doesn't look like they've made any impactful changes to reduce the number of compromised websites.

In terms of products, the vast majority of affected websites were running Apache:

I'm planning on periodically revisiting this subject to see how things change over time, especially with regards to the newly-listed organisations!

PS: Credit to @Viss for the dramatic hacker background image at the top.

I've written and presented on the topic of insecure databases for nearly 2 years now. The example I use the most to demonstrate the problem is MongoDB because it's popular and had terrible defaults. Invariably though the focus of the conversation ends up on MongoDB and not that there are hundreds of thousands of databases on the Internet without any authentication.

So for today I decided to take a look at something else: Memcached. Their website explains it best:

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

Do you operate a website? Does it get a lot of traffic? Then memcached is what you need to speed up response times by caching database lookups, web responses or anything else that takes more than a second to accomplish.

Shodan shows there are more than 130,000 Memcached servers running on the Internet. And they also return a lot of detailed information about their status:

Memcached provides its uptime, version, current number of connections, how much is being stored and much more. For now, I just took a look at the amount of data stored and how much memory is made available. Aggregating all the information from the publicly-available Memcached instances here are some stats:

• 8 TB of data stored
• 49,153 PB of memory collectively available

Since Memcached is a caching layer we wouldn't expect to see a lot of data stored in it on a permanent basis (records also usually have an expiration attached). And it doesn't offer advanced querying as a regular database would, which makes navigating the 8 TB of data more difficult than with MongoDB. That being said, there is still a lot of sensitive information that is temporarily stored on these instances. However, there is also a ridiculously giant amount of memory available on public Memcached servers. For people not familiar with petabytes, the total amount of memory advertised is 49,153,000 TB.

The organizations that are hosting the most instances are:

1. ColoCrossing
2. GoDaddy
3. Enzu
4. Aliyun
5. Alibaba Advertising

One of the reason for all these publicly accessible instances is the same as with MongoDB: the official, default configuration of Memcached listens on all interfaces. Curiously, the Linux distributions I looked at that are offering Memcached packages provided secure defaults; i.e. only listen on localhost. This means that most likely the above organizations installed Memcached from source.

I hope this has provided some evidence that it's not just MongoDB facing insecure-by-default issues when it comes to data storage services. I could've performed the same analysis as above for Redis, Cassandra, CouchDB or Riak.

In light of the recent incident of MacKeeper exposing 13 million accounts through a public, unauthenticated MongoDB instances I wanted to quickly revisit my earlier blog post on the subject.

At the moment, there are at least 35,000 publicly available, unauthenticated instances of MongoDB running on the Internet. This is an increase of >5,000 instances since the last article. They're hosted mostly on Amazon, Digital Ocean and Aliyun (cloud computing by Alibaba):

The most popular versions of MongoDB are:

1. 3.0.7: 3,010
2. 2.4.9: 2,624
3. 2.4.14: 2,535
4. 2.4.10: 1,879
5. 3.0.6: 1,256

By default, newer versions of MongoDB only listen on localhost. The fact that MongoDB 3.0 is well-represented means that a lot of people are changing the default configuration of MongoDB to something less secure and aren't enabling any firewall to protect their database. In the previous article, it looked like the misconfiguration problem might solve itself due to the new defaults that MongoDB started shipping with; that doesn't appear to be the case based on the new information. It could be that users are upgrading their instances but using their existing, insecure configuration files.

In terms of data volume, all of the exposed databases combined account for 684.8 TB of data. And the most popular database names are:

1. local: 33,947
2. admin: 23,970
3. db: 8,638
4. test: 6,761
5. config: 859
6. test1: 612
7. mydb: 549
8. DrugSupervise: 382
9. Video: 376
10. mean-dev: 252

The database names are mostly the same as before, with the exception of: DrugSupervise and mean-dev. Notably absent is hackedDB which was at #8 last time.

Finally, I can't stress enough that this problem is not unique to MongoDB: Redis, CouchDB, Cassandra and Riak are equally impacted by these sorts of misconfigurations.

HTTP/2.0 is the next version of the protocol powering websites and it promises many improvements over HTTP/1.x. There are a few different ways that a server can advertise support for HTTP/2.0 but the most common one is during the SSL handshake. I've added support in Shodan for tracking the negotiated HTTP versions and the data can be searched using the ssl.alpn filter.

Without further ado, here are the most popular HTTP versions on the Internet for HTTPS servers (port 443):

The above chart was generated using Shodan Reports, which now automatically show a summary of the negotiated HTTP versions when possible. Here are the top 10 versions in text:

1. HTTP/1.1
2. SPDY/3.1
3. SPDY/2
4. SPDY/3
5. HTTP/2
6. HTTP/2 Draft 14
7. HTTP/2 (Cleartext)
8. HTTP/2 Draft 17
9. HTTP/1.0
10. SPDY/0.9.4

Unsurprisingly, HTTP/1.1 remains the most popular version and I'd expect it to stay that way for a while. SPDY also remains a fairly popular choice, mostly due to CloudFlare's support:

Taking a deeper look at the HTTP/2 results we can see that the following organizations are leading the charge in HTTP/2 support:

Singlehop has the most servers supporting HTTP/2.0 at the moment! The 2nd organization "GetClouder EOOD" is actually SiteGround followed by Hostwinds and Cloudflare.

Some general oddities in the data:

1. There are a few servers that support HTTP/2 and SSLv2: https://www.shodan.io/search?query=ssl.alpn%3Ah2+ssl.version%3Asslv2
2. Google is the only provider of TLS with IMAP, POP3 and SMTP that also supports HTTP/2: https://www.shodan.io/search?query=ssl.alpn%3Ah2+port%3A%22993%22

You can get a full breakdown of all the organizations and versions using the Shodan command-line. Here's a video that shows how to do a few things which can be run automatically every month to keep track of changes in HTTP/2 adoption:

TL;DR: ~115,000 web servers on the Internet support HTTP/2 as of December 2015.

Dell has been hit with 2 security issues the past few days. I wanted to quickly summarize my findings from an external network perspective:

1. Laptops come pre-installed with a root certificate

https://blog.hboeck.de/archives/876-Superfish-2.0-Dangerous-Certificate-on-Dell-Laptops-breaks-encrypted-HTTPS-Connections.html

The root certificate is issued by eDellRoot. Initially, the story mentioned just one certificate but it quickly became clear that there was a 2nd certificate that can be found on live web servers using Shodan with the search query:

ssl:eDellRoot

At the moment, the search returns 28 results that are located mostly in the US with a few in Switzerland, Canada, Singapore and Malaysia:

Even though there are very few results, at least one of them has turned out to be a control system. This isn't a big surprise since there are millions of control systems connected to the Internet but it's a good reminder that the Internet has much more than just web servers.

Dell has issued a statement explaining the existence of the root certificate and released a tool/ instructions on how to remove it.

2. Webserver runs on port 7779 that provides unauthenticated access to the Dell service tag

http://www.theregister.co.uk/2015/11/25/dellbackdoorpart_two/

There are ~12,800 webservers on the Internet running on port 7779. Out of those, roughly ~2,300 are running software that looks like it's from a Dell computer:

I wrote a quick script to grab the service tags from those IPs and was able to collect ~1,000 service tags. The other 1,300 devices didn't respond in time or otherwise errored out when trying to query the information. Of course, much of the threat is the ability to execute Javascript to gather the information from localhost but I wanted to get a sense of how many are Internet-connected. I've also added port 7779 to Shodan so it will be possible to keep track of how the issue gets resolved over time.

I've made some improvements to the way SSL is indexed and added 2 new filters:

1. ssl
   Search all SSL-related information that Shodan collects.
   Example: ssl:Google
2. has_ssl
   Boolean filter to only show results/ banners that contain SSL information.

There was also a bug in how the SSL serial numbers were indexed so after that got patched I kept an eye on the results. To do so I used the command-line interface and faceted on the ssl.cert.serial property to get a list of the most popular SSL serial numbers:

The top 5 SSL serial numbers are:

1. 15264109253415148488
2. 17803741903183845083
3. 0
4. 40564819207326832829647457238321
5. 295

I wasn't sure what to expect so lets take a look at what the most popular SSL serial on the Internet is used by:

There are more than a million devices that use the serial number 15264109253415148488 and none of them return a banner. They're all self-signed certificates that are running a service on port 443 but otherwise aren't responding to HTTP requests. Hmmm, ok well what about the 2nd most popular serial number?

Once again a huge amount of devices are responding on port 443 and not providing any banners but this time for Motorola Mobility devices. In both instances the devices are located on AT&T's network, and based on the netblock ownership the IPs are being used for U-verse. I started searching for more information about these certificates and eventually found an answer:

Apparently, AT&T is running a service on port 443 to manage their wireless set top boxes. I don't have any way to verify those claims but they seem plausible. If nothing else it's now very easy to see how many of AT&T's users purchased their wireless Internet package (~2 million households).

I've recently started crawling for BitTorrent trackers on a few ports (69, 80, 6969) and got the following picture from the results:

Why are there so many trackers located in India? I would've expected to see a lot more in the US or Europe. There are probably other ports I should look into (if you have any suggestions please leave a comment) but it was unexpected to find so many results in India. I posed the question to Twitter and @jupenur came to the rescue:

@achillean Most of the IPs are from broadmax.in, an ISP that offers "Transparent Torrent Caching locally, for Super High Speed On Torrents."

— Juho Nurminen (@jupenur) September 4, 2015

Broadmax is an ISP primarily serving the city of Mumbai and as seen above advertises their ability to cache torrents/ improve download speeds. I've never seen this sort of thing by an ISP in the United States but it's actually been around for a while. There's even an official cache discovery protocol to help ISPs know which torrents are popular and should be cached. From a technical perspective, it makes a lot of sense actually for ISPs to locally cache the data. BitTorrent accounts for ~3.35% of the worldwide Internet traffic (2013 data), with certain countries being significantly higher such as the US at ~25%. I don't expect torrent caching to show up anytime soon in my area though. Either way, I'll be adding more ports to track whether more ISPs around the world are starting to use this sort of technology.

I've started collecting screenshots for a few services, most notably VNC, and something stuck out at me:

The top 5 ports where VNC is running with authentication disabled are:

1. 5900 (default port): 4,029
2. 5901: 3,995
3. 84: 25
4. 83: 14
5. 13579: 7

Out of ~8,000 results, 50% of the results came from services that were operating VNC on a non-standard port. It's not unusual to see common services running on different ports, but that was a surprising amount. My guess is that a lot of people change the default port thinking that it will hide their service. Because Shodan scans for 250+ different ports however, there's a small chance that Shodan will discover it anyways. And for a lot of the popular protocols, Shodan actually also crawls for one-off ports (thank you to @Viss for that idea).

I've seen this sort of behavior in other services as well, this isn't limited to VNC. If you've read my previous blog posts this might sound familiar to you. In fact, I observed much of the same when looking at SSH. For SSH, the choice of ports is a bit wider but in general people don't work well as random number generators.

Furthermore, this sort of behavior can be observed across the industries. For example, you might know that Shodan crawls the Internet for industrial control systems (ICS). One of the most popular protocols in ICS is called Modbus that runs on port 502. At the moment, there are about 17,000 devices listening to Modbus on the default port. It turns out there are also 700 devices listening on port 503, again a one-off sort of situation.

If you're looking to hide your service putting it on a different port is a temporary band-aid at best and a false sense of security more than anything.

The Roku is a small computer that enables you to stream videos and music to your TV. Before the rise of smart TVs it was one of the easiest ways to watch Netflix in your living room and it still seems to be thriving. I hadn't thought much about them recently until I saw a great series of posts on Reddit recently on the security of the Roku:

• Roku API doesn't have authentication and allows remote reboot: http://x42.obscurechannel.com/2015/07/25/restart-a-roku-via-bash/
• Roku WPS Pin cracked: http://x42.obscurechannel.com/2015/07/26/cracking-the-roku-v2-wpa2-psk/

Much of the smart TV world is full of low-hanging fruit in terms of security. For example, this is me running a network scan on my Vizio TV:

In case you can't make it out: scanning the TV with Nmap launches an update and shows the application menu - no authentication required. As such, it isn't a huge surprise to learn that the Roku offers an API to control the device that doesn't have authentication enabled. And to be fair, the use case for the API is to allow local users to control their Roku over the phone. They're not meant to be directly exposed on the Internet. Aside from the security implications, this also provides an opportunity to learn a bit about which Roku devices are most popular and which apps users install the most. First, I scanned the Internet for devices then downloaded the results. If you have access to the Shodan command-line client you can get the data using:

shodan download --limit -1 roku-data "port:8060 Roku"

It seems there are around 1,868 Roku devices directly on the Internet as of July 26, 2015. I expect this number to fluctuate depending on the timezone that the scan is performed, but it's a good starting point to learn more about Roku's usage. To start off, I wanted to learn which Roku devices sell the most so here is a ranking of the Top 10 Most Popular Roku devices:

1. Roku 3: 514
2. Roku Stick: 376
3. Roku 2: 169
4. Roku 2 XD: 163
5. Roku 2 XS: 161
6. Roku LT: 121
7. Roku 1: 116
8. Roku HD: 93
9. Roku Streaming Player 2050X: 41
10. Roku Streaming Player 2100X: 28

The total number of devices isn't huge but I think it's awesome that we can empirically measure which products sell the most using real data. And it's interesting that the most expensive model, the Roku 3, is also the most popular one. Usually, the low- and mid-range models for a product are most visible on the Internet but that isn't the case this time. In terms of specific model numbers the breakdown is as follows:

1. 4200X: 538
2. 3500X: 350
3. 3050X: 163
4. 3100X: 162
5. 2720X: 146
6. 2500X: 93
7. 2400SK: 61
8. 2050X: 41
9. 2100X: 28
10. 2400X: 28

Finally, I wanted to see which channels are most commonly installed on Roku devices. The Roku API will happily tell you all the channels that the device has running, so I gathered all the data and am making it accessible via 2 Gists:

• List of Channels: https://gist.github.com/achillean/110dd0fdd8d42c6fe08e
• List of Channels with Versions: https://gist.github.com/achillean/32b8f31b9072fd98a986

The Top 10 Channels as determined via Shodan are:

1. Netflix
2. Amazon Instant Video
3. Hulu Plus
4. VUDU
5. Pandora
6. YouTube
7. Crackle
8. Blockbuster
9. Popcornflix
10. Rdio

I was really surprised to see Blockbuster on this list, since I thought they were dead but apparently the video streaming is still online. Naturally, I wanted to compare my list to the official most popular channels on the Roku website. Theirs is:

1. Netflix (-)
2. Hulu Plus (+1)
3. Amazon Instant Video (-1)
4. Sling TV (+22)
5. HBO GO (+11)
6. Crackle (+1)
7. Time Warner Cable (+39)
8. PBS (+10)
9. VUDU (-5)
10. Acorn TV (+55)

The difference between the Shodan ranking and the Roku rankings is provided in the parenthesis. For example, Hulu Plus moved up 1 rank in the Roku ranking while VUDU fell 5 compared to Shodan's. The sample size is much smaller than what Roku has and maybe people that put Roku devices on the Internet simply prefer YouTube over PBS or Acorn TV. But Sling TV, Time Warner Cable and Acorn TV aren't anywhere close to the top 10 in the Shodan ranking yet they're very high in Roku's list.

It's also possible to determine how often people update/ patch their channels. For example, this is the breakdown for the various versions of the Netflix channel:

Based on these results it looks like most customers don't update their channels/ apps on the Roku. For a complete breakdown of all version and apps please check out the CSV. Let me know if you find anything interesting/ cool/ weird in the data!

I would like to take a moment to discuss databases. Most people use Shodan to find devices that have web servers, but for a few years now I've also been crawling the Internet for various database software. I usually mention this during my talks and I've tried to raise awareness of it over the years with mixed results. At least with MySQL, PostgreSQL and much of the relational database software the defaults are fairly secure: listen on the local interface only and provide some form of authorization by default. This isn't the case with some of the newer NoSQL products that started entering mainstream fairly recently. For the purpose of this article I will talk about one of the more popular NoSQL products called MongoDB, though much of what is being said also applies to other software (I'm looking at you Redis).

Note: This article isn't about the way MongoDB scales.

Firstly, in an effort to make it a bit easier to understand the results for MongoDB I've updated the way they're represented in the search results:

A quick search for MongoDB reveals that there are nearly 30,000 instances on the Internet that don't have any authorization enabled. This was actually a bit surprising since by default MongoDB listens on localhost and has done so for a while based on the oldest Github checkin for their mongodb.conf. This made my results very confusing: how could there be so many open MongoDB installations if the defaults were to listen on localhost?

Configuration History

So I started downloading older versions of MongoDB to figure out when they changed the configuration defaults. It turns out that MongoDB version 2.4.14 seems to be the last version that still listened to 0.0.0.0 by default, which looks like a maintenance release done on April 28, 2015. I'm a bit confused why a configuration file was checked-in to Github September 2013 that listened on localhost by default, but then they kept distributing versions that didn't include that change?! I dug around some more and eventually found the official issue in Jira that tracked this configuration issue:

https://jira.mongodb.org/browse/SERVER-4216

Roman Shtylman actually raised this problem back in February of 2012! It ended up taking a bit more than 2 years to change the settings. Based on the distribution of versions I'm seeing, my guess is that early versions of 2.6 might've also lacked binding to localhost:

The lack of secure defaults explained some of the 30,000 results but just looking at the data made something else obvious.

The Cloud

The vast majority of public MongoDB instances are operating in a cloud: Digital Ocean, Amazon, Linode and OVH round out the most popular destinations for hosting MongoDB without authorization enabled. I've actually observed this trend across the board: cloud instances tend to be more vulnerable than the traditional datacenter hosting. My guess is that cloud images don't get updated as often, which translates into people deploying old and insecure versions of software.

Problem Scope

There's a total of 595.2 TB of data exposed on the Internet via publicly accessible MongoDB instances that don't have any form of authentication. To determine the scale of the problem I downloaded the data using the Shodan command-line tool:

shodan download --limit -1 mongodb "product:MongoDB"

And then I ran a small Python script to aggregate the total size of all exposed databases. I also looked at which database names were most popular:

1. local: 27,108
2. admin: 22,286
3. db: 9,895
4. test: 6,818
5. config: 1,119
6. mydb: 498
7. Video: 409
8. hackedDB: 319
9. storage: 315
10. trash: 309

Faceting on the database name reveals widespread installations that might've been misconfigured or otherwise exposed. There are a lot of instances that have some sort of administrative database, so the app that uses MongoDB probably has authentication but the database itself doesn't... The name that really sticks out is hackedDB. It's unclear whether those instances have been compromised or whether it's a large deployment of MongoDB servers from a company that uses "hackedDB" as its database name. Or maybe it's a honeypot? The interesting thing to note when looking at the results is that 40% of the instances are running a very old version of MongoDB (1.8.1).

I could go on and on about these sorts of problems because they're everywhere and haven't been resolved for years. Hopefully, more people will start looking at services that are responsible for the actual data and not solely focus on the web interfaces.

The field of presidential candidates has started to heat up and the websites are the first stop for a lot of prospective voters. For my purposes though, I was less interested in their political platform and more curious about the technology behind the websites. Others have already compared the SSL security of the candidates, so I wanted to check out what sort of information the presidential hopefuls' robots.txt files and 404 responses return. To generate the 404 response I chose a random URL /test (turns out I'm really bad at being random).

Without further ado, let me show the results of the requests:

Democrats

Hillary Clinton

https://www.hillaryclinton.com

robots.txt

User-agent: *
Disallow: /api/

Looks like there's an API for their website that is undocumented publicly.

404

Bernie Sanders

https://berniesanders.com/

robots.txt

User-agent: *
Disallow: /wp-admin/

The website uses Wordpress as its framework.

404

Martin O'Malley

robots.txt

User-agent: *
Disallow: /wp-admin/

The website uses Wordpress as its framework.

404

Jim Webb

robots.txt

User-agent: *
Disallow: /wp-admin/

Sitemap: http://www.webb2016.com/sitemap.xml

404

Lincoln Chafee

robots.txt

User-agent: *
Disallow: /wp-admin/

404

Republicans

Jeb Bush

robots.txt

No robots.txt file available.

404

Rand Paul

robots.txt

User-agent: *
Disallow:

404

Ted Cruz

https://www.tedcruz.org

robots.txt

User-agent: *
Disallow: /wp-admin/

The website uses Wordpress as its framework.

Rick Santorum

http://www.ricksantorum.com/

robots.txt

User-Agent: *
Disallow: /admin/
Disallow: /utils/
Disallow: /forms/
Disallow: /users/
Sitemap: http://www.ricksantorum.com/sitemap_index.xml

Based on this information the website is a hosted CMS at nationbuilder.com

404

Ben Carson

https://www.bencarson.com/

robots.txt

No robots.txt file available.

404

Most of them didn't turn out to be very interesting to look at, with the exception of the final candidate I'd like to show:

Carly Fiorina

https://www.carlyfiorina.com

robots.txt

User-agent: *
Disallow: /standing-desks2
Disallow: /standing-desks2.html
Disallow: /privacy-policy.html
Disallow: /privacy-policy
Disallow: /terms-of-use.html
Disallow: /terms-of-use
Disallow: /adjustable-height-desk.html
Disallow: /adjustable-height-desk
Disallow: /blank
Disallow: /test

404

It turned out that my random URL of /test wasn't random enough and I accidentally stumbled upon a location on Carly Fiorina's website that requires authentication.

I took away 4 lessons from this exercise:

1. Wordpress remains incredibly popular
2. robots.txt can tell you where the administrative area is
3. 404s must be generated enough that it is worth investing time into making them nicer
4. I'm bad at generating random URLs

PS: Did you know that Shodan also grabs the robots.txt data for each IP? You can access all the information via the Shodan API.

I've recently added the ability to search for devices in Shodan based on the state they're located in. This provides the interesting possibility to start comparing the security posture of US states by looking at what sort of things they expose publicly. To start off, I will be taking a look at how prevalent industrial control systems (ICS) are in the various states.

Shodan currently crawls for roughly 15 different ICS protocols on more than 20 ports and within the past week or so has discovered ~40,000 of them that are publicly accessible on the Internet worldwide. I presented on a few of these protocols at 4SICS in October 2014, and you can download the data and view the maps that were generated for the talk.

Lets start taking a look at the US in particular though: so far in May Shodan has discovered 20,445 ICS devices in the US. This is definitely a lower bound and more will be discovered until the end of the month (many ICS devices have high latencies - more on that later).

The most popular protocol is Tridium's Fox, followed by BACnet and Modbus. Fox and BACnet are commonly used by building management systems (BMS), while Modbus is used across a wide range of products. The full Top 10 Protocols are as follows:

1. Tridium Fox: 7,706
2. BACnet: 4,525
3. Modbus: 1,625
4. EtherNet/IP: 1,578
5. ProConOS: 1,018
6. General Electric: 956
7. OMRON FINS: 777
8. Mitsubishi: 615
9. Red Lion: 551
10. Codesys: 407

By faceting on state using the Shodan API we can get a breakdown of ICS devices for each US state:

The above map chart shows a breakdown of which states have the most ICS devices on the Internet. As you'd probably expect the larger, more populous states also tend to have more devices online:

1. CA: 2328
2. TX: 1422
3. NY: 739
4. MA: 559
5. IL: 552
6. PA: 482
7. OH: 466
8. NJ: 465
9. FL: 416
10. MI: 390

Bigger states have more people, more devices and therefore more control systems required to provide services. This means larger states would always be at the top of any ICS ranking, which means it's not entirely fair to compare states based on absolute numbers. So lets normalize the results and look at states based on the % of devices in the state that are control systems:

And now we get a slightly different picture. Maine is at the top of the list with 0.23% of the state's devices being industrial control systems on the Internet, followed by Hawaii (0.19%) and Nebraska (0.17%). The top 10 are:

1. Maine: 0.23%
2. Hawaii: 0.19%
3. Nebraska: 0.17%
4. Vermont: 0.17%
5. West Virginia: 0.16%
6. Montana: 0.14%
7. Rhode Island: 0.14%
8. Iowa: 0.13%
9. Arkansas: 0.12%
10. Washington DC: 0.11%

If you want to analyze the results further or look at the data yourself, check out the Shodan API documentation for information on what you can search and facet on. There are also a bunch of libraries available in Python, Ruby, NodeJS and Go to make getting started easy.

I will be keeping track of how these numbers change in the coming months/ years, especially as federal policies change and cyber insurance becomes more popular.

PS: If your browser supports WebGL you can also check out the following visualization that was generated for my talk at the Department of Homeland Security ICS Joint Working Group conference in June 2014: https://ics-radar.shodan.io/