Measuring the Minecraft Playerbase

For fun I decided to see whether I can figure out how many Minecraft players are online at the moment. And it turns out that it's fairly straight-forward so here's how I did it.

As of now June 1st 2017 at 18:55 there are 96,418 players online on public servers.

To get started I downloaded the latest list of Minecraft servers from Shodan:

shodan download --limit -1 minecraft-servers product:minecraft port:25565

Now the next task is to parse that list of servers and request the number of players that are currently online. To speed things up the plan is to asynchronously perform the requests to the Minecraft servers using the gevent library in Python. It lets you write code that looks synchronous but actually runs asynchronously which means you can perform many connections in parallel. This is the usual template I use when grabbing a bunch of data using gevent:

#!/usr/bin/env python
#
# Shodan Async Workers

## Configuration
NUM_WORKERS = 100


# Make the stdlib async. This is where the gevent magic happens
import gevent.monkey
gevent.monkey.patch_all(subprocess=True, sys=True)


from gevent.pool import Pool
from shodan.helpers import iterate_files
from socket import setdefaulttimeout, socket, AF_INET, SOCK_STREAM

setdefaulttimeout(2.0)

def worker(banner):
    # Here's where you do the network stuff
    # Example:
    # con = socket(AF_INET, SOCK_STREAM)
    # con.connect((banner['ip_str']
    # con.send('hello world\n')
    # data = con.recv(5120)
    return True

def main(files):
    pool = Pool(NUM_WORKERS)

    # Loop through the banners in the file(s) and launch a worker
    # for each banner. When the pool is full it will cause the loop to
    # block until a worker finishes and opens up a spot in the pool.
    for banner in iterate_files(files):
        pool.spawn(worker, banner)

    # Wait for the workers to finish up
    pool.join()

    return True


if __name__ == '__main__':
    import sys
    sys.exit(main(sys.argv[1:])

If you're working with Shodan data files I recommend checking out the shodan.helpers.iterate_files() method since it'll make it easy for you to access the banners. You can give it either a single file:

for banner in iterate_files('minecraft-data.json.gz'):
    ...

Or you can provide it a list of files:

for banner in iterate_files(['minecraft-2017-04.json.gz', minecraft-2017-05.json.gz']):
    ...

To get the player count I added a method in the worker() that looks up the Minecraft info based on their current protocol and kicked it off:

$ python global-player-count.py minecraft-data.json.gz
96418

And that's how I'm now keeping track of how many players are at any moment online on Minecraft!

Note that this method only looks at Minecraft servers running on the default port (25565) and that are publicly-accessible on the Internet.