Writing a Subdomain Discovery Tool in Crystal

Crystal is a new, compiled programming language that aims to mimic the Ruby syntax as closely as possible. Recently, @mil0sec wrote a Shodan client library for Crystal and to get a better feel for it I wanted to write a "Hello World" equivalent in the security world: a subdomain enumeration tool. I'll be cheating a bit in that we're going to use the Shodan API to do so. The final command will work as follows:

$ bin/subs
Usage: subs <api key> <domain>  
$ bin/subs YOUR_API_KEY cnn.com
ablink.emailalerts.cnn.com  
admin.ref.alertshub.cnn.com  
agility.cnn.com  
alertshub.cnn.com  
amp-ref.cnn.com  
amp-staging.cnn.com  
api.electiontracker.cnn.com  
api.etp.cnn.com  
api.platform.cnn.com  
app.cnnespanol.cnn.com  
apps.money.cnn.com  
audience.qa.cnn.com  
bea4c.cnn.com  
c.bea4.cnn.com  
ca.commerce.cnn.com  
...

You can also skip ahead and check out the code at:

https://github.com/shodan-labs/crystal-subs

The tool requires having at least a Shodan Membership.

Installing Crystal

Follow the instructions for your operating system on how to install the Crystal language. In my case, I followed the directions for Ubuntu which were straight-forward enough:

curl -sSL https://dist.crystal-lang.org/apt/setup.sh | sudo bash  

Initializing the project

To create the basic structure of a Crystal application we will be using the crystal command as follows:

crystal init app subs  

This will create all the necessary files to have our own command-line application and setup the initial shards.yml where we define our dependencies. The subs command is very simple so the only dependency we need to add is for the Shodan library. To do so edit the shards.yml file by adding:

dependencies:  
  shodan:
    github: percussiveelbow/shodan

After you've done so run the following command from within the subs directory to actually install the new dependency:

shards install  

To make sure everything's working as intended so far try running the following command to confirm that Crystal is able to compile the current code:

crystal run src/subs.cr  

It should run without showing any messages (good or bad).

Adding Shodan DNSDB

Now for the actual program! Follow the in-line comments for an explanation of what's actually going on:

We can test the program by running the command:

crystal run src/subs.cr -- APIKEY cnn.com  

If everything works as intended then you will see a list of subdomains for cnn.com. At this point, we can also create a standalone binary file:

crystal build --release src/subs.cr  

And we're done! We now have a compiled binary that will grab a list of subdomains. You can confirm that it's still working as before by running:

bin/subs APIKEY cnn.com  

It was fun learning a new language and exploring some of the Crystal library for Shodan. Check it out and let us know what you create! And thank you to @mil0sec for creating the library and sharing it with the community.