IT Infrastructure Reconnaissance – Part 2 (Shodan, Censys, ZoomEye)

In the first part of this series I discussed the use of search engines as a source of information in the initial phase of the reconnaissance. As I mentioned in that article, search engines such as Google, Yahoo and Bing can allow you to search for information critical to security tests.

The information presented below is for educational purposes only. Access to data located as a result of the reconnaissance may be subject to a breach of the law. Make sure you are working legally before use.

Let us assume that this phase of exploration has been carried out. So what is the next step in the reconnaissance of infrastructure? Also search engines, but only those created for security testing, not based on URLs only, and additionally supported by banners captured during port scanning. These search engines scan, or actually use special crawlers that collect data about all kinds of devices connected to the Internet (computers, servers, printers, routers, IoT devices and many other things).

What is a banner that is captured by such a search engine, then processed and shared?

Let’s assume that the website is running on a server with access to the Internet. The network search engine will collect information from such a server (so called banner) with different information about it. Banners will vary depending on which service or device they refer to. Example of HTTP server banner:

Exemplary Siemens industrial device banner:

As you can easily see, banners can vary, so search engines provide different ways to filter out this information. If you are looking for devices with Internet access, we have three powerful search engines to choose from. Get to know all of them: Shodan, Censys and Zoomeye.

SHODAN

Shodan is the most popular search engine for network devices of the above. In this article I will focus on how to properly construct queries, what to look for and how to do construct queries in the fastest way.

Let’s start by considering how Shodan can be helpful in the reconnaissance phase of infrastructure or service.

The basic skill when it comes to Shodan’s operation is to construct filters. I will try to present here the way of thinking that should be followed during the tests, and how to specify your inquiries to choose pearls from hundreds of millions of banners that Shodan stores.

Let’s assume that we are preforming a facebook.com reconnaissance, and our goal is to find official Facebook pages with general access.

The first command I use is 200 OK 200 OK hostname:facebook.com, where “200 OK” means HTTP 200 response code; i.e., returning the content of the desired document.

Since Shodan obtains information, among others, from banners, if a page has the phrase “facebook” in the host name, it will be displayed in search results. A large part of the searched phrases are octet-stream files (binary), which in case of our search purpose are needless. In this case, the easiest way to filter out these files with the “!” character, the syntax of the query is as follows: 200 OK hostname:facebook.com !application/octet-stream. 

To limit the domains only to those official, signed with Facebook brand, just add the appropriate org filter. The whole query looks like this: 200 OK hostname:facebook.com !application/octet-stream org:”facebook”.

There are many possibilities of using Shodan for reconnaissance. Queries can be freely modified and precise depending on the expected search results.

A very useful function for implementing filters is the ability to “throw away” unwanted information. Shodan, even if we use quotation marks, does not use the whole phrase to search, and matches individual information from banners to the keywords used.

For example, for the query “Server IIS 4.0”, Shodan can return the banner as in Picture nr 4.

As we can see in the picture above, such banners do not fully meet our expectations. Shodan searches such a large number of differently constructed banners that a precise search is practically impossible.

How to deal with it? An interesting practice is to use a hyphen to exclude unwanted phrases. This is a laborious method, but it can be very helpful in case of many unsuccessful results.

In this case it was easier for me to find Microsoft IIS 4.0 server, excluding those more popular versions than the way I showed above: Server IIS  -“7.5” -“8.5” -“10.0” -“8.0”.

We filter something, we look for something, and in fact, what is worth focusing on and why?

First of all, passively (without any intervention in the server logs; in other words, without “leaving traces of your activity”) we can establish IP addresses belonging to the company or the technologies used by them:

It may not be spectacular, but in the reconnaissance phase such information can be very valuable, give the tester an idea of what technology they will work with, and indicate the first potential attack vectors.

All the information concerning which software and which version our goal is to use is crucial. This phase of the reconnaissance can be very profitable, especially if you manage to find outdated software with serious vulnerabilities. This will be important information for our customer. Certainly, people associated with the security industry drew attention to the header “Server” in the picture above, which reveals full information about the version of the used web server. How do we find such gaps and vulnerabilities in the easiest way?

Example possible query: org:“name of the audited company” Microsoft IIS. Shodan will list all captured banners containing the name “Microsoft IIS” in the Server header. Then we can filter out subsequent versions, or manually search for interesting banners, the method doesn’t really matter. However, if Microsoft IIS 6.0 could be found, for example, it would mean that the administrator forgot to update a given service for years, which put the company at great risk.

Since the text concerns the reconnaissance of infrastructure, it is clear that we will be interested in network devices such as switches and routers. An easy way to find the devices of a given supplier: org:“name of the audited company”cisco (or name of another company whose devices are of interest to us):

Another query, which we can use as a way to discover Shodan capabilities is “cisco default”; it focuses on searching default logins and passwords.

In the subject of reconnaissance of network devices we have a really wide field to show off, and the examples I show here are only a drop in the ocean of possibilities.

From the perspective of infrastructure testing, an important issue that can be checked with Shodan is the verification of the possibility of conducting Subdomain Takeover; i.e. taking, over a domain belonging to the tested company and placing any content there.

It may seem difficult to find such domains, but in practice it is enough to be a little clever. Heroku, for example, for a domain that can be used to display in the title of the page “No such app”. A quick query in Shodan, and to our eyes appears the result as in Picture nr 9.

If, after entering an address like this, and a page such as in the picture below appears, we detect a vulnerability.

Finally, it’s impossible not to mention Shodan’s plug-in to Mozilla Firefox. When you access a page, we can immediately display basic information such as IP address and ports used.

Censys

What is Censys? It is a search engine for devices connected to the Internet, and an another often inseparable element of reconnaissance. I will start with a question that is often asked in this area: what is the difference between Censys and Shodan?

The situation in this case looks a bit like in the case of search engines. Each tool used uses different search algorithms, has a different interface and provides some functionality, which is missing for the others.

As far as Censys is concerned, I like Data Definitions very much; it is a built-in mini tutorial on how we can filter information for selected ports or protocols. This is an intuitive solution and gives you the opportunity for a very detailed search, which is not possible on such a scale in Shodan. In the picture below I present this option for port 80; i.e., all well-known HTTP protocol.

Another option, which I often use personally are tags, which can be used freely in Censys, and in Shodan they are possible only for the Enterprise version.

Why is it useful? I will show this on the difference between two queries relating to the word “printer”. In the first one I used the “printer” tag, while in the second one I entered “printer” in the search field.

As you can see, the tag showed less results, but they are more precise, often giving the opportunity to exclude “garbage”; i.e., banners not related to our search purpose at all.

The search engine has three search categories:

• IPv4 hosts – a classic search for hosts as in Shodan.
• Websites – basic information about websites in relation to the “Alexa Top” ranking. You can look there, but it does not contain as interesting information as in the other two categories.
• Certificates – something that Shodan also does not have; a huge database of certificates.

Since I have already outlined the profile and capabilities of Censys, it’s time to get started with its practical use.

IPv4 hosts

IPv4 hosts tab allows for the same type of search as Shodan, using banners that can be searched with the appropriate filters.

At the very beginning, let’s try to find (as in the case of Shodan) all hosts that give the response of the server 200 (i.e., these are theoretically available sources) and belong to the Facebook organization:

As I mentioned above, these ready-made filters (marked in red in the picture above) make it very easy for the user to narrow down the results. What’s interesting when you click on an address? Information about the network, protocols used, TLS vulnerabilities or exact location, as shown in Picture nr 15.

Censys also collects information such as the name of the organization, as well as contact addresses and telephone numbers of the company or administrators. Below you will find information for Facebook:

Thanks to data definitions we have access to all the detailed search options, which is an unquestionable advantage of this search engine compared to other search engines. To fully describe the functionality of this tool, there is no information that Censys supports basic logical operators such as OR or AND.

With this knowledge, searching for the information we are interested in during the reconnaissance is only a matter of the ability to build an appropriate query. For training, let’s imagine that we are looking for Apache servers (80.http.get.headers.server: Apache), with available telnet (“23/telnet”) and simultaneously belonging to the autonomous system Amazon. com, Inc. (autonomus_system.organization: Amazon.com, Inc.). All you need to do is to connect this with the operator “and” and here is the result of our search:

An interesting topic is to target printers, which can be found in almost every infrastructure. So let’s see what safety tests can be found there. Let’s start by listing printers by printer tag, as in Picture nr 18.

How can we see a lot of results? But what interesting thing can be found during such a search? As shown in Picture nr 19 we can even access the entire panel of the printer.

In the case of these devices, safety is practically not taken care of at all. The safety panels of most of them look as follows:

It is easy to see that the level of safety is zero in this (and many other cases).

Certificates

This option is a mine of knowledge about websites. The information that can be obtained from it is:

• the certificates used by a particular page
• subdomains using the same certificate or other domains for which the certificate is registered
• filtering with tags allows you to search, for example, for certificates that have expired
• filter out certificates issued by the organisations concerned

By knowing how and which options to use, you can now look for interesting phrases. A simple example of how to list trusted certificates for facebook.com:

In my opinion, Censys is currently one of the best services containing information about certificates. It is characterized by a large amount of information and a great interface for filtering and sorting (which is definitely missing in the crt.sh environment).

After entering a given search result, everything is collected as in Picture nr 23.

It is a great place to check certificates or search for other domains for a given certificate. Often such domains are omitted by tools created to find them, and sometimes it is in such “forgotten” domains that you can look for critical errors, which will lead us to the purpose of testing.

Zoomeye

Zoomeye is the last with this search engine for internet devices. In my opinion it’s the most “hacking” of all the search engines I’ve described.

Zoomeye, in my opinion, is a combination of the “danger” of Shodan according to urban legends and a convenient interface with the ability to use tags. Additionally, for many queries Zoomeye shows up to ten times as many results as the other two search engines. It is impossible not to mention it.

Since we get a lot of questions about this portal, we add a small download about how to use it and what phrases can help in searching.

app: Application name

ver: Application version
Example: app: “Apache httpd” +ver:“2.4.33”.

country: Abbreviation of the country (e.g., US, PL, UK, FR, etc.)
city: Name of the city

Example: country:PL +city:Kraków
port: Port number

os: Name of operating system used
Example: os:linux +port:22 +country:US

service: Used service (all possibilities: https://svn.nmap.org/nmap/nmap-services)
Example: service:ftp

hostname: Hostname
Example: www.facebook.com +service:ftp

ip: ip address
Example: ip:8.8.8.8

cidr: the cidr segment for the given IP address
xample: cidr:8.8.8.8/24
For logged in users it is possible to use this option five times a day.

site: site name
Example: domain name site:google.com

headers: name of the HTTP header
Example: site:google.com +headers:Server

keywords: search by keywords, without specifying in which part of the banner they may be found.
Example: keywords:Drupal

desc: search for a phrase in the description
Example: desc:Wordpress

title: page title
Example: title:hacked

The character of this search engine can be proved by the fact that next to the “Result” tab, where search results are listed, there is a “Vulnerability” tab, which contains a list of vulnerabilities for the most popular programs, technologies or servers (Picture nr 25).

Let’s take a look at the vulnerability marked on the picture above. In this case, just look for Nginx servers in version 1.4.0 or 1.3.9, and for convenience, exclude HTTP 403 – Forbidden and 301 – Moved Permanently and everything is given as on the tray (Picture nr 26).

Summary

In this article I tried to show how great possibilities during the passive phase of reconnaissance are offered by specialized search engines. You can look for everything from the range of information needed for security testing, from addresses and subdomains to routers and switches that do not require any authentication. In principle, it is limited by the time and the ability to “dig” for information.

My goal in this article was to show a pattern of thinking that is worth following during the tests. This is what makes this type of search engines always contain things that amaze the whole industry. Step by step, a question is created, which leads to incredible discoveries. Remember that you cannot favour any of the described browsers; each has its advantages and disadvantages, and other crawlers. There is no “the best one” among them, but together they create a very powerful tool.

Author: Michał Wnękowicz

Tagged: Censys, Reconnaissance, Shodan, Zoomeye