As far as I can see, Shodan seems to be choosing the IP and ports randomly from a list. Is this really the best way to scan the entire internet for IoT devices?

  • 1
    How else would you go about it? You can crawl HTTP, which has hyperlinks. How else would you find IoT devices, other than brute force (ouch! for IP v6) - unless you know that a manufacturer was allocated a block of fixed IP addresses, which seems unlikely. Dec 6 '16 at 18:17
  • 2
    If it chose ports sequentially, it would be far easier for manufacturers to recognize a Shodan scan in their firmware and block it. The randomization is an effort to obfuscate the source of the HTTP query, which in turn makes it harder to recognize as a scan or intrusion attempt, which in turn (theoretically) makes the resultant data set more reliable.
    – John
    Dec 6 '16 at 18:21
  • 1
    That is an execllent point. Maybe worth posting as an answer? Otoh, that extra code to check, and storing the data may be too much for soem small devices. It could make a good FOSS (or even commerical) product, though. Dec 6 '16 at 18:23
  • 1
    @WayToDoor Random - I see. Nope, I can’t think of a reason for that. Otoh, you might be able to make it slightly more efficient by excluding ranges of addresses, such as those known to be allocated to major ISPs. Dec 6 '16 at 18:23
  • 2
    Reliable for what? For finding all devices in a particular domain? For finding all devices of a certain type? For detecting whether a particular device is visible from the Internet? etc. Dec 7 '16 at 0:37

In order to scan the entire Internet, you need to scan the entire Internet: every combination of IP address and port. Further, you need to do it in a way that hides what you're doing, so you don't get blocked.

A simple sequential scan is the obvious way to do it, but a sequential scan is trivial to spot. Instead, Shodan scans in an apparently-random pattern (a permutation of the complete list of address/port combinations), and does so from a wide range of source addresses. Doing this means it takes longer to say that a given address does (or does not) have an IoT device on it, but it also makes the scan look more like the random noise of the Internet.

Not the answer you're looking for? Browse other questions tagged or ask your own question.