Is MQTT scalable with 1000+ clients?

Question

Scenario
IoT device (currently IPv4 device) that sends via TCP socket a payload to a server once per day. The server has a public IP address, the device is behind a router/NAT. I'm going to use a module based upon ESP8266 (i.e. Olimex one)

Goal
The server should be able to send data to any client whenever it needs to. I'm no interested in direct client-to-client communication (i.e. connect to a device from my smartphone) like the hole punching is supposed to do.

Other requirements
The IoT devices might grow up to several thousands. Their Internet connection is provided by many 4G-enabled routers/modems. Each one will handle 10-20 clients.

Proposed solution
As far as I understand a common solution is MQTT. The clients periodically send data to the broker (i.e. Mosquitto running on the hosting server), that in turn updates the main web app that runs on the same server.

Question
Is MQTT approach suitable for a "large" number of devices (1000+) most of them behind a 4G router?

Answer 1

1,000 clients can easily be handled by any decent MQTT broker; there's a benchmark from Scalagent which shows that a PC with:

• a 3 GHz Intel Core 2 Duo processor
• 4 GB of RAM

could handle 60,000 publishers running Mosquitto. This is far in excess of your required 1,000 publishers, so even on a relatively weak server, you should still be able to handle the required number.

Some other brokers claim even better performance (with correspondingly greater server power, of course), such as HiveMQ, which claimed to handle 10 million publishers.

MQTT brokers do generally expect a persistent connection, and will timeout clients that don't send ping responses (or other activity) periodically. You could disconnect from the network after publishing, but, obviously, you then won't be able to receive anything if you disconnect.

MQTT does support the concept of 'retained' messages which might be useful. The web client can publish something to a topic with the retained flag, and this message will then be stored by the broker. Whenever your clients reconnect and subscribe to the topic, they will then receive the retained message (even if it was published hours ago). The retained message is published every time a client subscribes to that topic, so that might help you if you have a patchy connection and need a message to be stored until the client reconnects.

Answer 2

You can use persistent sessions from clients, e.g. clean flag set to false upon connect. In that scenario event when your client is offline broker will buffer message for it into own cache and deliver it once device will connect.

About the quantity - 10K is a relatively low amount even for one server. You can configure Linux server to hold 500K active connections and if your broker will be cloud-based, e.g. provided as service by some provider, then you can hold even millions of active connections to it.

By the way, I think Mosquitto or any other local installation is perfect choice for development and testing, but when you will go in production you need SaaS MQTT broker with all features like HA, redundancy, failover, etc.