I have an IoT network of devices sending data to each other, with data being stored in a database.

If I have a device sending 10 packets/API requests in sequence, sometimes only a few reach their destination. For example, packets 1, 3 and 9 might reach their destination, while the others do not.

How can I track these packets and ensure that all reach their destination with no duplication or leakage? I expect that there will be thousands of devices in a real-life scenario, rather than just one device losing packets.


Typically, you would need to choose a protocol with strong guarantees on whether the client will receive any packets/messages, in which order, and whether duplication is permitted.

For a network of IoT devices sending small- to moderately-sized messages to each other, using MQTT with Quality of Service 2 would seem to fit your use case well. As stated in the HiveMQ link:

The highest QoS is 2, it guarantees that each message is received only once by the counterpart. It is the safest and also the slowest quality of service level. The guarantee is provided by two flows there and back between sender and receiver.

Note that QoS 2 does preserve order of messages, and, as stated, prevents duplicated messages.

There is a substantial overhead in using MQTT QoS 2 compared to the standard QoS 0 (which is akin to a fire-and-forget message; if it doesn't reach the broker, then the message doesn't get resent and it's gone forever) — QoS 2 requires 4 messages (PUBLISH from sender, PUBREC from broker, PUBREL from client, PUBCOMP from broker), so it'll generally take longer to process, take more resources (hence longer radio transmissions and more power usage on any constrained endpoints).

An MQTT QoS 2 message will just be resent from the sender repeatedly until it receives acknowledgement from the broker, so eventually your message should get through, even if your connection is imperfect.

Whether a topic-based publish-subscribe protocol is appropriate for your use case is up to you to determine; the Wikipedia article might help you to get an idea.

| improve this answer | |
  • 1
    Thanks @Aurora0001 for your work and reply. Any mechanism on HTTP protocol based message communication/async, or IP packet based or can say kind of fire and forget ? Let say one scenario - System A send message to System B. In system A 5 message created/prepared to send in sequence like - 1,2,3,4,5. In system B received message like - 1,4,5. Here system B missed the record/packet/data for 2 3. So what could be mechanism to make sync between System A-B with no missing, no duplication-record using minimum trip/bandwidth/network – user1467280-Satyajit the truth Aug 8 '17 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.