I am developing a helper app to clean the retained messages on my Mosquitto MQTT service. The problem I have is how to process the queue once with Paho MQTT.

I know how to

  • block the thread (loop_forever())
  • query the queue asynchronously (loop_start()/ loop_stop())

What I would like to do is to process the queue once and quit (and process each of the topics gathered this way, which are (most likely) retained messages)

loop() was the most promising:

Call regularly to process network events. This call waits in select() until the network socket is available for reading or writing, if appropriate, then handles the incoming/outgoing data.

Unfortunately I do not see any topics when using it.

Right now my code starts a thread, waits 2 seconds and stops it. It does the job but I would like to understand how to do that cleanly though a one-pass processing:

import paho.mqtt.client as mqtt
import time

class MQTT:

    def __init__(self):
        print("initializing app")
        self.client = mqtt.Client()
        self.client.on_connect = self.on_connect
        self.client.on_message = self.on_message
        self.client.connect("mqtt.example.com", 1883, 60)

    def on_connect(self, client, userdata, flags, rc):
        print("connected to MQTT with result code " + str(rc))

    def on_message(self, client, userdata, msg):
        # EDIT: added a check for actually retained messages
        if msg.retain:
            print(f"removing retained {msg.topic}")
            self.client.publish(msg.topic, retain=True)


Retained messages will only be delivered once (per connection).

And there can only be 1 retained message on a given topic at any one time.

So just connect, start the loop and subscribe to the topics you are interested in. When the message is delivered check the retained flag on the messages. If the message is retained then you can publish a new message with a null payload and the retained bit set to clear that topic. There is no need to do anything strange with the network loop.

  • The idea to check the retained flag is very good - it will filter out the other messages. Thanks. This does not however solve my problem of processing the queue only once: if I leave a loop_forever() then new (legitimate) retained messages would be cleaned as well. My intent is to clean up the queue from old messages (irrelevant at that point) but not break the retention capability (by cleaning up as soon as a new retained message arrives)
    – WoJ
    Apr 6 '18 at 10:59
  • Yes, but you can't do this with the publishers running as you have no way of knowing when they will publish a new retained message. They could publish 1 ms before you start this "clean"
    – hardillb
    Apr 6 '18 at 12:14
  • They could publish 1 ms before you start this "clean" You are absolutely right. I cannot do anything for existing relevant retained messages (they should be updated via telemetry after some time, hopefully) but I also do not want to wait longer than necessary so that future relavant retained messages (the ones which may pop-up during my cleanup) are less likely to be nuked. This is why I wanted to process only once the existing queue (made of correct and legacy retained messages) and do not take into account future ones. I currently have a 2 seconds window to allow or messages to be gathered
    – WoJ
    Apr 6 '18 at 12:20
  • In that case you may be better looking for a broker with an admin interface that lets you dump all retained message instantaneously on command than trying to do it from another client.
    – hardillb
    Apr 6 '18 at 12:32
  • It's that or keep a list of the topics that you've already processed and ignore new messages on those topics
    – hardillb
    Apr 6 '18 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.