We're running a cluster with 2 VerneMq brokers. Everything works fine if we start one broker at one time, but as soon as we need to re-start both the brokers, all the retained messages are lost.

To fix the issue, we tried configuring persistent volume with vernemq, and we can see that the claim is bound to vernemq and volume is created. Even after this when we tested our scenario by restarting both the pods, we found that the retained messages were not synced. This is leading to data loss.

As per our understanding, all we need to do to use persistent storage with VerneMq is to set this value to true:

  ## If true, VerneMQ will create/use a Persistent Volume Claim
  ## If false, use local directory
  enabled: true

This should create the relevant persistent volume and map it correctly to our pods. When we run our cluster with this setting, we can see that Persistent volumes are created and mapped, but the messages are still being lost.

I've been trying for around 2 weeks now to find a solution, could be that I am missing something small. Will appreciate any help I can get.

Code in statefulset.yaml that creates and maps the volumes:

{{- if .Values.persistentVolume.enabled }}
    - metadata:
        name: data
        {{- range $key, $value := .Values.persistentVolume.annotations }}
          {{ $key }}: {{ $value }}
        {{- end }}
        {{- range .Values.persistentVolume.accessModes }}
          - {{ . | quote }}
        {{- end }}
            storage: {{ .Values.persistentVolume.size }}
      {{- if .Values.persistentVolume.storageClass }}
      {{- if (eq "-" .Values.persistentVolume.storageClass) }}
        storageClassName: ""
      {{- else }}
        storageClassName: "{{ .Values.persistentVolume.storageClass }}"
      {{- end }}
      {{- end }}
{{- else }}
        - name: data
{{- end }}

Volume mapping:

- mountPath: /vernemq/data
  name: data
  • 2
    I'm not sure this is the right place for this question, it's not really IoT based (I can see it's about running your messaging infrastructure for IoT, but still) I suggest you raise an issue directly with VerneMQ e.g. here github.com/vernemq/vmq-operator/issues
    – hardillb
    Commented Oct 10, 2022 at 10:35
  • 1
    Sorry, it looks lke the right repo is the docker one github.com/vernemq/docker-vernemq
    – hardillb
    Commented Oct 10, 2022 at 10:46
  • 1
    Also look at the default volume access mode and how your brokers are spread across multiple nodes...
    – hardillb
    Commented Oct 10, 2022 at 10:49
  • Have raised a bug with docker vernemq repo. Let's see if I get any feedback :)
    – Logan
    Commented Oct 10, 2022 at 13:07


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.