We are developing an IoT cloud platform with known components like Cloud Gateway, Stream Processing, Device Shadowing, etc...

The question is what is the proper sequence of locating pre-validation of incoming device data (maybe there is no need to do this first!) and updating device shadow.

And could you give me some advice what is the best path of data from the creation to completion.

  • Cloud gateway — MQTT broker from device sensor data first to ingest data to Apache Kafka.
  • Device shadow service — which holds latest states of devices in the cloud side of platform that users can be informed about the latest state of their devices — (this service is using event sourcing method which stores all sensor data as well)
  • What does 'proper sequence of locating pre-validation of incoming device data' mean? What do you want to validate? What kind of data? What do you want to locate?
    – Helmar
    Nov 12, 2017 at 20:58
  • @Helmar from the creation to the storage of the device sensor data, could you give me summary advices how/where to "validate my sensor data is valid or not", "update my device shadow/twin to store latest state somewhere", "process if there is a rule defined on the device in Apache Flink or Spark" and "when to store into Cassandra for analytics later use". Thanks
    – ccobanoglu
    Nov 12, 2017 at 21:09
  • That sounds like an enormous task with very little information to go on. I'd suggest you pick one of the areas validation, device shadow or persistence layer for the question and add some question parameters to get a good answer. Parameters could be expected regional distribution (my garden vs global), number of devices, amount of shared device data (bits or TB), complexity of device shadow (on/off vs. everything in an airplane) et cetera, otherwise the usefulness of any answer will be rather little and the question probably closed as too broad.
    – Helmar
    Nov 13, 2017 at 16:42
  • 2
    If you add these information please do not answer with a comment but edit them directly into your question.
    – Helmar
    Nov 13, 2017 at 16:44


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.