3

I have some sensor nodes (Bosch XDKs) that send information to an MQTT broker and an application reads the information and stores it into InfluxDB. Simultaneously, I have RFID readers that scan some tags and send the information to a MongoDB. Based on some logic I wish to merge the data from InfluxDB and MongoDB together and would like to send to a custom platform which would have its own Database where the clubbed information will be stored to be visualized.

Basic Idea

  • The information flow is from sources of data is uni-directional (i.e. from sensors and RFID is only to Databases)

  • The Combine service can query the databases and combine the information and send it to the platform

Requirement

  • I wish to push real-time information to the platform. That is, whenever the MongoDB instance is updated, use some parameters to query the information from InfluxDB, Combine and send push it to the platform.

What I a unable to grasp is whether to use webhooks or websockets for the Combine App and also how is it possible to know if MongoDB was updated with some new information?

I read a good SE Query for Webhooks v/s Websockets and found that the scenario is more in the direction for server-to-server (webhooks) where the client interaces only to the platform via its internal services and not direclty via the two DBs but I am not sure if this is the case.

I see this as a very relevant usage of IoT Real-Time data application and am looking for some clear architecture and implementation answers for the mentioned case above.

Are there repositories that somehow provide help as to how to create webhooks/websockets that fulfill my requirements?

Improve this question
2
  • An excellent question! Which, alas, I cannot answer. Is there a real need to have three different databases? Could you not re-architecture to have only one?
    – Mawg says reinstate Monica
    Jan 8, 2019 at 8:49
  • 1
    I wish I could be I was looking into Query performances between MongoDB and InfluxDB and found no good signs of shifting things to one. The whole aspect of InfluxDB is the Timeseries stuff which suits sensors and Event-based collections for RFID stuff that fits MongoDB very well
    – Shan-Desai
    Jan 8, 2019 at 9:07

2 Answers 2

Reset to default
2

If someone needs an answer / solution for this query, I created a landing page of repositories that provides a complete End-To-End solution of a prototype of the application. Right from Arduino Sketch for the Sensors, as well as Backend API.

Landing Page: iotfablab/TrackNTrace

Also I have a written a scientific paper for it:

IEEE Xplore

If you do not have access, please ask for the paper on ResearchGate.

Improve this answer
2

Congratulations on writing a research paper on it.

This can be done with streaming platforms such as kafka or AWS KDA (kinesis data analytics). There are clauses in the query language in KDA (and also in KSQL, I think) which will allow you to link the events that happen very close together.

https://aws.amazon.com/kinesis/data-analytics/ https://docs.aws.amazon.com/kinesisanalytics/latest/dev/windowed-sql.html

If all you're looking for is to get the last known sensor values when an RFID is read, you can do it very simply in just AWS IoT Rules. Let the sensors write to a dynamodb table (using an IoT rule) and let the RFID reader pick up values from the dynamodb. You'll just need a way to associate the two together. That can just be another dynamodb table that is first looked at in a rule that gets an rfid reading (along with reader id) and uses the reader id to look up the associated sensors. Then, a second rule executes that picks up the last known values from those sensors. The end result can then be used to write to a queue, etc.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.