MongoDB/Redis/CouchDB/Any Graph DB with S3 for IoT implementation

Question

I have an IoT use-case (Should I use the MQTT protocol?) and decided to implement the SMACK architecture. I hope, Up and running the SMACK stack would be very much cost. Right now I am not expecting that much data. In future, based on the increase of sensor data I'll implement SMACK stack.

---

Use case

The IoT devices will send approx. 100k messages/minute to the gateway via repeaters and the gateway will transfer the messages to the cloud. I want to track the employees in an organization. The sensors will be fixed on their ID card. The sensor sends the location related data (approx. 15KB/message) to the gateway via repeaters using BLE. It's for the analytical purpose. After the data passed to the cloud, I'll do some analytics and store into the DB and display on a web page. Based on this analytics data, I'll show the user's current location and also the user's moving locations of a certain passed time span (last 1hr or 2hr or 1day).

I'll do some processing over the data and send it to the front end/DB.

---

So If I use the below simple architecture with the use of Redis/MongoDB/CouchDB/Any GraphDB NoSQL layer and S3 as the storage to store historical data. So the cost would be very much low.

Workflow:

1. Ingest into Redis/MongoDB/CouchDB/Any Graph DB,
2. Copy into S3 over time and
3. Expire in Redis/MongoDB/CouchDB/Any Graph DB.

So the NoSQL layer will have only the recent data and the S3 will have the historical data.

Question:

What would be the best choice in the NoSQL layer (i.e) MongoDB/Redis/CouchDB/Any GraphDB?

Answer 1

As you specially asked me to look into this question I will post some kind of advertising, because again, I think that your project can be reached with lower amount of running cost, with higher performance and significantly faster implemented by using already specialized in this platforms or services.

More specifically - you can use approach described in the article event-driven web applications development with mqtt.

In your case smart meters can post to the platform via MQTT, HTTP or even UDP with custom payload, platform can store all the historical data for you and provide you with MQTT or REST API to access this data and perform required analytical actions.

In that case simplicity means - you are concentrating on smart meters and on application(portal) for your users and let 3rd party platform do all data flows.

Another approach will be to use MQTT broker, where smart meters publishing there all the data and few non-clean sessions subscribed to the data from meters => one for NoSQL DB-engine(any of), another one for constant delivery historic data to S3, may be another to send same messages to front-end. In such approach you will make these processes unrelated and have more choices to change systems during lifetime of your project.