What IoT services are available for storing/sending/publishing generic data in the cloud?

Question

What IoT services are available for storing/sending/publishing (and opposite operations) generic small amount of data in the cloud?

I am looking, for example, for a service where a device could store a value in the cloud. And some other entity (another device, a website with some JS code, a web server, a mobile app) could retrieve this value.

It could be some kind of asynchronous communication, for example for storing and retrieving something as small as a key&value pair, <255bytes, an integer, a string, at most a small JSON object. The service could offer a REST API (so it could be accessed by a large variety of languages) with some token to authenticate and the key & value to store.

A detailed example of a use case is:

There is a temperature sensor at home, and I want it to store the value in the cloud(somewhere out of home). In such way, I could access it, regardless of my home connection is down or not. Morevoer, this would avoid keeping and maintaining a dedicated server + portForwarding + DynamicDNS.

So far I have not been able to find something like this, but occasionally, I have found some examples of what I mean to describe:

• Particle.io publish() subscribe()

• Blynk Virtual Pins

What other similar (free/open) alternatives are there?

Answer 1

Take a look at these services:

• Ubidots
• ThingSpeak

Both of these services will accept simple key/value data from a device. I believe they both have libraries ready to import for Particle devices since you mentioned that.

Answer 2

Many cloud providers like Amazon, Microsoft, Google, IBM etc., trying to attract IoT space by providing easier ways to send/store/analyze sensors data to their cloud. Even they are to acquire hardware vendors to increase their scope in IoT.

I didn't use any services other than AWS, so I can explain my experience with AWS and how we integrated this for production usage.

---

Scenario:

We have hundreds of sensors each one sending 184-428 bytes of data every minute to the local gateway which aggregates the data and stores the data locally and sends the same data to AWS cloud. We also have special-cased sensors which send data directly to the cloud.

Cloud Services

We are using AWS IoT, AWS S3, AWS DynamoDB, AWS Lambda, AWS API Gateway, AWS SNS, AWS Cloudwatch, AWS RedShift to build entire solution. Basically, these are not specific to IoT(Except AWS IoT) as we can use them for mobile, web.

Gateway uses AWS IoT SDK to connect, authenticate, and exchange messages with AWS IoT using the MQTT, HTTP, or WebSockets protocols(We are using node JS SDK connecting via MQTT). We are MQTT broker locally on the device gateway and bridging it to AWS IoT endpoint from there we are running instant checks on the data received (using rules engine, AWS Lambda functions) and storing them into DynamoDB archiving into S3,Glacier(Storing is done without writing single line of done just using AWS triggers to store data).

Answer 3

It is for experimental or test use only but maybe it will change in the future.

So my suggestion is to use MQTT, more precisely its Mosquitto implementation. They host a test broker to which you could connect your subscriber and publisher clients. (_{^{Here is a guide about the setup process on Windows 7.}})

Note the following:

You are free to use it for any application, but please do not abuse or rely upon it for anything of importance. You should also build your client to cope with the broker restarting.

But basically you can publish temperature data to this broker.

On subscriber-client side I have recently used this Android application. It is a very basic application, still in development but for test purposes it is very great. Received messages are shown on a dashboard, nothing fancy just the bare values.

I have started to use both of these as a first stage of a my MQTT learning process and found both great for beginners.

Answer 4

There are two components to this:

1. How do you want your data stored? There is no real way to make a "generic" data service that will really fit all needs. What you want is called a "Time Series Databases", and there are hundreds of them because every detail of how you store data matters at scale. (If you are not at scale, just store it in any old database, it will work for a while.)

   Each time series database was written because the others didn't do exactly they way they wanted. For example, consider how Graphite stores it's data: Every metric (say temperature from one source) is stored in a fixed-size file. No matter how often you send metrics, or how long you send them, the file is a constant size.

   The downside is that older data is at a lower resolution, and after a set interval you define (such as 1 year) the data is thrown away. But the upside is that it's just as fast to graph a day as a year, and metrics don't grow in size over time.

   In other storage systems, generating a graph for a year might involve fetching millions of data points, and might require massive amounts of data storage.

   The big downside of Graphite is that every metric creates a new file, so if you have dynamic metrics (say cloud boxes coming and going), it might not be a good fit.

   Contrast that with Prometheus, where metrics are stored mostly by time. You can have lots of dynamic metrics, and it's fine. But don't try to store those metrics for the long-term, it will take forever to go back and read them.

   No one size will fit all.

   P.S. Graphana is an excellent way to visualize your data. It has plug-ins for most time-series databases.

2. Who is going to store your data? There are thousands of places like the ones you mentioned. It's so easy to spin up a time-series DB in the cloud, but it's REALLY hard to make money at it.Most of these companies will go out of business after a while, or start price gouging. (Even keeping their prices steady is price gouging -- since the cost of computing is going down constantly.) Many times, they find they can't attract as many new customers as they though, so they try to raise prices (under the guise of changing their pricing model). It turns out it costs a LOT of money to store everybody's data...

   I recommend self-hosting, or use a reputable cloud provider such as an AWS CloudWatch. (Expensive if you have a lot of metrics, but free for under 50 metrics!)

Answer 5

uBeac is a new freeware visualization tools which we have developed and it is Beta version. It is not opensource, but fully free to use.

You can define a gateway and you will get a unique URI. You can set the URI in you gateway or device to send HTTP/MQTT data to.

These are some of its features:

• Team, building, floor plan definition
• Real-time data visualization
• Custom dashboard design using different widgets
• Define sensor types, units, and prefixes Map visualization
• ...

It supports generic Json data format and different predefined gateway as well. If you don't want to use predefined payload formats, they are open to develop your custom payload processing.

Answer 6

I'm surprised no one on here mentioned Dweet. This is a super simple, super fun way to get things communicating. You should for sure try it out, because hey, it's free!

Answer 7

flespi provides free and commercial cloud services:

• MQTT Broker service for PUB/SUB operations and you may even store the data in the retain messages;
• Storage engine for key/value operations, targeted especially to timeseries data and generic file storage (CDN). Operates via REST API.

Obligatory disclaimer: I’m working for the company that develops the flespi platform. While I did my best to stay objective, as always on the Internet, please double check all information in this answer to rule out bias that may affect your decision.