How can IoT cloud services be made more reliable without excessive cost?

Question

In one of my previous answers, I mentioned that when designing a cloud service to connect to IoT devices, it would be best to make the servers redundant in some way so that if one data centre or server fails, the whole system still works.

Sean Houlihane pointed out that having true redundancy would probably double the costs for a provider, making it technically infeasible.

So, I'm interested to know how a cloud service (perhaps like the service used for the Nest thermostat, which went down in the S3 outage) could be made more reliable without duplicating every component, in a way that would be less disruptive to a company's business model.

The sort of device I'm thinking of is something like a smart thermostat that needs to synchronise data from a phone app (outside the home's local network) to the thermostat itself, and stores the state in cloud storage like S3.

What can I do to ensure that the cloud servers have a high availability without running two copies of every server in different locations?

Answer 1

...double the costs for a provider, making it technically infeasible.

Doubling costs isn't really "technically infeasible", it justs make the situation more expensive. However, it's not really that bad.

I work for a service that rents dedicated hardware at three different data centers from three different vendors, one in the west, one in the east, and one in the midwest. Our client application automatically, constantly, and seamlessly switches to whatever center responds quickest. (The server side doesn't even have to do "load balancing"; the client side does "load distribution" instead.) Since starting up years ago, there has never been a moment when at least 2 of the 3 centers wasn't responding.

The costs to rent and operate those servers is minuscule. We could easily expand to more data centers using petty cash, but it's not necessary, because the three existing systems provide plenty of redundancy and plenty of throughput.

Computers are cheap. Outages are expensive and get you bad press. Redundant servers is the cheapest solution.

Answer 2

Most importantly you'd need a reliable load balancing infrastructure. That's where the real problem is situated. The services that provide load balancing and indexing where the other stuff is. Having redundant computing or storage is easy. It's also not really that expensive. As you point out there are enough models where most of the cost only applies when actually used.

Like on this Wikipedia load-balancing page the load balancer is mostly pictured as singular instance. Even though that's a vast simplification there is very little cross-provider load-balancing. Thus, if your AWS load balancer shuts down there is no use in having tons of Google Cloud Functions and OneDrive storage available. Your IoT devices won't find their way towards those services. They will be looking at the false domains/IPs for your cloud service.

The rest is set up redundantly anyways. What happened to Amazon is that they inadvertently shut down their single point of failure. That one server with the big red never shut it down sign. By mistake they took down the indexing server for the S3 on the east coast.

One of these subsystems, the index subsystem, manages the metadata and location information of all S3 objects in the region. This subsystem is necessary to serve all GET, LIST, PUT, and DELETE requests.

Essentially the service that knows what's where. However you design your infrastructure there will always be similar points. Just take it from Amazon, even the number #1 cloud provider didn't dare to touch that system.

While this is an operation that we have relied on to maintain our systems since the launch of S3, we have not completely restarted the index subsystem or the placement subsystem in our larger regions for many years.

If you read their blog to the end, they have no actual solution to that. They are just trying to reduce the chance it happens again. There will always be critical points where you can't have redundancies—or redundancies that area extremely costly.

At the end it likely is cheaper to have the service go down and wait for the big three cloud providers to get it back up.

Answer 3

One idea might be to use multiple Function as a Service providers (e.g. AWS Lambda, Azure Functions or Google Cloud Functions) so that if one fails, the other services can be used instead.

As serverless (Function as a Service) providers typically charge by the number of invocations instead of by the time used, the cost implication is probably not so concerning—if you used AWS Lambda 99% of the time and only used Azure when AWS was down, you'd still only be paying once for each function invocation rather than paying for the use of multiple physical services.

Of course, most cloud services will rely on more than just computation—the storage aspect is still important, and would likely be a different single point of failure. You could potentially replicate that too across different providers, but at this point it starts to get expensive, because you are paying multiple times for the storage of your data, and it could easily get out of sync, adding complexity to the service.

Imagine setting your smart thermostat to 20°C on your phone—it might synchronise that data with cloud server #1, which then crashes before the data is pushed to the other storage, so cloud server #2 thinks that you still want your thermostat at 10°C, and you get home to a freezing house.

Essentially, making reliable IoT services is hard, but criticising poor design is easy, it seems.