-1

I have 10,000 machines (to make jute-bags), and each machine needs to send data to a remote server every 2 seconds. The data consists of machine id, machine state and various other information collected from different sensors.

I have successfully integrated the ESP8266 WiFi module with the machine. It collects all necessary data, and transmits it to the router, which then uses the internet to send data to my remote server. As a demonstration to the factory management, I successfully sent data of 10 machines, every 2 seconds.

But now, I have to do the same task for 10,000 machines.

I have absolutely no idea as to how much bandwidth I need and how many machines I can connect to a single router?

If my question seems incomplete, please feel free to ask me more questions.

4
  • 1
    Have you tried to measure the bandwidth at all? You must know the size of each message you send.
    – hardillb
    Feb 17, 2021 at 18:26
  • Where are these 10K machines? They’re probably not all in the same place, are they? I predict a world of pain if they are, especially as the ESP8266 is 2.4 GHz only. Also, what’s the environment? Lots of metal all over the place probably isn’t the easiest environment for wireless. What’s the size of the place(s)? Are you sure you need to send data every 2 seconds from each one?
    – jcaron
    Feb 18, 2021 at 13:27
  • Also, what protocol are you using? You use a lot more bandwidth if each update is its own HTTPS transaction rather than a single message inside a socket you keep open. And do you really need to have the server offsite?
    – jcaron
    Feb 18, 2021 at 13:31
  • server is offsite. the machines are all spread out. a single unit of area has about 10 machines. Mar 5, 2021 at 16:16

1 Answer 1

1

We're lacking quite a bit of data, but let's run a few numbers.

  • If each message is about 1000 bytes, then 10K * 1000 * 8 / 2 = 40 Mbit/s. Not even considering the Wi-Fi part, that's quite a lot of upstream bandwidth.

  • If each message is closer to 10 K bytes (quite possible if yous end each message as its own HTTPS request without keepalives or TLS session caching), then we're in the hundreds of Mbit/s.

  • If you manage to stay below 100 bytes (possible if you send short messages in an existing socket or Websocket), down to a few Mbit/s. More reasonable but still quite a bit of traffic.

Also consider that those 10K devices at 1 message every 2 seconds means your server receives 5K messages per second. If the server is just forwarding them to another connection, fine. If you are storing in a database, we're talking thousands of IOPS. By far not impossible with decent SSD-backed storage, but not on you run-of-the-mill single HDD of course.

Also remember that if that data comes about 1 Kbyte on disk per message (very possible if you include indexes, etc, possibly much more depending on your DB schema), you're writing 10K * 1024 * 86400 / 2 = 442 Gigabytes per day!

But let's consider the upstream and server parts are sorted.

You'll never run 10K devices on a single AP. It's just not possible. Most "consumer" APs will choke beyond a few dozen devices. The best ones will stop at a few hundred in the best of conditions (using multiple radios on multiple bands). That's just for maintaining state, keys, and so on, we're not even talking about the devices communicating yet.

The ESP8266 operates only in the 2.4 GHz band. That band is quite small and also quite busy. We'll consider the case of "most countries" (the US has more restrictions). You only have 3 non-overlapping channels in 802.11b or 4 in 802.11g/n at 20 MHz.

I don't think you'll be able to have more than about a 100 devices in 2.4 GHz on a single AP. So we're already reaching 100 APs!

But with only 2 to 4 non-overlapping channels, the risk of interference between neighbouring devices and APs is probably too great for things to run reliably.

Remember that the speeds quoted for Wi-Fi (72 Mbit/s max for the ESP8266) are just raw stream data rates. They do not take into account overhead, preamble, ACKs, RTS/CTS frames, guard intervals, inter frame space, collisions, retransmits... When sending very short frames, the overhead can quickly use a vast majority of the airtime, so you can't compare 72 Mbit/s with the 40 Mbit/s quoted above.

Remember as well you'll have (at least) TCP ACKs, and 802.11 is half-duplex.

At this scale, I'm not sure what the right solution is. My first instinct would go towards Ethernet (and possibly PoE), but this is a massive setup (that's quite a few switches, and quite a lot of cables).

A probably better option is to have a single ESP8266 with sensors for multiple machines to reduce the number of devices, but I have no idea if that's possible in your case.

2
  • Thank you sir for your valuable time and advices. After I posted the question here, I asked it to many other people who are in the networking field. Everyone gave me this exact answer, that LAN Ethernet is the solution. And it shall be a humongous amount of work. 10000 machines are divided in such a way, that each division has max 10 machines. I am using 1 module for 2 machines. The data is then sent to an onsite database. From there, Ethernet will transport it to remote server. Mar 5, 2021 at 16:22
  • @sukhbir1996 If you can somehow use 1 module for the 10 machines that would probably help a lot, but in many case, there is going to be a LOT of cabling. But heck, those 10000 machines probably already have a lot of cabling and other infrastructure, for power, for command and control, to feed them whatever material they use, and so on. That's quite a bit factory...
    – jcaron
    Mar 5, 2021 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.