I'm working with industrial machines, not always connected to the internet in industrial environment.
I'm new in IoT and for my current setup, I have 2 usage: analytics and live data. For the analytics, I collect data with fluentbit and it ends in elasticsearch, where I fetch the data later on. I don't have anything setup yet for the live data.
The type of data I am sending is position of my vehicles (autonomous machines and robots), and part of the sensor data.
Now, I see many applications are using MQTT to send the data to a streaming process like kafka. My question is, can I keep and extend my setup to use live data, or it's not realistic without MQTT?
Edit: My IoT devices are robots or material handling equipment (e.g forklifts). They are working between 12 and 18h/day, then charging. They will likely have no internet access and will be on a local network.
My current approach is to send with fluentbit to a local PC that has access to internet which forwards it later on to ES. The PCs on each robot will have at least 1 core free and 1 Gb of ram available.
As I am testing, I have only 2 robots to test latency... so i'm not so sure what i should expect.
Last but not least, regarding the live part, I believe what I lack understanding of is the pipeline. I would need to collect the data for live display but also for future analysis. My current setup puts everything in elasticsearch, which doesn't sound right, but I don't know why. I'm spending time on reading about it, but most resources I find have emphasis on the tools instead of their usage.
Edit 2:So, I'm ditching fluent bit regarding the data acquisition. Since i need to stream data with different and unreliable network environment, I am going with MQTT, then transfers the data to Kafka and stream it from there. I did a small POC this week and it works fine. The challenge is the scaling which I'm taking care of right now.