4

I am new to this forum and this field. I am looking forward to good criticism and tutorials/links/instructing suggestions to build a nice system with as less effort as possible while achieving all things I want to have.


The Situation

I am right now working on a sensorik-system with the following design: Design of already scalable, but still all internal sensorik system.

The main hub is the raspberry pi (newest v3).

  • hosts MQTT-Server for save message transfer
  • hosts python-based (socket-)server for communication with arduinos (hence: sensors)
  • hosts data-storage
  • hosts GUI using Kivy (nice and neat)

The Arduinos (nano) directly control the sensors/actuators/etc:

  • connected via I²C
  • handshake protocol exchanges possible functions of
  • receive situation dependend different scripts from Python-server

The raspicam (v2):

  • provides an image stream which can be previewed or stored.

Project-status: I started with a simpler approach and did not use any server. Hence right now, there is only 1 main program (Kivy-Gui) which has all the relevant code and interfaces. I am using the Pi-camera module and the Arduinos are connected to the pi via I²C interface. It works well. :)


So, why all this hassle if everything is basically running already on 1 machine?

Easily spoken: I want to rescale the whole setup. There will be:

  • multiple Raspberrys each with their own set of sensors
  • external (smartphone) for GUI/Human interfacing
  • external (network) storage
  • (optional) 3rd party devices for e.g. further sensing or IP-cam or...

Therefore I need to add:

  • mQTT-server on Raspberry #1
  • wifi-hub on Raspberry #2
  • script hosting and executing Python-(socket)server on every Raspberry
  • Network-streaming server to ip-stream Raspicam video etc to external GUI on every Raspberry
  • net-drive to storage data from different systems

Finally, I want to still have the possibility to connect a screen to one of the raspberrys and directly have a GUI running on it. Therefore I will modify my existing GUI by outsourcing the executing code. Having the possibility to run the GUI directly on a raspi of choice inherently leads to various problems with the new setup. Just to mention 1: streaming the Raspicam-data to ip and reading it out with the streaming device might be slow or impossible. Here (e.g.) I am thinking to implement a workaround that uses different code decided by MQTT if topic-subscriber and content-poster is the same device.


My question:

What do you think regarding speed and efficiency of this setup with respect to the different tasks? Do you see any major mistakes in my reasoning and approaches? As the Arduinos have the init routine to be used correctly, it is good to have 1 instance running per rasperry with all the parameter etc as active objects. Do you see a more code efficient, stable and faster approuch instead of writing my own python-server? I e.g. read about using a node.js for this, but this might lead to problems with the arduino-handshaking...

  • 3
    Welcome to the IoT Stack Exchange site, Please read the docs on asking good questions (iot.stackexchange.com/help/how-to-ask), at the moment I think this question is a bit too broad to get a good answer – hardillb Aug 25 '18 at 9:31
  • 1
    Welcome to stack exchange. It will be better if you can chunk down your questions to small questions. – Shubhendu Yadav Aug 25 '18 at 13:18
  • 1
    hardillb & @ShubhenduYadav : Thank you very much for the hint. You mean chunking them down by creating now threads or splitting the questions in this thread? – ReneR Aug 26 '18 at 6:16
  • 1
    Yes and try to implement and whenever you are stuck just ask here – Shubhendu Yadav Aug 26 '18 at 6:21
3

Looking at your design diagram, I’m assuming that is what you are thinking of building (noted that some part are already there). I’m going to try and answer based on that.

What do you think regarding speed and efficiency of this setup with respect to the different tasks? It could be a lot better.

Do you see any major mistakes in my reasoning and approaches? Yes.

The first thing I notice is you are not making full use of MQTT. MQTT is a message broker and a very good and fast one. Consider moving it to be the centre of all your communications between different sensors, computers, servers, phones etc. Have everything talk MQTT.

Create a topic structure around the sensors for example:

house/location/device_type/

followed by either in or out with in being the device listens for messages and out is where the device sends messages. This will allow other things to listen to what interests them through the topic wild card methods eg to listen to all humidity sensors in the house you use house/+/humidity/out, so any humidity readings will be received.

With this model, you can have a database that listens and records data. A web GUI that sends on/off messages to the light, an air conditioner that listens to a temperature sensor. It opens up your options. Note the MQTT broker can be running on the same RPi as the data storage and gui, however they should be communicating via MQTT

As the Arduinos have the init routine to be used correctly, it is good to have 1 instance running per raspberry with all the parameter etc as active objects. Do you see a more code efficient, stable and faster approach instead of writing my own python-server? As suggested in the answer to the previous question, I would have the Arduino’s (assuming they are actually ESP’s) use MQTT directly which they are very capable of doing.

  • 1
    Hi Trevor, thank you for your answer. Regarding your questions: Although everything may be connectable via MQTT, the Arduinos (nano) are in fact alreidy connected via I²C to the Raspi given a prior project (used as a first approach). The storage has to be a different device (network storage), because this is the fastest to be accessible over the whole network. How would you organize a video-stream (full quality) from Raspi to other clients in the network? By e.g. using an additional TCP/UDP server? MQTT is not made for traffic by big packages. – ReneR Sep 7 '18 at 8:47
  • 1
    Hi @ReneR, sorry for delay. For your video and any large file data I would use alternatives depending on the requirement. I would use the MQTT to control the devices and communicate that new files are available maybe including URL's and other meta data. I may even look at using html5 for the video, but that needs a lot more analysis. It is a challenge when you have already started a project and come across alternative approaches. Only you can determine the best way forward. – Trevor Sep 11 '18 at 6:54

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.