11

I've recently read about neural networks in constrained environments (in particular, A Neural Network Implementation on an Inexpensive Eight Bit Microcontroller) and their applications to IoT devices (e.g. regression for predicting things based on sensor inputs, etc).

This seems ideal for simple applications where processing is not time-critical, and the data to process will be relatively infrequent. However, further research suggests that training a neural network in a resource-constrained environment is a poor idea (see the answer to Is it possible to run a neural network on a microcontroller).

Does this still apply for Cotton, Wilamowski and Dündar's approach that I linked? Would it be necessary to train a network designed for low resource usage on a more powerful device in my IoT network?

For context, if I had a sensor transmitting the heat setting, I am considering a neural network as described in the paper to predict the desired boiler setting based on that and the time of day, etc. Training would be useful to change the neural network's outputs based on more data provided by the user. This Quora question describes a similar scenario well, and discusses the implementation details for a neural network, but my question is more focused on whether running the network on the actuator itself would work.

  • Curios, do you intend to run a neural network on each sensor/actuator node or at some (semi-)centralized "brain" (then of course the 8-bit restriction wouldn't necessarily apply)? – Ghanima Dec 14 '16 at 18:44
  • @Ghanima if possible, I'd like to do it at the actuator node to save the extra layer of complexity, although I'm not sure whether that would work with the limited constraints. – Aurora0001 Dec 14 '16 at 18:45
  • @Aurora0001 The particular controller you site here may not be suited to train your NN but, there are concerted efforts in the embedded computer-vision world to do exactly that. If you are trying to find a micro-controller with an architecture suited for such tasks, I suggest that you look to the companies developing these kinds of hardware for the computer-vision industry. I am sure some of it can be repurposed and adapted to your requirements. A good place to start – grldsndrs Dec 15 '16 at 5:12
  • @grldsndrs fantastic, thanks for the reference. Feel free to post it as an additional answer if you feel that it's enough (I'd certainly be happy with it) – Aurora0001 Dec 15 '16 at 16:58
  • 1
    An 8 bit processor can do anything a wider-word processor can, only perhaps (depends on the task) more slowly. However, 8 bit processors tend to have limited native address space, which means they must use indirect means to manage very large memories, and in the case of microcontrollers tend to ship with comparably small amounts of on-chip memory. There's increasingly little cost difference beyond the lowest-end - the primary driver of MCU cost is arguably memories, not ALU width. – Chris Stratton Jan 7 '17 at 2:39
9

According the first paper, running is not a problem. That was the purpose. Only there is a limitation on the maximum weights:

Currently the limitation on the architecture embedded in this microcontroller is limited only by the number of weights needed. The neural network is currently limited to 256 weights. However for most embedded applications this 256 weight should not limit the system.


As for training, as far as I understand the implementation described, the PIC controller receives parameters from an external source.

The neural network forward calculations are written so that each neuron is calculated individually in a series of nested loops. The number of calculations for each loop and values for each node are all stored in a simple array in memory.

[...]

These arrays contain the architecture and the weights of the network. Currently, for demonstration purposes, these arrays are preloaded at the time the chip is programmed, but in the final version this would not be necessary. The microcontroller could easily be modified to contain a simple boot loader that makes use of the onboard RS232 serial port which would receive the data for the weights and topography from a remote location. This would allow for the weights or even the entire network to be modified while the chip is in the field.

I suspect that the training is performed externally as well.

The paper also gives references for Neural Network Trainers which were probably used to determine the values preprogrammed into the PIC's memory.

Now, I have looked into the first one which describes network architectures and algorithms to use with them. But the Neural Network Trainer software used here is implemented in MATLAB.

Currently, there is very little neural network training software available that will train fully connected networks. Thus a package with a graphical user interface has been developed in MATLAB for that purpose. This software allows the user to easily enter very complex architectures as well as initial weights, training parameters, data sets, and the choice of several powerful algorithms.

I have to mention that the fully connected networks has lower weights number for a same task than a layer by layer architecture. That makes it more suitable for microcontrollers.

I am not a neural network expert and it is quite complex so I can be wrong, but based on these papers I would say that Cotton, Wilamowski and Dündar's approach requires an external, more powerful platform to perform the training.


About running a neural network on a microcontroller, ST Microelectronics just announced a toolkit STM32Cube.AI: Convert Neural Networks into Optimized Code for STM32 to convert pre-trained neural networks from popular libraries to most of their STM32 MCUs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.