How can I do incremental updates with a flash that can only be erased block-wise?

Question

Scenario

I want to update a low cost IoT device over-the-air with new firmware updating the device's microcontroller(s). The microcontroller memory is flash memory in the 32k to 128k range (every cent counts). This cheap memory has one major limitation: it can only be erased block-wise.

Question

Does that mean I cannot make an differential (delta) updates? Do I always have to update the whole controller memory (or at least substantial parts)?

I want to reduce the need to flash everything and risk bricking the device completely as much as possible. Are there existing strategies when flashing microcontrollers over the air?

Answer 1

The simple answer is yes - you need enough blocks of flash to support bootloader and A/B code images if you want high reliability. Before activating the new image, you can write the whole thing, verify it and potentially retry.

However this is an expensive/reliable strategy and there are things you can do to reduce the overhead. Low level support for OTA updates may also come as part of the device firmware or OS, so you can avoid rolling-your-own unless you want to learn. This feature might be described as FOTA.

Partitioning your code base allows for incremental updates, best case the bootloader is able to bring up the network connection, download and verify code without needing any fall-back user code. With a local gateway, management of this task can be delegated from the low-cost endpoints.

Many devices have a small amount of word-erase flash, and even failing this you can usually set bits without needing to erase a whole block. These features can be used to manipulate jump tables and stitch together code which is updated in block sizes chunks. Even if you initially planned for a full A/B code space, you might need to fall back to a more complex scheme when the codebase grows too much.

To clarify the functionality which can be achieved with a sophisticated firmware-over-the-air solution, the bootloader and potentially a primary communication stack can remain resident whilst the full remaining user application space is re-flashed. This doesn't need any overhead (particularly if the block partitioning is soft). In the scenario where the communication stack needs to be upgraded, the region generally used for application code can be temporarily used during download and verification. Achieving this requires some support in the SoC, but 2nd and 3rd generation devices designed with this in mind do exist already.

Answer 2

I want to reduce the need to flash everything and risk bricking the device completely as much as possible. Are there existing strategies when flashing microcontrollers over the air?

Aside from your code that does the updating which would be relatively static, you need to keep two images in your storage: an active image and a backup image. Whenever, you need to update, do it in the backup, then switch that to be active. Once stable, update the old active image which should now be your backup.

With that in mind, you can use wear-leveling algorithms when updating both images. The code for such algorithms might take around 10-15% of the total storage, but it's well worth it in extending the device's lifespan.

Wear leveling is typically managed by the flash controller, which uses a wear leveling algorithm to determine which physical block to use each time data is programmed. There are two types of solid-state drive (SSD) wear leveling: dynamic and static. Dynamic wear leveling pools erased blocks and selects the block with the lowest erase count for the next write.

Static wear leveling, on the other hand, selects the target block with the lowest overall erase count, erases the block if necessary, writes new data to the block, and ensures that blocks of static data are moved when their block erase count is below a certain threshold. This additional step of moving data can slow write performance due to overhead on the flash controller, but static wear leveling is considerably more effective than dynamic wear leveling for extending the lifespan of solid state devices.

(Techtarget.com: Wear leveling)

Answer 3

Freescale Semiconductor describes a robust way of over-the-air firmware upgrade for their Kinetis Microcontrollers.

It is called: Program Flash Memory Swap.

Systems using flash memory swap

In devices with two or more internal flash blocks that support swap, the memory base address of each flash block can be exchanged. The address location of each flash block will thus be swapped in the logical memory map of the device. After a reset, the built-in flash swap system essentially selects which software executes by the location of the flash block in the logical memory map. This allows a code back-up system with the added ease of programming. You can execute out of one block while erasing/programming the other block. On Kinetis devices, the flash swap system monitors/controls all the steps of switching from the old application to new; there is an added assurance of reliable operation in the case of a power loss during one of those steps.

Advantages

• Ease of programming. Application always executes out of the lower block in the memory map.
• Power loss tolerant.
• No bootloader required. No delay to the start of the main application.
• Well suited for a multi-tasking OS. Minimal application downtime. In a multi-tasking system it is possible to continue to execute the main application tasks while background tasks are running to update the new copy of the application.
• Backup copy of code. Possible to revert to the known working application.

Disadvantages

• Additional flash memory space required to store backup copy.

You can update blocks and then swap them.

The document linked holds detailed description.

It ensures safer firmware upgrades but as it requires more flash memroy, certainly costs more. Also not applicable for any type of microcontroller, only those which support internal flash blocks swap.