Bootloader black magic - kiwi & dflash

Introduction

We know that "flashing" an ioNode involves getting dflash to send a sequence of "instructions" to it through USB. But how exactly does this work?

Here we will attempt to demystify this process by looking at the different components and some of the technical details behind it.

Clearing any misconceptions

First, let's get some common misconceptions out of the way.

Basic electronics

As presented in USB communication, the ioNode features a microcontroller and a dead-simple USB-to-serial chip (FT232).

No magic here - dflash communicates with the ioNode exactly the same way your custom application would.

Microcontrollers are simple

Microcontrollers like the AVR family are simple machines - they execute instructions stored in their flash memory as soon as power is applied, and do so until power is removed.

The microcontroller has no 'hardwired' or internal circuitry built-in to understand programming instructions coming through its UART from dflash.

Therefore the atmega1284p chip on the ioNode wouldn't know what to make of these instructions.

Hinting at the magic

We already know a few important details. To flash our ioNode we need to reset it (pulling the RST line down to 0V and bringing it back up to 3.3V).

Once the ioNode resets, we have a few seconds during which we can start flashing it.

If nothing happens within this time, whatever application was last flashed starts executing.

This is telling us something important: something very important is happening at startup, just before the application starts. Something is running _before_ our application, and that something seems to be where the flashing occurs.

That "something" is often referred to as a bootloader.

What's a bootloader?

The name "bootloader" probably rings a bell for many people. Some readers may even start getting confused at this point, rightfully thinking that "bootloaders like GRUB are used to load a kernel into memory and start the operating system! What use could that be on a microcontroller with no operating system?".

In the context of microcontrollers, bootloaders have a very different (though somewhat related) purpose: they allow writing to the microcontroller's flash memory (program space).

Microcontroller security

To prevent an application going crazy and messing itself up, the program space in the microcontroller is "protected" against writing. The only way to write to the program space is to use specific AVR instructions, but most importantly, these instructions must be executed from a specific region of the flash.

This brings us back to the idea of a "bootloader", a small "mini-program" that is located in a specific region of flash memory and that runs before anything else. Because that tiny program is located in the specific region of flash memory we mentioned above, it has the ability to write to the rest of the flash.

Introducing Kiwi

Kiwi is Dooba's bootloader for AVR microcontrollers, and every ioNode is pre-loaded with it.

When power is applied to the ioNode, Kiwi starts executing. It sets up the USB UART and waits for a few seconds (4 secs at the time of this writing).

If a programming instruction is received by Kiwi within that time, the timer is dropped and the flashing process starts.

If no instruction is received during that time, Kiwi starts executing whatever application is stored in the flash memory.

dflash

Having a bootloader is nice, but it doesn't allow us to do much until we have something that can send the correct instructions to that bootloader.

Kiwi understands something called the "dflash protocol", which, as the name implies, is also implemented by the dflash tool.

This is where the magic happens. We can use dflash to send "flashing" instructions to Kiwi, which will in turn write our application to the flash memory of the microcontroller.

So... how is Kiwi flashed? - The chicken and egg problem

So now that we know all this, a question may start to bubble up: how do we initially flash Kiwi itself?

When we get a fresh new microcontroller from the manufacturer, it can not yet be flashed through UART/USB since Kiwi is not present yet.

Well, microcontrollers such as the AVR family usually feature a built-in mechanism that allows flashing through a crude manufacturer-defined interface.

In the case of the AVR, programming can be done through ICSP using specialized programmer hardware. This is how we pre-load the ioNode with Kiwi.