What is a bootloader, and how would I develop one?
I've heard that a bootloader is code that runs on a microcontroller, and is used to get your application code onto the micro. How does that work? Is there anything special that needs to be done to write a bootloader?
If a bootloader is for getting code onto a micro, how do you get the bootloader onto the micro in the first place?
1 answer
Bootloader definition
A bootloader is a program that runs in the microcontroller to be programmed. It receives new program information externally via some communication means, like a serial port, and writes the information to the program memory of the processor. This means that bootloaders can only be used on micros that allow writing to their program memory from the program.
Programming without bootloader
This is in contrast with the normal way of getting the program into the microcontroller, which is via special hardware built into the micro for that purpose. On PICs, this is a SPI-like interface. Some AVRs have a similar interface, others use Jtag. Either way, this requires some external hardware that wiggles the programming pins just right to write the information into the program memory. The HEX file describing the program memory contents originates on a general purpose computer, so this hardware connects to the computer on one side and the special programming pins of the micro on the other. My company makes PIC programmers among other things as a sideline, so I am quite familiar with this process on PICs.
The important point of external programming via specialized hardware is that it works regardless of the existing contents of program memory. Microcontrollers start out with program memory erased or in a unknown state, so external programming is the only means to get the first program into a micro.
When to use what programming method
If you are sure about the program you want to load into your product and your volumes are high enough, you can have the manufacturer or a distributor program chips for you. The chip gets soldered to the board like any other chip, and the unit is ready to go. This can be appropriate for something like a toy, for example. Once the firmware is done, it's pretty much done, and it will be produced in large volumes.
If your volumes are lower, or more importantly, you expect ongoing firmware development and bug fixes, you don't want to buy pre-programmed chips. In this case blank chips are mounted on the board and the firmware has to get loaded onto the chip as part of the production process. I go into detail on hardware programming in this answer.
So far so good, and no bootloader is needed. However, consider a product with relatively complex firmware that you want field upgradable or even allow the end customer to upgrade. You can't expect the end customer to have a programmer gadget, or know how to use one properly even if you provided one. Actually one of my customers does this. If you buy their special field customizing option, you get one of my programmers with the product.
However, in most cases you just want the customer to run a program on a PC and have the firmware magically updated. This is where a bootloader comes in, especially if your product already has a communications port that can easily interface with a PC, like USB, RS-232, or ethernet. The customer runs a PC program which talks to the bootloader already in the micro. This sends the new binary to the bootloader, which writes it to program memory and then causes the new code to be run.
Bootloader architectures
The above sounds simple, but it's not, at least not if you want this process to be robust. What if a communication error happens and the new firmware is corrupt by the time it arrives at the bootloader? What if power gets interrupted during the boot process? What if the bootloader has a bug and craps on itself?
A simplistic scenario is that the bootloader always runs from reset. It tries to communicate with the host. If the host responds, then it either tells the bootloader it has nothing new, or sends it new code. As the new code arrives, the old code is overwritten. You always include a checksum with uploaded code, so the bootloader can tell if the new app is intact. If not, it stays in the bootloader constantly requesting a upload until something with a valid checksum gets loaded into memory. This might be acceptable for a device that is always connected and possibly where a background task is run on the host that responds to bootloader requests. This scheme is no good for units that are largely autonomous and only occasionally connect to a host computer.
Usually the simple bootloader as described above is not acceptable since there is no fail safe. If a new app image is not received intact, you want the device to continue on running the old image, not to be dead until a successful upload is performed. For this reason, usually there are actually two special modules in the firmware, a uploader and a bootloader. The uploader is part of the main app. As part of regular communications with the host, a new app image can be uploaded. This requires separate memory from the main app image, like a external EEPROM or use a larger processor so half the program memory space can be allocated to storing the new app image.
The uploader just writes the received new app image somewhere, but does not run it. When the processor is reset, which could happen on command from the host after an upload, the bootloader runs. This is now a totally self-contained program that does not need external communication capability. It compares the current and uploaded app versions, checks their checksums, and copies the new image onto the app area if the versions differ and the new image checksum checks. If the new image is corrupt, it simply runs the old app as before.
I've done a lot of bootloaders, and no two are the same. There is no general purpose bootloader, despite what some of the microcontroller companies want you to believe. Every device has its own requirements and special circumstances in dealing with the host. Here are just some of the bootloader and sometimes uploader configurations I've used:
- Basic bootloader. This device had a serial line and would be connected to a host and turned on as needed. The bootloader ran from reset and sent a few upload request responses to the host. If the upload program was running, it would respond and send a new app image. If it didn't respond within 500 ms, the bootloader would give up and run the existing app. To update firmware therefore, you had to run the updater app on the host first, then connect and power on the device.
- Program memory uploader. Here we used the next size up PIC that had twice as much program memory. The program memory was roughly divided into 49% main app, 49% new app image, and 2% bootloader. The bootloader would run from reset and copy the new app image onto the current app image under the right conditions.
- External EEPROM image. Like #2 except that a external EEPROM was used to store the new app image. In this case the processor with more memory would have also been physically bigger and in a different sub-family that didn't have the mix of peripherals we needed.
- TCP bootloader. This was the most complex of them all. A large
PIC 18F was used. The last 1/4 of memory or so held the bootloader, which
had its own complete copy of a TCP network stack. The bootloader ran from
reset and tried to connect to a special upload server at a known port at a
previously configured IP address. This was for large installations where
there was always a dedicated server machine for the whole system. Each
small device would check in with the upload server after reset and would
be given a new app copy as appropriate. The bootloader would overwrite
the existing app with the new copy, but only run it if the checksum
checked. If not, it would go back to the upload server and try again.
Since the bootloader was itself a complicated piece of code containing a full TCP network stack, it had to be field upgradeable too. The way we did that was to have the upload server feed it a special app whose only purpose was to overwrite the bootloader once it got executed, then reset the machine so that the new bootloader would run, which would cause the upload server to send the latest main app image. Technically a power glitch during the few milliseconds it took the special app to copy a new image over the bootloader would be a unrecoverable failure. In practise this never happened. We were OK with the very unlikely chance of that since these devices were parts of large installations where there already were people who would do maintainance on the system, which occasionally meant replacing the embedded devices for other reasons anyway.
Hopefully you can see that there are a number of other possibilities, each with its own tradeoffs of risk, speed, cost, ease of use, downtime, etc.
0 comment threads