Learning Tock from the ground up · Chapter 5
‹ ContentsChapter 4's guarantee came from the compiler: a capsule cannot reach what it cannot name. That works because the compiler saw every line. This chapter is about code the compiler never saw.
Plan on forty minutes. Chapters 1 to 4 are assumed: what an address is, how the kernel gets started, and what a capsule may touch. There is no new Rust here. Almost every line in this chapter is arithmetic.
In the book: doc/processes and doc/tock_binary_format, which specify the header this chapter parses by hand.
Each is one sentence now and repeated in context below. Two of them are about time rather than memory, which is new for this series.
Chapter 4's answer rested on one fact that was never said out loud. Every line of a capsule went through the same compiler, in the same build, under a rule its crate could not switch off.
Take that away and there is nothing left. An application is not built here. It is a separate binary, produced by a separate set of tools, and the kernel meets it for the first time as bytes sitting in flash.
Tock's specification states the position outright. Applications are untrusted, and so "the kernel uses hardware memory protection to isolate the kernel from processes".
That is what buys the rest. Applications may be "written in C (or even assembly)" and are still meant to work, because none of the guarantee depends on how they were built. Everything on the far side of that line, Tock calls userspace.
So chapter 4's question — what may this code touch? — has to be answered a second time, and none of the first answer survives.
Read down the two columns. Four of the five questions are not about the code at all — find the one that is.
| A capsule | A process | |
|---|---|---|
| who compiled it | The same build as the kernel, in this tree, at the same moment. | Not here. It was built somewhere else, possibly on another machine and possibly years earlier, and it arrives as bytes. |
| what language | Rust, and Rust with #![forbid(unsafe_code)] at the top of its crate. | Anything at all that ends up as instructions this processor will execute. |
| where it lands | Inside the kernel binary, in the 255 kB of flash it is linked into. | A separate 256 kB region that the kernel binary reserves and does not occupy. |
| what checks it | The compiler, once, before anything ran. | Sixteen bytes of header at boot, and hardware for every instruction after that. Figure 2 is the sixteen bytes. |
| what stops it | Nothing. Chapter 4 ended on exactly that. | The scheduler. A process has a timeslice, and the kernel gets the processor back whether the process agrees or not. |
Something has to be trusted, or the kernel cannot even work out where one application ends and the next begins. What it trusts is small, fixed in size, and sits right at the front: the five required fields of a TBF header.
Click a field. The offsets are byte positions from the very start of the application.
The version must be 2. On anything else the kernel does not try to read further, and it does not report a broken application either — it treats what it found as the end of the list. This is the field that ends the search, and Figure 3 is why that matters.
How many bytes of header follow. Below 16 is refused, because 16 is what these five fields take. Above total_size is refused too, because a header cannot be longer than the thing containing it.
The length of the whole application: header, code, everything. This is the number that says where the next one starts, and it is believed before anything has been verified. The parsing code says so in as many words: "we trust the value in flash".
The lowest bit of this word is an on switch. An application with that bit clear is skipped rather than started, which is how a tool can leave one installed and switched off without erasing it.
Every four-byte word of the header except this one, exclusive-or'd together — the loop that builds it has an empty body for word three, which is this field. It catches a header damaged on the way in. It is not a signature and does not pretend to be one: anyone who can write flash can write a matching checksum.
With those sixteen bytes the kernel can do the only thing it needs to do at boot: find every application, one after another, starting from a single address.
The kernel boots and needs to know which applications are installed. Where does it read the list from?
There is no such table. Nothing writes one at flash time, and nothing would keep it in step if an application were replaced.
Right. It starts at one linker symbol and walks, reading a header, stepping forward by the length that header gives, and stopping at the first thing that is not an application. Every boot, from scratch.
The board declares how many it will make room for, which is a different thing: it is the size of an array, not a record of what is actually there.
Nothing in an application runs before the kernel has found it, so there is nobody to ask.
Put something in each slot, then walk flash from _sapps. Erase the middle one and count how many applications the kernel finds — the answer is the reason nothing here is a list.
Nothing walked yet. The kernel has one number, _sapps, and no idea how many applications are out there. Walking. Each entry is believed as far as its length, which is what says where the next one starts. Finished, and it finished by running off the end of what was written. Erased flash reads 0xFF in every byte, so the version field of the next entry reads 0xFFFF, which is not 2. The list ends because the next entry fails to be an entry. Finished early, and this is the one worth sitting with. An entry whose version is not 2 ends the walk where it stands, so every application after it is never looked at. Nothing reports this: from the kernel's side it is indistinguishable from having reached the end.
A name the linker script leaves behind, and nothing else. On this board it works out to 0x10040000, the first address past the end of the kernel. Whatever has been written there since is what the kernel is about to find.
Eight is enough: version, then header_size, then total_size. That is all it takes to know how long this application claims to be, which is all it takes to find the next.
Two is the only version this kernel knows. Any other value ends the walk, and the comment beside it says why: "it is very possible the header we started to parse is intentionally invalid to signal the end of apps".
Under sixteen bytes cannot hold the five fields; longer than the application cannot fit inside it. Either way this one is skipped — but the walk carries on, because total_size is trusted on its own.
Applications are laid back to back with nothing between them. Adding total_size to where this one started gives where the next one starts. There is no pointer anywhere: the length is the link.
Erased flash reads as 0xFF in every byte, so a region nobody has written has a version field of 0xFFFF, which is not 2. The list ends because the next entry fails to be an entry at all.
That address is not a decision the kernel makes while it runs. It is fixed when the kernel is built, by the linker script: the file that decides what lands where. On this board it comes out at 0x10040000, and the same file divides the whole chip into regions.
Pick a region. Every number here is read off this board's linker script.
Chapter 3's kilobyte. The boot ROM searches the first four kilobytes of flash for the block that lives in here, before a single instruction of Tock runs.
Everything in this tree that ends up in the binary: the scheduler, the syscall layer, the chip crate, and every capsule chapter 4 was about. The 255 kB is what the region reserves, not what the binary comes to; the slack at the end is unused flash.
The applications. The linker marks this section as one the loader should not fill in, which is why applications are flashed separately from the kernel. Tock does have a capsule that lets an application write its own slice of it back; this board does not include one. _sapps is the first of these addresses.
The only one of the four whose start is not a round number. It begins wherever the kernel's own zeroed data happens to end, so it moves whenever the kernel is rebuilt, and it runs to the last byte of the 520 kB this chip has.
Nothing in this repository builds an application. They are built out of tree, in whatever language, and arrive as a .tbf file: a TBF header with a program behind it. Tock's own getting-started guide points at two places to get one, libtock-c for C and C++ and libtock-rs for Rust.
Getting one onto the board is two lines of objcopy and a choice of target, and all three are about the region Figure 4 just showed.
One thing to settle before you plug anything in: the Makefile those targets come from is this board's, and this board is the plain Pico 2. There is a raspberry_pi_pico_2_w too, on the same chip, and its Makefile is one line that includes this board's. Nothing in this chapter turns on which one you own — flash and RAM do not know what the pins reach — but chapter 4's panic blink does.
Click each one. Two of the three are the same on any desk; find the one that depends on what you have plugged into yours.
The section the linker marked as one not to load becomes one the loader will write. Figure 4's third row said the loader should leave it alone; this is the line that changes its mind. It changes it in a copy of the kernel, not in the kernel itself.
Your .tbf goes in at 0x10040000, which is _sapps, which is where Figure 3's walk begins. Nothing rewrites it on the way. What Figure 2 reads out of it at boot is byte for byte what you handed over here.
Both targets run the same two lines, and on this board those two lines make an ELF that claims file content for RAM. program stops there: picotool refuses it with ELF contains memory contents for uninitialized memory and leaves an empty file — issue #4770, open since April. program-openocd runs the same two lines and hands the result to a debug probe rather than a converter; whether that fares better is untested. Once it is fixed, program copies a UF2 to a mounted drive. That path defaults to a Linux one, so on a Mac it builds the file and tells you to set the variable yourself.
make flash-app, which several other boards define and this one does not. The two objcopy lines are the part that does not vary.Figure 4's last row was the pool. This is one slice out of it.
The kernel starts at _sappmem, the first free byte after its own data, and hands a run of RAM to each application it found in flash, in the order it found them. This board has room for four at a time.
A process gets one such run and nothing else. Seven boundaries divide it, and they are not all owned by the same side. What follows is ProcessStandard, the implementation every board in this tree uses; the kernel itself asks only that a process be something the scheduler can run.
Move the two lines that move. They come towards each other out of one gap, and when it is gone both sides start failing — then click a boundary to read what it is.
Room either way. Both lines can still move, and neither side has been told anything about the other's appetite. The gap is gone. The process asking for more heap is refused, and so is a driver asking for its first grant — and whichever asked first got it. Neither is a bug in anything: it is one allocation deciding another's fate, in a system with no allocator to arbitrate.
The end of the allocation. Everything between here and kernel_memory_break belongs to the kernel, sits inside the process's own slice of RAM, and is what Figure 7 is about.
The kernel's floor. It starts at the very top and moves downward as grants are handed out, which is chapter 8. Below it is the process's.
The ceiling, and the only line here the process can move. Above it is refused. It begins thirty-two bytes above memory_start — thirty-two, not thirty-two thousand — and the application has to raise it with a syscall before it can do anything at all.
Where the heap begins. It grows upward, towards app_break, and the distance between the two is how much room the application has left.
The application's globals: whatever it started life knowing. Below this line the stack begins, and the two never move towards each other.
How far down the stack has got so far. Everything between here and memory_start is unused, and running out of it means running into the bottom.
The bottom, and the number the kernel hands the process as its second argument in Figure 9. The process is told this address; it has no way to work it out. Everything it may touch at that moment has been written to zero first, so it cannot read what the last process to hold this memory left behind.
Chapter 4 noticed something in passing that pays off here: there is no allocator. The kernel never asks for memory, because there is nothing to ask.
So when it needs somewhere to keep its record of a process, there is exactly one place available — the memory it has just handed that process.
Click each one, and watch where it lands. Three things the process never asked for, and one thing all three have in common.
The kernel's whole record of this process: which state it is in, its saved registers, its identifier, its counters. The reservation is written as size_of::<ProcessStandard>() plus its alignment, so the structure describing the process is stored in the process.
Room for ten calls waiting to go back into the application. Ten is a single constant, CALLBACK_LEN, and it belongs to the process implementation rather than to the board — every board in the tree uses that one. What puts things in this queue is chapter 7.
One empty slot for every grant the board's capsules declared, counted once at boot and never counted again. Filling them is chapter 8, and this reservation is the reason that chapter can exist.
That is what isolation costs in memory. It costs something in time too, and the shape is the same — the charge falls on every single visit to a process.
Six operations wrap each one on this board. The kernel programs a piece of hardware to hold the process inside its own memory, switches it on, arms a timer, hands over the processor, disarms the timer, and switches that hardware off again. A board that wants a seventh has somewhere to put it: there is a hook immediately before those six, empty unless a board fills it.
An application is built without knowing where it will end up. Its RAM depends on how much the board has and what else is installed, and neither is known when it is compiled.
There are three ways out of that, and two of them are in this tree.
Guess which one this kernel does by default, then click it.
What a desktop operating system does, and what this kernel never does. Nothing in Tock walks an application's binary and edits it, and flash is not the kind of memory you could edit in place anyway.
Real, and rarely used. An application can state in its header exactly where it insists on being put. The kernel either finds it that spot or refuses to load it, with an error of its own: MemoryAddressMismatch.
The default, and the reason for the next figure. The kernel places the application wherever it likes and hands it four numbers on the way in, and the application works everything else out from those.
An application is compiled once and flashed to two identical boards. On the second board another application is already installed. Do the four numbers the kernel hands it at startup differ?
The binary is the same and the numbers are not. Not one of the four is a constant; they describe where this copy ended up, not what it is.
Right. Where its memory starts, how far it runs, where its break is and how big its region is are all worked out at load time. An application already in place moves every one of them.
Flash placement moves too — a second application is found by walking past the first — but so does RAM, and the four are handed over together.
Different memory would move them. So does anything else that got there first, which is the point: the same binary lands somewhere different on every board it meets.
Click an argument. These are registers 0 to 3 the first time the application ever runs.
Not the start of the header: the start of the code, past the header and past any bytes the kernel is keeping to itself. The first instruction is at another offset again, and that one goes into the program counter rather than a register.
The bottom of Figure 6. Every address the application works out for itself is worked out from this one, which is why it is handed over rather than assumed.
How much was allocated — and notice what that includes. The three structures in Figure 7 are inside this length. The application is told the size of a region whose top it may never touch.
The ceiling, and the number the application will immediately try to raise. On this processor it arrives thirty-two bytes above memory_start, which is one stack frame and nothing else.
Chapter 4 ended on a capsule that loops forever, and on the fact that nothing stops it. A process is the other case entirely: the kernel keeps a state for it, and can move it between them.
Drive it. Every button is a real call in process_standard.rs, and three of them refuse from some states — the refusals are the shape worth finding. Two states have no way out except through another state first.
last call none yet
result a process the kernel has just started is Running
Six states, and the two drawn with a dashed edge are the ones a process ends in rather than passes through. Getting out of either takes another call first, and one of the two will not take the obvious one.
Stopped carries the state it interrupted, which is why the readout has somewhere to put it. Resuming does not guess: it puts the process back exactly where it was, so a process stopped while waiting goes back to waiting rather than to running.
Faulted, and this is the one worth pushing at. Try to restart it. A faulted process cannot be made runnable directly — try_restart returns unless the state is already Terminated, so the fault has to be cleaned up by terminating first.
Terminated, and a process here can be run again from the beginning. That is the difference between the two ending states: one is a dead end until something clears it, and the other is a starting line.
That call was refused, and nothing happened. The kernel does not fault a process for asking at the wrong moment; the method checks the state, returns, and leaves the process where it was.
Faulted is a state, not an outcome. What actually happens to the process next is a decision the board makes, in one line, and boards do not agree.
Pick a board. One application faults on each; the other applications are innocent.
The whole board panics. Not the process — the machine. The handler chapter 4 ended on takes over, every other application stops with it, and the LED starts blinking its message out on GPIO 25.
On imix that one process stops and is never scheduled again. Everything else carries on, and nothing is printed. Seven of these policies exist in the tree, and choosing between them is one line of the board's main.rs.
Every boundary in this chapter is a number in a struct. app_break is a field. memory_len is a field. The kernel can read them, compare them and print them.
An application that stores one byte past app_break consults none of it. Chapter 1's store instruction does not ask the kernel's permission, and there is no line of Rust anywhere that could make it.
Click each one. Two of the three are held up by nothing this chapter has shown you.
The checksum catches a header damaged on the way in, and that is the whole of what it catches. total_size is believed outright, before anything has been verified. Anyone who can write this flash can write a header saying whatever they like, and Figure 3's walk will follow it.
A field in a struct. Chapter 1's store instruction does not read fields, and no arrangement of Rust could make it. What refuses the write is a piece of hardware holding the same number, and putting the number there is chapter 6.
The one boundary in this chapter that anything already enforces. The processor's own counter, clocked at 125 MHz on this board, is armed before the switch and disarmed after it. When it fires, control returns to the kernel whether the application cooperates or not.
So this chapter has described a fence and not built one. The thing that builds it is the memory protection unit: a small piece of hardware inside the processor, reprogrammed before every switch into a process. Chapter 6 is about it.
The first four bytes of an application read 02 00 60 00. What has the kernel learned?
The header. Bytes 0 and 1 are the version, lowest byte first, so 02 00 is 2. Bytes 2 and 3 are header_size the same way, so 60 00 is 0x0060, which is 96. The length of the whole thing is in the next four bytes, which have not been read yet.
Why is a process told the address of its own memory as it starts?
Because it was placed rather than adapted. Figure 8's first option is the one nobody took. No code in this kernel edits an application's binary, so it arrives still holding whatever addresses it was built with, and has to be told the rest.
A process is given two kilobytes of RAM. How much may it touch at its first instruction?
Thirty-two. app_break starts one exception frame above the bottom, which is the least that lets the kernel switch into the process at all. Raising it is a syscall, and it is one of the first things any application does.
A process on this board stores one byte past app_break. What happens?
On this board, the second. All three are real policies and four more exist beside them; which one applies is one line near the top of the board's main.rs. The Pico 2 names PanicFaultPolicy, so one misbehaving application takes the machine down with it.
A process is a binary this tree never compiled. It is found by walking flash from one linker symbol, and sized by sixteen bytes at its front. It is given a run of RAM with the kernel's own record of it hidden at the top, and told in four numbers where it landed.
Every edge in that description is a field the kernel holds. Not one of them stops an instruction. What does is a piece of hardware nobody has mentioned yet.
Next
The fence this chapter described and did not build. Eight regions, two registers apiece, and a size rule that rounds every allocation up to a multiple of 32 bytes. On the way: what happens in the instant a process touches the wrong address, and why the exception that runs is not the one the chip's own documentation names.
Every claim on this page comes from the Tock tree at commit 83bad9388, the same commit chapter 4 was pinned to. Nothing here was written from memory.
doc/reference/trd104-syscalls.md:24–28libraries/tock-tbf/src/types.rs:114–120libraries/tock-tbf/src/parse.rs:36–56total_size — libraries/tock-tbf/src/parse.rs:44–47flags — libraries/tock-tbf/src/types.rs:698–702. The comment there calls it "bit 1"; the code tests flags & 0x00000001, which is bit 0libraries/tock-tbf/src/parse.rs:81–95, where the loop body for i == 3 is a comment and nothing elsetotal_size forward, then again — kernel/src/process_loading.rs:295–354kernel/src/process_loading.rs:329–333_sapps, _eapps, _sappmem and _eappmem, and handing all four to the loader — boards/raspberry_pi_pico_2/src/main.rs:108–127, "End of the ROM region containing app images"boards/raspberry_pi_pico_2/layout.ld:14–17_sapps at the start of the prog region and _eapps one length later — boards/build_scripts/tock_kernel_layout.ld:304–324.apps section marked NOLOAD, so applications are flashed separately from the kernel — the same file, :293–3040xFF placeholder bytes the linker writes there — the same file, :318–321_sappmem beginning after the kernel's zeroed data, and _eappmem at the last byte of RAM — the same file, :350–366memory_start, kernel/src/process_standard.rs:449–473kernel/src/process_standard.rs:1824–1829, drawn again at :1995–2015kernel/src/process_standard.rs:1737, "Number of upcalls stored in the upcall ring buffer (10 element length)"size_of::<ProcessStandard>() plus its alignment — :1753 and :1755app_break starting initial_process_app_brk_size() above memory_start — :1850–1852 and :2092arch/cortex-m/src/syscall.rs:196 and :273–277kernel/src/process_standard.rs:2054–2075kernel/src/process_standard.rs:2303–2343memory_len being the whole allocation, kernel structures included — :2055 and :2225arch/cortex-m/src/syscall.rs:404 and :410–413, in set_process_function at :376libraries/tock-tbf/src/types.rs:204–213 and kernel/src/process_loading.rs:54–58kernel/src/process.rs:963–997, "Process stopped executing and returned to the kernel because it called the `yield` syscall"kernel/src/process.rs:959–962 and :989–992capsules/system/src/process_policies.rs, at :16, :25, :36, :49, :58, :75 and :100PanicFaultPolicy and imix choosing StopFaultPolicy — boards/raspberry_pi_pico_2/src/main.rs:32 and boards/imix/src/main.rs:95boards/raspberry_pi_pico_2/src/lib.rs:63, "const NUM_PROCS: usize = 4"kernel/src/process_loading.rs:203–250, which carries remaining_memory from one to the nextProcessStandard being one implementation of a trait, and what the trait asks for — "a generic process that the Tock scheduler can schedule", kernel/src/process.rs:331–333doc/Getting_Started.md:246–247kernel/src/platform/platform.rs:189 and :193–194.tbf file — boards/raspberry_pi_pico_2/README.md:44–51objcopy lines that put one in the reserved section, and the program target that runs them — boards/raspberry_pi_pico_2/Makefile:33–39make flash-app, which this board's Makefile does not define — boards/raspberry_pi_pico_2/README.md:50 against Makefile:33. Several other boards do define it; boards/apollo3/redboard_artemis_nano/Makefile:36 is oneprogram copies to, and why it does nothing on a Mac — BOOTSEL_FOLDER?=/run/media/$(USER)/RP2350 at boards/raspberry_pi_pico_2/Makefile:17, tested at :34 and :44, which print a message when the directory is absent. That board's README gives the default as /media/$(USER)/RP2350 at :41, which is a third path againprogram-openocd running the same two objcopy lines and pushing the result down a debug probe instead — Makefile:47–53, with OPENOCD_INTERFACE=picoprobe at :14 and the configuration it names at openocd-picoprobe.cfgcapsules/extra/src/app_flash_driver.rs:5–8 and :105, reaching it through ProcessId::get_editable_flash_range at kernel/src/process.rs:208kernel/src/kernel.rs:536–562, where the fence is configured and enabled, the timer armed and disarmed either side of switch_to, and the fence disabled againboards/raspberry_pi_pico_2/src/lib.rs:404, "systick: cortexm33::systick::SysTick::new_with_calibration(125_000_000)"Text, diagrams and interactive figures © Jon Hillesheim 2026, licensed CC BY-SA 4.0 — share and adapt freely with credit, under the same license. Tock source excerpts quoted above remain under their own Apache-2.0 OR MIT license and are not relicensed here.
Every line reference below links to that commit on the fork it was read from, at the lines it names.