Learning Tock from the ground up · Chapter 8
‹ ContentsChapter 7 handed the kernel two things and said it files them somewhere. Somewhere turns out to be inside the process's own memory, above a line that process can never reach. This is the last mechanism in the series, and the kernel running it cannot ask for memory.
Plan on forty-five minutes. Chapter 5 drew the memory this chapter fills, chapter 7 left the two things it files, and chapter 4 asked the question it answers. If you have read those three, nothing here is new vocabulary except the mechanism itself.
In the book: doc/syscalls and development/syscall, which say what the mechanism is for. This chapter counts the bytes.
Each is one sentence now and repeated in context below. Chapter 5 defined a grant in one line and pointed here; the first entry is that line with the rest of the sentence attached.
Chapter 4 ended on a sentence that has been waiting four chapters for an answer. The kernel is no_std and never asks for memory it did not already have. There is no allocator anywhere in it, and Figure 1 is about why adding one would not be the simple fix it sounds like. Nothing calls malloc, nothing calls Box::new, and there is no heap for either to work from.
That is easy to live with until a driver needs to remember something about a process. The console has to know how many bytes of a write are still outstanding. It cannot keep one copy, because two processes can be printing at once, and it cannot keep an array of copies without knowing how many processes there will be.
There are four ways out of that, and this kernel has ruled out three of them.
Move the sliders and read down the bytes column. One design's cost does not move at all, one cannot be built here, and the one this kernel takes is not the cheapest — find what it buys instead.
| the design | bytes | out of | serves |
|---|---|---|---|
| an array sized for the worst case | 64 | the kernel's own RAM | all of them |
| a heap | — | there is no allocator | — |
| one copy in the driver | 16 | the kernel's own RAM | the first caller |
| take it from the process | 108 | each process's own memory | all of them |
The one this kernel takes is costing more bytes than the array right now, and that is the ordinary case rather than a corner of it. What it buys is in the other two columns: the bytes come out of the processes rather than the kernel, and a process that never calls the driver pays only its eight-byte table slot.
Here it is cheaper outright, because most of the room the board makes is going unused. The array pays for every process the board could ever run; the grant pays for the ones that turned up and called.
Nobody is using the driver, and the two on the kernel's side are still paying full price. That is the accounting the fourth design changes: with no callers it costs the table slots and nothing else.
Past the first caller the third design stops being a design. It does not cost more, it refuses — and a driver that serves one process is not a driver for a board that runs several.
Chapter 5 drew a process's memory as seven boundaries and said one of them starts at the very top and moves downward as grants are handed out. This is that movement. Everything above kernel_memory_break belongs to the kernel; everything below app_break belongs to the process; the gap between them is what neither has claimed yet.
The process cannot read any of it. Chapter 6's region for a process's RAM stops at app_break, so the grant region is outside the fence — inside the process's allocation and outside its reach. The kernel is on the privileged side, where those regions do not apply, so it can read both.
Click each band to light it above. Two of the six are lines, not regions — find which way each of those can move.
Cut first, before the process runs a single instruction, and never resized. One entry per grant number the board declared, each holding a driver number and a pointer, eight bytes on this chip. Every process carries the whole table whether it uses any of those drivers or not, and every entry starts null.
Cut one at a time by a bump downward, the first time a driver is entered for this process. Each is the size that driver needs, rounded to its alignment. Each is a bump and only ever downward, so the order of the grants below the table is the order in which this process first used those drivers.
The bottom of the grant region, and of everything the kernel has taken. Every new grant lowers it by that grant's size, and nothing raises it while the process runs. Chapter 5 named it; this chapter is what moves it.
Unclaimed. The process can grow into it from below by moving its break, and the kernel can grow into it from above by cutting a grant, and whichever asks first gets it. When it is gone, both requests start failing.
The top of what the process can touch, and the far end of the region chapter 6 built. A process asking for more heap is asking to move this line up, and the kernel refuses if it would pass the mark chapter 7 left or reach what the kernel has taken.
Everything chapter 5 walked through: the process's stack, its data, its heap, and at the bottom the thirty-two bytes the first context switch needs. All of it readable and writable by the process, none of it readable by another one.
The console is a good one to price, because chapter 7 used it and because its per-process state is four fields. It declares how much of everything it wants in its own type, which is where the arithmetic starts:
apps: Grant<
App, // four fields, 16 bytes
UpcallCount<{ upcall::COUNT }>, // 3
AllowRoCount<{ ro_allow::COUNT }>, // 2
AllowRwCount<{ rw_allow::COUNT }>, // 2
>,A capsule that needs more than its fixed size can cut extra memory from the same region, which is rare and comes out of the same place as everything else here.
Those three counts are constants in the type, not values in a variable. How many function pointers the console can hold for one process is fixed when the kernel is compiled, and a capsule that wants one more has to be recompiled to get it.
The console's grant costs one process 76 bytes. How much of that is the console's own state — the fields the driver actually wrote?
A grant is the driver's state plus whatever the kernel needs to find it again. The driver's own struct is the smaller half by a long way.
That is the other way round. Sixty is what the kernel's own bookkeeping takes, and it is the part no driver author ever writes.
Right, and the ratio is the point. Four fields: three words and a flag. The other sixty bytes are a counters word plus seven slots at eight bytes each, held open whether the process ever uses them or not.
Nothing here is split evenly. The slot count comes from the driver's own declaration and the state size from its struct, and the two have no relation.
Set the slot counts and the driver's own state, and the total is what one process pays that driver. Start from the console's own numbers, then find the setting where the padding row stops being zero — there is exactly one thing that turns it on.
Every slot is eight bytes, whichever kind it is. An upcall slot holds a function pointer and the word the process passed at subscribe time that chapter 7 traced all the way back to the callback. An allow slot holds an address and a length, which is exactly what the process handed over. Read-only is for the buffer a write comes out of, which chapter 7 said may sit in flash. Read-write is for the one a read is delivered into.
| the part | bytes |
|---|---|
| the counters word | 4 |
| the slots, at eight bytes each | 56 |
| padding, to reach the state's alignment | 0 |
| the driver's own state | 16 |
| what one process pays | 76 |
These are the console's own numbers: three upcall slots, two read-only, two read-write, and sixteen bytes of its own state. Seventy-six bytes, and this is not a number added up on this page — it is what the kernel's own grant_size returns, compiled for this board.
Nothing is padded. The kernel's part is a counters word and some number of eight-byte slots, so it always lands on a multiple of four. A state wanting four-byte alignment is already where it needs to be. Slide every count and the padding row stays at zero.
Four bytes, wasted, and always exactly four. The kernel's part is four plus a multiple of eight, which is four past an eight-byte boundary no matter how many slots there are. So a driver whose state wants eight-byte alignment pays the same four bytes with no slots as with all of them.
No slots at all, which prices the slots on their own. What is left is the counters word and the driver's own state, and the counters word is there even when there is nothing to count.
The slots are now most of it. Every one is eight bytes whether the process ever uses it or not, and the count comes from the driver's own declaration rather than from anything the process asked for.
grant-sizes.rs beside this chapter and compiled for this board. The same driver with no slots at all would be twenty.Three of those seven slots are never touched. The console's upcall numbers are 1 and 2, its read-only buffer is number 1, and its read-write buffer is number 1. Arrays are indexed from zero, so declaring the highest number you use declares every number below it too, and slot zero of all three arrays sits in every process that has ever printed, empty.
The source says why, in the same comment three times over. Watch the quotation marks in it: the quoted number is what an application used to pass, and the bare one is what it passes today. They had to stay the same. "Before the allow syscall was handled by the kernel, console used allow number “1”, so to preserve compatibility we still use allow number 1 now."
Click each. Three slots paid for and never filled — check whether they cost the same, and why.
Eight bytes for a function pointer and a word of application data, in a slot the console never schedules. Its two upcalls are numbered 1 for a finished write and 2 for a finished read, and the count in its type is therefore 3. The comment beside that count says as much: even though we only use two, indexing starts at 0.
Eight bytes for an address and a length, in the slot before the one the console reads out of. The number kept was 1, from before the kernel handled the allow class at all, so that applications built against the old numbering would keep working.
Eight bytes again, for the read buffer's neighbour. Same reason, same comment, same cost, and every process that opens the console pays it for as long as it lives.
A driver holds its grant from the moment the board boots, and that object contains no memory at all. It is two numbers: which driver this is, and which index in every process's table belongs to it. Everything else happens later, per process, and only if that process asks.
A board installs twelve drivers. One process runs and calls exactly one of them. What do the other eleven cost that process?
Almost. There is one cost, and it is small and unavoidable. The table that lets the kernel find a grant has one entry per driver the board installed, and every process pays for all of them the moment it starts.
Right. The table at the top is eager and the grants under it are lazy. Eleven pointers, eight bytes apiece, all of them null — and the grant itself, the seventy-six bytes the console's costs, is not allocated until the first syscall that needs it.
That would be the eager design, and it is the one Tock deliberately does not use. A driver installed and never called allocates nothing.
There is no such deferral. What each process pays is decided at its own first call, one driver at a time.
Step through all five in order and watch the running total. Three of the five change nothing, and the fifth is worth comparing with the second.
No memory anywhere. The kernel hands out a grant number, counts up by one, and the driver keeps that number for the life of the board. Once the first process is created the count is finalised, and asking for another grant panics outright, because every process's table was sized from that count.
Eight bytes, in the table at the very top of the process's memory, holding a driver number of zero and a null pointer. Null is the flag: the kernel cannot use the driver number to tell whether a grant exists, because zero is a real driver number, so the pointer carries that meaning instead.
Now it costs something. The kernel works out the size from the driver's type, moves kernel_memory_break down by it, aligns it, checks it has not run into the process's own break, and writes the pointer into the table. This is the only moment in the five where the region gets smaller.
Free, every time after the first. The pointer is already in the table, so entering is a bounds check and a closure call. A driver that talks to a process a thousand times allocates once.
Everything goes back. Restarting recomputes the whole layout from the process's original allocation. The table is rebuilt, kernel_memory_break returns to the top, and every grant is gone. The mark chapter 7 said a running process could never lower comes down with them. It is the only refund in this chapter, and the price is that the process is no longer the one that was running.
A capsule never holds a pointer into a grant. What it holds is the two numbers, and to reach the bytes it has to ask, once, for as long as one closure runs. The kernel opens the grant, hands the closure two references, and closes it again when that closure returns. It closes it even on an early return, because closing is what happens when the kernel's own bookkeeping value is dropped.
Click each. Four steps from two numbers to a mutable reference; find where the process's memory first appears.
What the capsule was handed at boot, and what it keeps in a field for the life of the board. A driver number and a grant number, plus the types that carry the three slot counts. It refers to no memory and can be copied freely, which is why a driver can hold one without holding anything belonging to any process.
The same two numbers with one process attached. Producing one is where allocation can happen. If that process has no grant at this number yet, this is the step that cuts it, and the step that can fail for want of room. A capsule that wants to know whether the memory exists asks for this and looks at the answer.
Two references, valid only inside the closure. The first is a mutable reference to the driver's own state. The second is the kernel's half — the filed function pointers and the filed buffers — and a capsule can schedule an upcall or read a buffer through it but cannot take either out. Chapter 7 said the core kernel keeps both; this is the shape of keeping them.
The grant is marked not-entered again, by a value going out of scope rather than by anything the capsule remembers to call. That matters because a closure can return early, and an early return that skipped the closing would leave the grant locked for the rest of the board's life.
Entering a grant that is already entered does not fail politely. It panics, and the board stops. That is a deliberate choice made against the alternative, and the source explains itself at length where it happens.
The reason is that entering twice would produce two mutable references to the same bytes, which is the one thing Rust's rules exist to prevent. The kernel cannot hand out the second reference and stay honest, and it will not hand out something weaker, so it stops. What makes this a real risk rather than a theoretical one is a common shape in capsules. A loop walks every process's grant, and something inside the loop reaches for one particular process's grant.
Click each. Three refusals, and only one of them stops the board. Decide which before you read why.
Panic. Two mutable references to one object cannot both exist, so the kernel refuses to create the second and takes the board down rather than weaken the guarantee. There is a second way in, used by the loop over every process, which silently skips a grant that is already open instead of panicking. So a capsule that means to iterate can, and a capsule that reaches in twice by accident finds out at once.
An error, returned. Every grant a process holds carries the driver number it was cut for, and asking to allocate a second one for a driver number already in the table is refused before any memory moves. One driver, one grant, one process.
An error, returned, and the most ordinary of the three. The kernel works out where kernel_memory_break would land, sees it below the process's own break, and gives up without moving anything. The capsule finds out that this process cannot afford it, which is a different thing from the board being unable to afford it.
The process grows its heap upward by moving app_break. The kernel grows its grants downward by moving kernel_memory_break. Between them is memory neither has claimed, and it is the same memory: there is no separate pool for grants and no reserve held back for the heap.
Which of them gets a given byte is decided by which asks first, and neither is told about the other's appetite in advance. A process cannot know how much its drivers will want, and a driver cannot know how much heap its process will want.
Set the gap, then spend it. Press the two requests in one order, start over, and press them in the other order — the same two requests swap answers, and nothing about either one changed.
Nothing has been asked for yet. Everything between the two lines is free, and either of them can move into it: the process's break grows up, and grants grow down from the top of its memory.
Granted. The break moves up by thirty-two, the region chapter 6 builds is rewritten to match, and what is left is left. The process is told where its new break is and knows nothing about what is above it.
Granted, and the process was never asked. The console's grant is seventy-six bytes, taken off the top of the same gap the break grows into, on the first syscall this process made to the console.
Refused. There is not enough gap left for thirty-two bytes, so the kernel works out where the break would land, sees it above the grants, and moves nothing. The process gets a no-memory error back.
Also refused, and this is the one worth sitting with. The process asked for heap it could have had a moment earlier, and the thing that consumed it was a driver acting on the process's own earlier request. The failure is real and the process has no way to see the cause.
Refused. Seventy-six will not fit, so the kernel works out where the grant would land, sees it below the process's own break, and moves nothing. The console gets an error and the process never hears about it — the syscall that triggered it comes back with a no-memory error instead.
It already has one. A driver's grant is allocated once per process, on the first syscall that needs it, and every syscall after that finds it already there. Nothing more is taken.
Chapter 2 ended on one sentence, the one it says everything Tock does is an answer to: any code can write any address. Six chapters have been taking pieces out of that, and this is the last of them.
Click each chapter. The sentence is still true of the kernel and no longer true of a process.
Nothing taken away, because there is nothing yet to take it from. This is where the sentence gets its terms: an address names a block, an offset names a control inside it, and one bit names a pin. Until a store can be followed all the way to a wire, "any address" means nothing in particular.
The problem, stated. A store is a store, and a peripheral register is an address like any other. The processor running the instruction has no opinion about which address it lands on. Everything after this is somebody deciding which addresses a given piece of code may land on.
Nothing taken away yet, but the ground laid. The first instruction is never yours: the chip fetches from zero, inside a boot ROM nobody can change, and that ROM hunts through flash for something it recognises. Everything after it runs because something earlier chose to run it.
The first real cut, and it is not the hardware's. A capsule reaches only what its board handed it, and the crate it lives in will not compile the raw pointer that would reach anything else. Nothing stops a capsule writing a wild address while it runs. What stops it is that the code to do so was refused at build time.
A second kind of code, compiled separately, that the compiler behind the kernel never saw. The kernel cannot check what it does. What it can check is sixteen bytes at the front, which it decides to trust so that it can find where one application ends and the next begins.
The cut that is hardware. Two pairs of registers, checked on every access a process makes. The sentence stops being true of a process: it may write its own memory and nothing else, and no wrong pointer changes that.
The one way back out, made so narrow that no pointer can find it. Not an address the process may write, but an instruction that stops it, and eight kinds of question it may ask on the way through.
And the last piece: the kernel's own answer to needing memory it never asked for. It takes it from whoever caused the need, in the process's own allocation, above the fence, and never gives it back until that process is not that process any more.
A board has twelve drivers that use grants. A process starts and never makes a single syscall. How much of its memory has the kernel taken?
The second. The table is eager and the grants are lazy. Twelve entries of eight bytes each are cut from the top before the process runs, every one of them holding a null pointer. The first answer is right about the grants and forgets what makes them possible — the kernel needs a fixed place to record that a grant does not exist yet.
A capsule enters a grant, and inside that closure enters the same grant again. What happens?
The first. Two mutable references to the same bytes cannot both exist, and the kernel would rather stop than hand out something weaker. The third answer is the one to be careful with: there is nothing to wait on, because the first closure is on the stack below the second and will not return until it does.
A process calls exit-restart. What becomes of the seventy-six bytes its console grant was using?
The third. Restarting recomputes the layout from the original allocation, so the break returns to the top and every grant with it. The first answer is true of a running process and only of a running process, which is the distinction worth keeping: nothing frees one grant, and one thing frees them all.
The gap between the two breaks is forty bytes, and a driver needs seventy-six for its grant. What does the process see?
The first. Running out of grant region is an ordinary failure with an ordinary answer, and it arrives back through the same four registers chapter 7 described. The second answer is the interesting mistake: a fault is what a malformed request gets, and this request was perfectly well formed — it just could not be afforded.
A driver keeps its state about a process inside that process's own memory, above the line the process can reach and below a table cut before the process started. The memory is taken the first time that driver is used by that process, by moving one pointer down, and nothing moves it back while the process lives. What it costs is a number you can add up: for the console on this board, seventy-six bytes, twenty-four of them slots kept empty so that an older numbering still works.
A capsule never holds any of it. It holds two numbers, and it borrows the bytes for the length of one closure, which is chapter 4's rule applied to memory that belongs to somebody else. Reaching for the same grant twice at once stops the board, because the alternative is two mutable references to one object and the kernel will not pretend otherwise.
And that is the last mechanism. Chapter 2 said that nothing stops a store from landing anywhere. The answer took eight chapters, and it is that nothing on the chip does, so the rest was built. A compiler that will not compile the reach. A header the kernel parses before it trusts anything. Two registers per region, checked on every access. One instruction wide enough for eight kinds of question. And this, the place where the kernel puts what it has been handed. Every one of them is in a file this series has quoted, and every one is a decision somebody wrote down.
The end
There is no chapter 9. What there is instead is a board on your desk, a debug probe, and a tree you can now read. Every claim on these eight pages names where it came from: a file and a line in this tree, or a section of the datasheet. All of them are still there at the commit these pages pin. The most useful next thing is not more reading. It is make flash-openocd in boards/raspberry_pi_pico_2, which pushes the kernel down the debug probe and needs no mounted drive at all. Chapter 5 is where the other route is, and why it does nothing on a Mac. Then break something on purpose, and watch which of these seven mechanisms catches it.
Every claim on this page comes from the Tock tree at commit 83bad9388, the same commit chapters 4 to 7 were pinned to. The byte counts were compiled rather than added up: grant-sizes.rs ships beside this page and the gate runs it.
kernel/src/lib.rs:93, #![no_std], with no allocator crate anywhere in its dependencieskernel/src/grant.rs:5–40, "Grants allow capsules to dynamically allocate memory from a process to hold state on the process's behalf"grant_size, the same file, :376–396: a counters word, then the three arrays, then padding to the driver type's alignment, then the typeMachineRegister and CapabilityPtr are both #[repr(transparent)] over one pointer, at kernel/src/utilities/machine_register.rs:29 and kernel/src/utilities/capability_ptr.rs:29grant-sizes.rs beside this page, which transcribes grant_size and is compiled for thumbv8m.main-none-eabi with the nightly rust-toolchain.toml pinscapsules/core/src/console.rs:104–109GrantPointerEntry, kernel/src/process_standard.rs:400–410, and null being the flag rather than the driver number, at :403–406get_grant_count_and_finalize, kernel/src/kernel.rs:273–276, with grants created at :246allocate_grant, kernel/src/process_standard.rs:1213–1275, calling the region allocator at :1256allocate_in_grant_region_internal, the same file, :2583–2622, "Ensures that the allocation is of `size` bytes and aligned to `align` bytes"app_break, "let align = cmp::max(align, 2)"kernel/src/grant.rs:1360–1400, "we panic!() to notify the developer that they tried to enter the same grant twice which is prohibited because it would result in two mutable references existing for the same memory"Drop, we leave the grant. This protects against calling grant.enter() without calling the corresponding grant.leave()"reset, kernel/src/process_standard.rs:2447–2463, reached from try_restart at :795 and from exit variant 1 at kernel/src/kernel.rs:1406–1408flash-openocd, boards/raspberry_pi_pico_2/Makefile:23–25, which needs no APP and no mounted drive; the drive route is flash at :27–30, tested against BOOTSEL_FOLDER at :13allocate_custom_grant, kernel/src/process_standard.rs:1278–1300Text, diagrams and interactive figures © Jon Hillesheim 2026, licensed CC BY-SA 4.0 — share and adapt freely with credit, under the same license. Tock source excerpts quoted above remain under their own Apache-2.0 OR MIT license and are not relicensed here, as does grant-sizes.rs beside this page, which is source rather than prose.
Every line reference below links to that commit on the fork it was read from, at the lines it names.