A pending interrupt survives the reset that silenced it

1What was measured

The panic arrives before the main loop, on a board that had booted fine minutes earlier:

panicked at chips/rp2350/src/chip.rs:78:17:
unhandled interrupt 14

chips/rp2350/src/chip.rs The line is deliberate. service_pending_interrupts walks the pending NVIC lines and refuses to continue past one nothing claims:

while let Some(interrupt) = cortexm33::nvic::next_pending_with_mask(mask) {
    if !self.interrupt_service.service_interrupt(interrupt) {
        panic!("unhandled interrupt {}", interrupt);
    }

That is not an rp2350 quirk. Twelve of the twenty-three chip crates in the tree panic the same way, and the same line is what surfaced an unrouted IO_IRQ_BANK0 on this chip earlier. It is doing its job: it found something.

IRQ 14 is USBCTRL_IRQ. What makes it fatal here rather than merely noisy is an absence — chips/rp2350/src/ has no usb.rs at all. The RP2040 crate has one, and chips/rp2040/src/chip.rs services USBCTRL_IRQ, so the same stale bit would be handled there rather than fatal — read from the source, not run; see Section 8. On the RP2350 there is nothing to route the line to.

2The pending bit is a latch, not an interrupt

The board was halted over SWD while it sat in its own panic handler, and the interrupt controller and the USB block were read out before anything reset them. Two numbers decide the whole question:

NVIC ISPR

bit 14 set

USB MAIN_CTRL

00000000

USB INTR (raw)

00000000

USB INTE (enable)

00000000

USB INTS (masked)

00000000

NVIC ISER

all zero

The controller is asserting nothing. It is not enabled, no raw interrupt condition is set, and with INTE at zero its output to the interrupt controller cannot be asserted at all. The whole 256-byte block is byte-for-byte identical to the same registers read on a boot that did not panic.

Yet the pending bit is set. A pending bit that is set while its source is quiet was latched earlier and never cleared — the NVIC records that a line was asserted, and nothing about the source going away afterwards takes that record back.

This is the measurement that separates the two candidate stories. A live interrupt would show the controller asserting, and the fix would be to service or mask it. A stale latch shows exactly what was read here, and the fix is to clear it once the source is genuinely gone.

For comparison, the same three dumps on a clean boot of the same image: ISER has three lines enabled — TIMER0_IRQ_0, SIO_IRQ_FIFO, UART0_IRQ — and ISPR is empty.

3The four lines of boot that leave it behind

boards/raspberry_pi_pico_2/src/main.rs The order is the whole defect, and every line in it is reasonable on its own:

ChipHw::init();                    // disable_all() + clear_all_pending()
...
resets.reset_all_except(&[ ... ]);  // USBCTRL is reset here
init_clocks(peripherals, clocks, resets);
resets.unreset_all_except(&[], true);

The first step is measured rather than assumed. Halting the chip while the bootrom was enumerated as 2e8a:000f and reading the same two blocks:

NVIC ISER

bit 14 set

USB MAIN_CTRL

00000001

USB ADDR_ENDP

00000024

USB SOF_RD

00000398

USB INTE

000113f0

USB INTR

00100008

The controller is enabled, a host has given it address 0x24, frames are arriving, its interrupt enables are set, and the bootrom has enabled USBCTRL_IRQ in the interrupt controller itself. The original rate of two in nine needs no mechanism to explain it, only the condition: both sightings were resets taken out of a live session and both panicked, while the rest of that day’s resets either had no bootrom involved or had let the board self-boot out of it first, ending the session. The rate was low because the condition was rare.

1. The bootrom leaves the USB controller enabled, addressed and permitted to interrupt, because it was in the middle of talking to a host.
2. Chip::init() runs disable_all() and clear_all_pending(). That clear cannot stick. The line is still asserted, so the NVIC re-latches the bit immediately; a level-sensitive source cannot be cleared out from under. Measured rather than argued: halt the core at the reset vector and the bit is already set before the kernel runs at all — Section 4.
3. reset_all_except resets the USB controller and the source goes quiet — leaving behind a pending bit that is now unreachable by any of the mechanisms that would normally clear it.
4. Nothing on this board claims IRQ 14, so the first pass through service_pending_interrupts panics.

disable_all() does not save it either. next_pending_with_mask reads the pending register, not the enable register, so an interrupt that is disabled and pending is still found and still panics.

4Reproducing it with nobody in the room

Both original sightings followed a bootrom USB session, and reaching one meant holding the BOOTSEL button — which makes a rate impossible to measure and a fix impossible to test. The software route to the same condition turns out to be one openocd command:

1. Erase the first flash sector. The RP2350's IMAGE_DEF block goes with it, so the bootrom finds no valid image and falls back to its own USB device.
2. Wait for 2e8a:000f to enumerate on the host. Now a real bootrom USB session is live, with no button pressed.
3. Reprogram and reset from the debugger while it is still live. That is the condition the second sighting described.

The panic went from twice in nine resets, cause unknown, to thirteen in thirteen on demand — every run of this variant on an unpatched kernel, with openocd ending the session.

Both debuggers produce it, and not at the same rate. Ending the session with probe-rs instead panicked once in six. The reading that reset method was not a variable at all had been resting on a single occurrence per tool, and this kills it: something about the two paths differs, and the rates say so.

The obvious candidate was session length — the probe-rs path takes 96 seconds an iteration against 18, so its reset lands far later, and a transient assertion would have faded by then. That is wrong. Holding the bootrom session open for a further 78 seconds before letting openocd reprogram and reset brings its iteration to 96—97 seconds, against probe-rs's 96:

Same tool and same commands at both durations, and the rate does not move. Length is not the variable; the tool is. That also disposes of the idea that the assertion fades — over a session more than five times longer, it does not.

Each tool does two things to a live bootrom session, programs and then resets, so the halves swap independently — probe-rs download does not reset unless asked:

Whoever resets decides it; the programming does not matter. That also kills the next guess in line, that probe-rs holding the core halted through an eighteen-second download was what let the line go quiet.

What an openocd reset leaves, read before a single instruction runs

reset halt holds the core at the reset vector, so this is the state the kernel is handed rather than a state it produced. Beside the same registers taken while the bootrom was still running:

                 MAIN_CTRL  INTE      INTS      NVIC ISPR bit 14
bootrom running  00000001   000113f0  00000000  0
after the reset  00000001   000113f0  00000010  1

The reset does not touch the USB controller. MAIN_CTRL and INTE come through it unchanged — still enabled, still permitted to interrupt. That is the same register Section 2 reads as zero: this is the board before the kernel runs, and that one is after the kernel has reset the controller.

What changes is INTS, the interrupt state after masking. It is zero while the bootrom runs, so nothing is pending then. After the reset an enabled condition is asserted, nothing is left to service it, and the NVIC latches IRQ 14 — before the kernel executes anything at all. Why that condition appears across the reset is not established here; what is measured is that it does, and that the controller is in a position to raise it because the reset left it running.

openocd's own target file says why, at scripts/target/rp2350.cfg:140:

# srst does not exist; use SYSRESETREQ to perform a soft reset
cortex_m reset_config sysresetreq

There is no hardware reset line to drive, so the reset is whatever SYSRESETREQ covers on this part — and the dump says the USB controller is not in it. rp2040.cfg is configured the same way. As the next section shows, probe-rs does not reset it either.

probe-rs leaves it pending too, which was not the guess

probe-rs gdb --reset-halt resets and then holds the core, giving the same instant on the other tool. The obvious guess — that probe-rs's reset leaves the line quiet, and that is why it rarely panics — is wrong:

                  MAIN_CTRL  INTE      INTS      NVIC ISPR bit 14
openocd  reset    00000001   000113f0  00000010  1
probe-rs reset    00000001   000113f0  00010000  1

Both leave the controller enabled, both leave an enabled condition asserted, and both leave IRQ 14 pending at the reset vector. What differs is which condition: bit 4 against bit 16 — BUFF_STATUS and SETUP_REQ if the RP2040 field names carry over, which is worth checking against the RP2350 datasheet before anyone leans on it.

So the difference is not in what the reset leaves. Being pending at the vector is not the same as surviving Chip::init()'s clear a few microseconds later — that needs the condition to still be asserted then. After an openocd reset it evidently is, every time. After a probe-rs reset it usually is not, and why has not been established.

That the probe-rs row is the right one to compare was checked in probe-rs's own source at the tagged version, since gdb --reset-halt is not the command the rates were measured with. probe-rs reset calls core.reset(); --reset-halt calls core.reset_and_halt(). In the Armv8-M core both run the identical line:

self.sequence.reset_system(&mut *self.memory, crate::CoreType::Armv8m, None)?;

Same function, same arguments, so whatever that does for this chip, both paths do it. reset_and_halt wraps more around it — a DEMCR vector-catch set before, described in the code's own comment as “this will halt the core after reset”, then a wait for the halt, an XPSR Thumb-bit fixup, a catch clear and a read of the program counter. Every one of those is a debug or core-register operation, and every one happens either before the reset or after the core has already stopped. The stimulus itself is the same by construction.

What both rows do share as a limit: the core is halted while the USB controller keeps running, so these are the registers shortly after each reset rather than frozen at the instant of it. A behavioural check — resume from the vector and see whether the panic rate matches a plain reset — was attempted and came back inconclusive, the console staying silent on four iterations of six.

None of this changes the defect, which is that the kernel panics on a pending bit nobody owns however it got there.

5Four theories, and one of them was right all along

Four mechanisms were proposed and all four written down as eliminated. Three of them genuinely were:

Conventional and causal are independent. “Is this unusual?” and “did this cause the failure?” are different questions, and only the second one eliminates anything. A pattern every board in the family shares is still the cause when exactly one board in the family has no driver for the line it strands.

6The fix, and what measures it

One statement, after the peripherals have been reset and while nothing is asserting:

resets.unreset_all_except(&[], true);

cortexm33::nvic::clear_all_pending();

Measured on a Pico 2 W over a Debug Probe, running the upstream raspberry_pi_pico_2 board crate, with the bootrom reproduction above as the trigger:

Run as A/B/A rather than A/B, so an ordering effect or a bench that had simply changed under us would have shown up as a clean third arm. Under the fix both installed applications ran to completion with the console answering throughout, so it is not buying quiet by breaking something else.

One detail worth keeping, because it nearly hid the difference: the two kernels have identical .text, 69,164 bytes each. The size summary would have called them the same binary. Only cmp and the hashes distinguish them.

7Where the line belongs is a real question

Three placements are defensible and only one of them has been measured.

8What this does not establish

• The hardware is a Pico 2 W running the upstream raspberry_pi_pico_2 board crate. There is no plain Pico 2 here. The two differ in what is wired to the pins, not in the boot sequence or the chip, but they are not the same board.
• No RP2040 board was involved. The four RP2040 boards share the same ordering, and the same stale bit will reach them — but USBCTRL_IRQ is serviced there, so it is handled instead of fatal. That reasoning is from the source; nothing was run.
• USB is the only source seen doing this. The mechanism is general — any peripheral asserting at the moment Chip::init runs would latch the same way — but the bootrom is the only thing observed arming one, and it arms USB.
• The rate outside the bootrom condition is not known to be zero. No reset the harness has logged without a bootrom step in it has ever panicked — twenty-five of them by the time this was written, and the logs are the count rather than this sentence. That bounds it low; it does not show it is zero.