Define the failure and preserve the evidence
Record the board and fitted-part revisions, firmware artifact and build settings, power setup, workload, expected result, and exact failure. Include reproduction steps and how often the fault occurs. If the board has never worked, start with Rapid bring-up.Prepare the investigation
- Select the intended project with
/projectand confirm the active platform and peripherals. Provide the exact datasheet, errata, and schematic revision for the populated board. - Check the build, flash, and observation commands in EMBEDDER.md. Identify the actual board, probe, port, or selected hardware bridge.
- In VS Code, run Embedder: Open Embedder Monitor. Use its logs and capture views beside the source; the terminal also supports investigations and saved results.
- Ask Embedder to check the needed instruments and complete their setup. Follow the tool-specific guide for channels, voltage limits, trigger support, and host dependencies.
- Run
/debugfor debugger state inspection or bounded J-Trace instruction/coverage captures. Ordinary instrument measurements and hardware scripts can also run in Act mode.
.embedder/hardware/cases/sensor-timeout/CASE.md. Keep the symptom, reproduction, candidate causes, observations, and next test there, with paths to raw captures. Update it after each test instead of treating the first explanation as settled.
Choose a test that separates the causes
A missing response can result from firmware, a physical connection, the device’s operating state, or the observation setup itself. First confirm the port, probe, artifact, decoder settings, and capture window. A missing or malformed capture is an incomplete observation.- Source, datasheets, and schematics: check intended pin mapping, clocks, reset order, address selection, register semantics, and driver behavior. A schematic describes the design; it does not prove that the assembled board matches it.
- Serial logs and GDB: inspect startup order, error paths, stack, variables, and available task information. Debugger and firmware support determine which state is available.
- Logic capture: check actual digital transactions, address, acknowledgments, framing, and edge timing. Decoded traffic alone does not establish signal-voltage margins.
- Scope and power measurements: check rail voltage, reset or enable signals, rise time, ringing, current, and workload transitions. Use a scope for short voltage transients; averaged power statistics can hide them.
Work through hardware and software hypotheses
A peripheral does not respond
A peripheral does not respond
Ask Embedder to compare the driver’s initialization and transaction sequence with the exact device datasheet and schematic. Then capture the bus using a supported logic analyzer.If the expected pins never toggle, check whether the code reaches the transfer, the peripheral clock and pin configuration, and the routed connection. If the address or command is wrong on the wire, trace how the driver constructs it. If the transaction is as intended but the device does not acknowledge, check supply, reset/enable state, address straps, and signal levels before blaming the parser.Use an oscilloscope to measure the relevant signal at the device and compare rise time and levels with its requirements. Ask for a physical check when the suspected problem is a missing pull-up, open connection, or incorrect fitted part. Specify the board’s limits before anyone changes wiring or components.Correct a demonstrated address, pin, or sequence error in firmware. For a demonstrated electrical problem, document the board correction and remeasure afterward. Reducing bus speed may improve a marginal connection, but that result alone does not establish the original cause or validate the required bus speed.
The board resets when a load starts
The board resets when a load starts
Use a scope to capture VDD at the target and a known workload marker in the same acquisition. Choose ranges and a trigger from the board’s supply and reset requirements, and arm before reproducing the load change.Register lookup supplies definitions when available; a live value requires a separate debugger observation. Check access rules before reading: some status clears on read, and startup code may already have cleared reset flags. Avoid treating a reference reset value as a measured value.A captured voltage drop together with relevant reset flags supports a supply-related hypothesis; determine whether wiring, supply capacity, decoupling, or firmware load sequencing caused it. Stable voltage at the measured point does not rule out an unmeasured transient elsewhere. A watchdog flag identifies the reset mechanism, but the missed service can still result from a driver deadlock, long interrupt, or stalled peripheral.Ask Embedder to inspect the relevant scheduling and driver paths or propose the next electrical measurement. Apply a firmware sequencing fix or have the identified board issue corrected, then repeat the original load transition under the same supply conditions.
A hang or timing failure changes under the debugger
A hang or timing failure changes under the debugger
Preserve the matching artifact, captured output, and any existing stopped state. Use GDB to inspect the current frame, stack, variables, and available task state, then compare the result with locks, interrupt masking, DMA completion, and bounded waits in the source.Stepping and halting change timing. If the failure disappears, use a bounded run with minimal logging or a measured GPIO marker, and check the external transaction while firmware runs normally. Inspect code and host tests for a lost wake-up or unbounded wait; inspect the bus and peripheral state for the missing completion event.Use trace when execution order can answer the question. Compatible firmware instrumentation is required for event traces. Bounded J-Trace captures reset and run the target; streamed ETM repeatedly halts it. Preserve the original fault first and treat a later trace as a separate run, not proof of uninterrupted real-time behavior.Correct the demonstrated synchronization or error-handling defect and add a regression. A timeout can prevent an indefinite hang while leaving the reason for the missing device response unresolved; record that distinction.
Apply a correction and test the explanation
Ask Embedder for a focused correction tied to the evidence. For a larger change, use/plan to review affected files and tests, then /act to implement it. Run the relevant software checks and target build before flashing the confirmed board.
For a hardware correction, have someone inspect or modify the identified connection or component, then record what changed. Embedder can coordinate supported instrument settings and measurements, but a proposed rework is not a completed repair.
Change one factor at a time where possible. A comparison with a known-good firmware build on the same board, or the same build on another matching board, can help isolate the problem. Keep wiring, supply, workload, and measurement settings fixed, and record any differences that prevent a clean comparison.
Close the RCA with a regression
Ask Embedder to finish the investigation record with:- The observed failure, affected board/build, trigger, and reproduction frequency.
- The causal sequence, supporting source/document references and captures, and alternatives tested or still open.
- The firmware or hardware correction actually applied, with before/after results and measurement limits.
- A repeatable test, pass criteria, and any remaining mitigation or follow-up work.

