Automated testing

Use this workflow to turn a firmware requirement or reproduced defect into a repeatable check that runs at the appropriate layer: host, target, simulator, or bench. Embedder can write and run project tests, program a target, wait for runtime output, coordinate instruments, and preserve evidence.

Embedder orchestrates your project’s test commands and reusable hardware scripts. It does not add a separate HIL test framework, generate a complete test suite automatically, or schedule regressions by itself. Keep durable tests in the repository and connect them to your existing CI or automation system.

Choose the lowest useful test layer

Start with the least expensive check that can prove the requirement. Add real hardware only when the behavior depends on the target, peripheral, timing, power, or electrical interface.

Question	Test layer	Typical evidence
Does pure logic handle the input correctly?	Host unit test	Test runner result
Does the project compile and link for the target?	Firmware build	Exit status and expected artifact
Does the image boot and reach a state?	Flash plus serial or RTT	Programming verification and runtime marker
Did the expected bytes cross a bus?	Logic analyzer	Raw capture and decoded transactions
Did the signal meet an electrical requirement?	Oscilloscope	Triggered waveform and measurement
Did the target stay within a power budget?	PPK2 or Joulescope	Current, voltage, charge, or energy result
Which code ran on the target?	Trace or coverage	Stored session or coverage report
What state caused the failure?	GDB and a hardware script	Registers, memory, frames, and structured result

A passing host test does not prove that the hardware works. A successful build does not prove that the image boots. A verified flash does not prove that the required runtime state occurred.

Record the project test workflow

Run /init and verify the commands placed in EMBEDDER.md:

/init

The project guide should identify:

the focused and broad host test commands;
the target build command and expected artifact;
the intended board, flasher, probe, and serial transport;
any simulator, hardware script, or instrument prerequisites;
the observable result that constitutes a pass;
actions that require confirmation or physical setup.

Ask Embedder to inspect the existing tests before it invents another harness:

Find the smallest existing test target that covers packet decoding. Report its
framework, fixture pattern, command, and expected result. Do not edit files or
run the full suite yet.

Run host tests and static checks

Use Act mode for ordinary source changes, test commands, builds, and static analysis:

/act

Add a regression case for a truncated status frame in the existing parser test
suite. Run that test first, then the parser suite. If both pass, build the
recorded firmware target and report the artifact path.

For C and C++ projects, enabled language servers and CLI analyzers can add diagnostics after an edit. Use project_diagnostics when you need a broader scan:

Assess the first-party C and C++ scope, then run the default project diagnostics
if the analyzer configuration is valid. Separate code findings from setup gaps.

Static analysis is opt-in. A detected analyzer is not necessarily enabled, and an automated MISRA or CERT result is evidence rather than certification.

Add an on-target smoke check

An on-target smoke check should build one known image, program one identified target, and wait for one bounded result.

Build and identify the artifact

Run the recorded build command. Stop if the command fails or the expected ELF, HEX, or BIN artifact is absent.

Confirm the target and transport

Identify the board, probe or remote bridge, power state, UART or RTT endpoint, and expected baud rate. Do not guess when several targets are connected.

Flash with early output capture

Use the dedicated flash workflow so serial capture begins before the reset that follows programming.

Flash the verified debug artifact to the selected board. Monitor UART from
reset and stop when "self-test: pass" or "self-test: fail" appears. Report
the flasher result and runtime result separately.

Bound the wait

Supply a stop string and timeout. A missing marker is a failed or incomplete observation, not a reason to wait indefinitely.

Use serial history when the device may already have produced the result. Use live monitoring when the check must wait for new output. A Monitor tab being open is not evidence that the agent read the expected line.

Create reusable hardware checks

Switch to Debug mode when a check needs GDB, coordinated target control, trace, coverage, or a bench instrument:

/debug

Store hardware automation under:

.embedder/hardware/

Use a case directory when the check belongs to a longer investigation:

.embedder/hardware/cases/spi-wake-regression/
├── CASE.md
├── capture_wake.py
└── verify_registers.py

Ask Embedder to reuse or update an existing script rather than create numbered copies:

Create a reusable hardware check for the SPI wake regression. Arm the logic
capture before sending the wake command, decode SPI, and pass only when chip
select stays low through the final clock and the response status is 0x00.
Save the raw capture and return a structured result.

Run the file with hardware_script_run, not plain Python or a shell command. The managed runner:

injects helpers for GDB, serial, RTT, logic, scope, power, or BLE;
selects providers from the script and detected equipment;
routes commands to a paired hardware host when needed;
takes an exclusive hardware lease;
validates helper usage before execution;
publishes supported captures to the Monitor.

Every script should print a final one-line JSON object containing a boolean success and string summary. Add structured data and metadata when the next step needs measurements or artifact paths.

Coordinate the bench

Use one hardware script when operations must occur in a controlled order or share the same target lease.

Check	Supported path	Important boundary
UART or RTT assertion	Serial helpers and buffered history	Define a timeout and exact marker
Register or memory assertion	Managed GDB helpers	Halting or attaching can perturb the target
I2C, SPI, UART, CAN, or other digital traffic	Saleae, Digilent, or PicoScope MSO logic capture	Record the achieved sample rate
Rail, edge, or pulse behavior	Supported Siglent or PicoScope path	Configure and read back the trigger
Current or energy budget	PPK2 or Joulescope	Confirm wiring and supply mode first
Execution path or source coverage	SEGGER J-Trace	Requires ETM routing and the matching ELF

Separate instruments do not automatically share a clock. Use a firmware GPIO marker, a known sequence, or an instrument-native combined capture when results must be correlated.

Turn a failure into a regression check

Reproduce and preserve evidence

Capture the original symptom before changing code. Record the build, target, trigger, instrument configuration, and result in the case note.

State the pass condition

Convert the symptom into a boolean outcome with a bounded timeout or numeric tolerance.

Write the smallest durable check

Prefer a host test when it can reproduce the defect. Otherwise, create or update one hardware script and keep raw captures needed for review.

Demonstrate the check

When safe and practical, show that the check detects the original behavior before applying the fix. If that would damage hardware or require destructive rollback, document why the failing run was not repeated.

Apply the fix and rerun unchanged

Run the same command, firmware workload, script, trigger, and pass condition. Report both the software result and the physical observation.

Run long or headless work deliberately

Set run_in_background=true on hardware_script_run for a soak, endurance test, or long capture. The script keeps the board leased until it exits or you stop its task. Only one background hardware script can run in a session, and a 24-hour backstop prevents an abandoned process from holding the board indefinitely. For repository-triggered work, the Embedder daemon can run a GitHub or Slack task on a machine that has the required toolchain or physical hardware. Parallel workers still serialize access to the same board through the hardware arbiter.

An active goal can continue Act-mode software work with automatic approval, but it is not a replacement for a test scheduler. Bounded loops can observe buffered or saved hardware telemetry; they cannot flash, start live captures, run GDB, or send commands.

Preserve reviewable evidence

Keep enough provenance to reproduce each result:

exact test and build commands;
source revision and build configuration;
artifact and matching ELF path;
board, MCU, probe, instrument, and firmware state;
requested and achieved sample rates;
reset, program, halt, and attach actions;
raw captures plus decoded or summarized output;
script result and CASE.md hypothesis verdicts;
trace continuity, dropped records, and symbol-match status.

Instrument captures persist under .embedder/captures/, coverage reports appear in the Coverage view, and trace sessions persist under .embedder/traces/. Hardware cases and scripts remain under .embedder/hardware/.

Hardware scripts

Write structured checks, hold hardware leases, and preserve case evidence.

Combined workflows

Coordinate serial, debugger state, trace, and instruments.

Coverage and trace

Measure executed source and inspect RTT, ITM, or ETM activity.

Tool approvals

Control confirmation behavior without bypassing hardware safety checks.

Get started

Core concepts

Complex workflows

Monitor and debug

Project tools

Customize

Integrations

Automation

CLI

Reference

API

Automated testing

Choose the lowest useful test layer

Record the project test workflow

Run host tests and static checks

Add an on-target smoke check

Create reusable hardware checks

Coordinate the bench

Turn a failure into a regression check

Run long or headless work deliberately

Preserve reviewable evidence

Hardware scripts

Combined workflows

Coverage and trace

Tool approvals

​Choose the lowest useful test layer

​Record the project test workflow

​Run host tests and static checks

​Add an on-target smoke check

​Create reusable hardware checks

​Coordinate the bench

​Turn a failure into a regression check

​Run long or headless work deliberately

​Preserve reviewable evidence

Hardware scripts

Combined workflows

Coverage and trace

Tool approvals

Choose the lowest useful test layer

Record the project test workflow

Run host tests and static checks

Add an on-target smoke check

Create reusable hardware checks

Coordinate the bench

Turn a failure into a regression check

Run long or headless work deliberately

Preserve reviewable evidence