CI/CD for the 6.5_GUI repository¶

A complete, beginner-friendly guide to how Continuous Integration and Continuous Delivery work in this repo, how to operate it as the principal maintainer, and how to grow it as the codebase and the team grow.

If you have never touched CI/CD before, read this top to bottom once. After that, use it as a reference. Nothing here requires prior DevOps experience.

1. CI/CD 101 (start here)¶

Continuous Integration (CI) means: every time someone proposes a change, a server automatically checks out the code, builds it, and runs the tests. If anything is broken, the author finds out in minutes — not after it has been merged and broken everyone else's work.

Continuous Delivery/Deployment (CD) means: once code is accepted, a server automatically produces the shippable output (a firmware binary, an installable GUI package, a release on GitHub) so releasing is a button press, not a manual ritual only one person knows how to do.

Why you want this now¶

Right now the repo is small and you know every corner of it. As it grows (full C++ port, planner + VAMP integration, firmware fixes, new features) and new maintainers join, three things happen without CI/CD:

People break each other's code and nobody notices until much later.
"It works on my machine" becomes the default failure mode.
Only you know how to build/test/release each piece, so you become a bottleneck.

CI/CD fixes all three by writing the build/test/release recipe down as code and running it automatically for everyone, identically, every time.

The mental model¶

flowchart LR
    write[Write code on a branch] --> push[Push to GitHub]
    push --> ci["CI: build + test automatically"]
    ci -->|red| fix[Fix and push again]
    fix --> ci
    ci -->|green| review[Human review]
    review --> merge[Merge to main]
    merge --> cd["CD: build artifacts / release"]

The key idea: the same checks run for everyone, automatically, before code can reach main. main stays permanently in a known-good state.

2. How GitHub Actions works¶

This repo lives on GitHub (orangewood-co/6.5_GUI), so we use GitHub Actions — GitHub's built-in CI/CD. There is nothing extra to install or pay for at our scale; it is configured entirely with text files in the repo.

The anatomy of a workflow¶

A workflow is a YAML file in .github/workflows/. Its structure:

name: gui-ci                 # human-readable name shown in the Actions tab

on:                          # TRIGGER: when should this run?
  pull_request:
    paths: ["gui/**"]        # only when files under gui/ change

jobs:                        # one or more JOBS (each runs on a fresh machine)
  test:
    runs-on: ubuntu-latest   # the RUNNER: a throwaway VM GitHub gives us
    steps:                   # STEPS run in order on that machine
      - uses: actions/checkout@v4      # a reusable "action" (checks out code)
      - run: pytest gui/tests          # a shell command

Terms, in plain English:

Workflow — a recipe file. We have one per component.
Trigger (on:) — the event that starts it (a pull request, a push to main, a version tag, a manual click, a schedule).
Job — a unit of work that runs on its own fresh virtual machine. Different jobs run in parallel by default.
Runner — the machine the job runs on. ubuntu-latest is a clean Linux VM provided by GitHub. It is destroyed after the job, so every run is reproducible.
Step — a single command or a reusable action (like actions/checkout, which downloads your code onto the runner).
Status check — the green tick / red cross a workflow reports back to the pull request. We can require specific checks to be green before merging.
Matrix — running the same job several times with different inputs (e.g. Python 3.10 and 3.12) to catch version-specific breakage.
Cache — saving slow-to-produce files (downloaded pip packages, compiler toolchains) between runs so later runs are fast.
Artifact — a file a job produces and uploads for you to download later (e.g. the compiled firmware.bin).
Concurrency — automatically cancelling an in-progress run when you push again, so you don't waste minutes on stale code.

Where to watch it happen¶

On GitHub, the Actions tab lists every run. Click a run to see its jobs; click a job to see each step's live logs. Failures are also annotated directly on the "Files changed" and "Checks" tabs of the pull request, so a contributor sees exactly which line/test failed without leaving the PR.

3. This repo's pipeline map¶

The repo is a monorepo: several independent projects in one Git repository. Each has a different build system, so we use one workflow per component and gate each with paths: filters. A firmware-only PR therefore does not trigger the GUI or planner pipelines — this keeps runs fast and cheap.

flowchart TD
    pr[Pull request opened / updated] --> lint["lint.yml (always): pre-commit"]
    pr --> which{Which paths changed?}
    which -->|"gui/**"| gui["gui-ci.yml: pytest on Py 3.10 + 3.12, headless Qt"]
    which -->|"6.5/**"| fw["firmware-ci.yml: PlatformIO build, upload .bin artifact"]
    which -->|"planners/**"| pl["planners-ci.yml: CMake build in Docker + collision test"]
    which -->|"web_app / bench / bridge"| tool["tooling-ci.yml: ruff + syntax check"]
    lint --> gate[Required status checks]
    gui --> gate
    fw --> gate
    pl --> gate
    tool --> gate
    gate -->|all green + review| merge[Merge to main]

Workflow	File	Trigger (paths)	What it does	Why build-only?
Lint	`.github/workflows/lint.yml`	every PR	Runs all `pre-commit` hooks (ruff, black-style format, clang-format, whitespace)	Fast, catches style/format issues everywhere
GUI	`.github/workflows/gui-ci.yml`	`gui/**`	Installs `owlgui_v2`, runs the `pytest` suite headless on Python 3.10 & 3.12	Real test suite exists here
Firmware	`.github/workflows/firmware-ci.yml`	`6.5/**`	`pio run` builds the STM32 image, uploads `firmware.bin`/`.elf`	No board attached to the runner; see section 8
Planners	`.github/workflows/planners-ci.yml`	`planners/**`	Builds the C++ library in a Docker image with OMPL/Pinocchio/Coal, runs `collision_check_test`	Heavy deps; container makes it reproducible
Tooling	`.github/workflows/tooling-ci.yml`	`web_app`, `bench`, `bridge`, `motor_test_cli.py`	Lints + byte-compiles the supporting Python	No test suite yet; guards against syntax/lint regressions

Why the firmware and planner jobs only build¶

GitHub-hosted runners are anonymous Linux VMs in a data center. They have no STM32 board, no motors, no CAN bus wired to them. So CI can prove the code compiles and links (which catches the majority of mistakes), but it cannot prove the motor spins correctly. Verifying real motion still requires a human with hardware, or a dedicated self-hosted runner (section 8).

4. Maintainer setup checklist (one-time)¶

Do these once, in the GitHub web UI, as the repo owner/admin. This is what turns the workflow files into an enforced process.

4.1 Enable Actions¶

Settings -> Actions -> General: - Allow actions and reusable workflows. - Under "Workflow permissions", pick Read repository contents (least privilege). The release workflow that needs write access requests it explicitly with a permissions: block, so the default stays locked down.

4.2 Protect the `main` branch¶

This is the single most important step. Settings -> Branches -> Add branch ruleset (or "Add classic branch protection rule") targeting main:

[x] Require a pull request before merging (blocks direct pushes to main).
[x] Require approvals — set to at least 1.
[x] Require review from Code Owners (uses .github/CODEOWNERS).
[x] Require status checks to pass before merging, and select: pre-commit, and the component checks (test, build, etc.) you want mandatory. Tip: a check only appears in this list after it has run at least once, so open a throwaway PR first, then come back and select them.
[x] Require branches to be up to date before merging (forces a rebase/merge of main so checks reflect the final combined code).
[x] Do not allow bypassing the above settings (applies rules to admins too — recommended once the process is stable).

4.3 Set code owners¶

Edit .github/CODEOWNERS and replace the @REPLACE_WITH_* placeholders with real GitHub usernames or teams. Combined with "Require review from Code Owners", this auto-requests the right reviewer for each subsystem.

4.4 Add people with the right permissions¶

Settings -> Collaborators and teams. Prefer teams (e.g. firmware, gui) over individuals. Useful roles: - Write — most contributors: can push branches and open PRs, cannot change settings or bypass protection. - Maintain — trusted leads: can manage issues/PRs but not destructive repo settings. - Admin — you (and maybe one backup).

4.5 Secrets (only when needed)¶

Settings -> Secrets and variables -> Actions. Store tokens/keys here; never commit them. Reference them in a workflow as ${{ secrets.NAME }}. You do not need any secret for the current pipelines — add them only when a job must publish somewhere authenticated.

4.6 Turn on the local hooks yourself¶

Even as maintainer, run pre-commit install in your clone (see section 6) so your own commits are clean.

5. The contributor workflow¶

This is the loop every contributor (including you) follows. The short version also lives in CONTRIBUTING.md.

sequenceDiagram
    participant Dev as Contributor
    participant GH as GitHub
    participant CI as Actions (CI)
    participant Rev as Reviewer
    Dev->>Dev: git checkout -b feat/thing
    Dev->>Dev: code + pre-commit (local)
    Dev->>GH: git push; open Pull Request
    GH->>CI: trigger path-matched workflows
    CI-->>GH: report status checks (green/red)
    Rev->>GH: review, request changes
    Dev->>GH: push fixes (CI re-runs)
    Rev->>GH: approve
    Dev->>GH: merge (only if checks green + approved)

Run the same checks locally before pushing to avoid red CI:

pre-commit run --all-files                          # lint/format everything
QT_QPA_PLATFORM=offscreen pytest gui/tests -v       # GUI tests
cd 6.5 && pio run -e nucleo_f446re                  # firmware build
# planner build (in the Docker toolchain):
docker run --rm -v "$PWD/planners:/work" -w /work owl-planners-ci \
  bash -lc 'cmake -S . -B build && cmake --build build -j && ./build/collision_check_test'

6. Local pre-commit hooks¶

pre-commit runs formatters and linters automatically on git commit, so issues are fixed on your laptop instead of failing CI minutes later. It uses .pre-commit-config.yaml.

pip install pre-commit
pre-commit install          # installs the git hook (one-time per clone)

pre-commit run --all-files  # run against the whole repo on demand

What runs: - ruff — Python lint + autoformat (config in ruff.toml). - clang-format — C/C++ formatting (config in .clang-format). - hygiene hooks — strip trailing whitespace, ensure final newline, block huge files and unresolved merge conflicts, validate YAML/TOML/JSON.

The same hooks run in CI via lint.yml, so "passes locally" means "passes CI".

Note: the config starts intentionally lenient (line length 199, many rules ignored) to match the existing code so it doesn't flag thousands of pre-existing lines. Tighten the rules in ruff.toml gradually as the tree is cleaned up.

7. Growing the pipeline as the repo grows¶

The whole point is that this scales with you. Common next steps:

Add a job for a new component / the C++ port. Copy the closest existing workflow, change its name, its paths: filter, and its build commands. For the ongoing C++ port, extend planners-ci.yml (or add cpp-core-ci.yml) and reuse planners/Dockerfile.ci as the toolchain.

Add VAMP integration. VAMP (vectorized motion planning) is another C++/ Python dependency. Add its install to planners/Dockerfile.ci so both local and CI builds get it identically, then add a build/test step. Keeping deps in the Dockerfile is exactly what makes this painless later.

Add new firmware targets. When a second board/env appears in platformio.ini, turn the firmware build into a matrix:

strategy:
  matrix:
    env: [nucleo_f446re, your_new_board]
steps:
  - run: pio run -e ${{ matrix.env }}
    working-directory: 6.5

Add code coverage. In gui-ci.yml, pip install pytest-cov and run pytest --cov=owlgui_v2 --cov-report=xml, then upload the report as an artifact or to a coverage service.

Add a scheduled job. Use on: schedule: - cron: "0 3 * * 1" for weekly dependency-audit or nightly full builds.

Speed things up. Every workflow already uses concurrency (cancels stale runs) and caching (pip, PlatformIO, Docker layers). Add more caches as new heavy steps appear.

8. Hardware-in-the-loop testing¶

CI cannot touch real hardware, but you can extend it to when you're ready:

Self-hosted runner. Install the GitHub Actions runner agent on a lab PC that has a Nucleo board + motors attached (Settings -> Actions -> Runners -> New self-hosted runner). Give it a label like hw-nucleo. Then a job with runs-on: [self-hosted, hw-nucleo] can pio run -t upload and run real motion tests. Guard it so it only runs on demand (workflow_dispatch) or on main, never on untrusted fork PRs (a self-hosted runner runs arbitrary PR code on your lab machine — treat that as a security boundary).
Until then, hardware verification stays manual: the PR template asks the author to confirm they tested on real hardware and describe the setup.

9. CD: artifacts and releases¶

"Delivery" here means producing the things you actually ship.

Build artifacts (already wired). firmware-ci.yml uploads firmware.bin/firmware.elf on every run. Download them from the run's "Artifacts" section — handy for flashing a specific commit without rebuilding.

Tagged releases (recommended next step). Adopt semantic version tags (v1.2.0) and add a release workflow triggered by tags:

on:
  push:
    tags: ["v*"]
permissions:
  contents: write        # needed to create a GitHub Release
jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # ... build firmware.bin and the GUI wheel (python -m build gui) ...
      - uses: softprops/action-gh-release@v2
        with:
          files: |
            6.5/.pio/build/nucleo_f446re/firmware.bin
            gui/dist/*.whl

Cutting a release then becomes: git tag v1.2.0 && git push origin v1.2.0. GitHub builds the binaries and publishes a Release page with downloadable assets and auto-generated notes.

A simple, effective versioning scheme: - vMAJOR.MINOR.PATCH - bump PATCH for fixes, MINOR for new features, MAJOR for breaking changes (e.g. a firmware protocol change that needs a matching GUI update).

10. Troubleshooting¶

Symptom	Likely cause	Fix
GUI job fails with `qt.qpa.plugin: could not load the Qt platform plugin "xcb"`	No display on the runner	It should use `QT_QPA_PLATFORM=offscreen` (already set) and run under `xvfb-run`; ensure the headless libs step ran
GUI test import errors on `trimesh`/`urdfpy`	Optional `viz` extras missing	Install with `pip install -e "gui/.[viz]"`
Planner job: `Could NOT find pinocchio / ompl / coal`	Toolchain not present	It builds inside `planners/Dockerfile.ci`; if editing deps, update that file so CMake finds `/opt/openrobots`
Firmware job slow every time	Cache miss	The `~/.platformio` + `6.5/.pio` cache key is tied to `platformio.ini`; a change invalidates it once, then it's fast again
A required check never turns green / can't be selected in branch protection	The check hasn't run yet	Open a PR that touches the relevant path so the check appears, then add it to the required list
`pre-commit` reformats files in CI and fails	You committed unformatted code	Run `pre-commit run --all-files` locally, commit the changes, push again
Workflow doesn't trigger at all	`paths:` filter didn't match, or the branch is a fork with restricted secrets	Confirm your changed files match the workflow's `paths:` block

Debugging tips¶

Read the failing step's log in the Actions tab; the error is almost always in the last red step.
Re-run a single failed job with "Re-run failed jobs" (top-right of a run).
Reproduce locally with the exact commands from the workflow file — the runner just runs shell commands you can run too.

11. Glossary¶

CI — Continuous Integration: auto build + test on every change.
CD — Continuous Delivery/Deployment: auto-produce releasable artifacts.
Workflow — a YAML recipe in .github/workflows/.
Job — work that runs on one fresh runner VM; jobs run in parallel.
Runner — the (throwaway) machine a job runs on; ubuntu-latest is GitHub-hosted, a self-hosted runner is your own machine.
Step / Action — a command, or a reusable pre-built step (uses:).
Status check — the pass/fail signal a workflow reports to a PR.
Branch protection — rules that block merging until checks pass + review.
CODEOWNERS — file mapping paths to required reviewers.
Matrix — running one job across multiple versions/inputs.
Cache / Artifact — reused inputs (speed) / produced outputs (downloads).
HIL — Hardware-in-the-loop: tests that run against real hardware.
Monorepo — multiple projects living in one Git repository.