CI concept: what runs when, and why #663

Closed
opened 2026-11-13 10:54:35 +01:00 by simon · 0 comments
Owner

Follows up #561 and #31. Absorbs the earlier idea of skipping the repeated full check after a merge; links fast-release-tag-steps.md, pr-preview-images.md, signed-release-tags.md, lint-gaps.md, browser-test-layer.md, timing-bound-tests-async.md.

Problem

The CI grew step by step, and one change now passes through more runs and gates than it needs:

push → check-fast → manual promote → check-full-promote → merge → check-full again → build-and-publish (compiles the app for image-tags) → tag → release (compiles the app again in release-files).

  • The full check runs twice on the same tree when the branch already contains current main.
  • The promote is a manual gate that only exists to start the full check.
  • check-fast has become fast enough that its split from check-full may no longer pay off.
  • Each pipeline runs mix deps.get and compiles several times (lint in dev, the tests in test, image-tags, release-files), and the steps run one after another.
  • CI tests against PostgreSQL 18.3 (.drone.jsonnet), production runs 18.6 (docker-compose.prod.yml).
  • concurrency: { limit: 1 } queues every repetition behind Renovate and other branches.

Goal

Write down from scratch what should run on which event, with its reason, and map today's setup to it: keep, change or drop each part. Then implement it.

Principles (to agree on)

  1. Every check runs once per tree that can still reach main. A tree that passed the full check is not checked again.
  2. A pull request that is ready to merge needs no extra manual step: opening or updating it runs the full check, and that check is the required status for merging.
  3. Fast feedback for work in progress stays cheap.
  4. Main is green by construction: it only receives trees that passed the full check, against the main they are merged into.
  5. Images are built once, from tested main commits; a release only promotes an image and builds nothing.
  6. Nothing a branch build can write (shared cache, files) is trusted by a pipeline that holds the registry or release token.
  7. CI runs the same service versions as production.

Proposed matrix (draft)

Event Runs Why
Push to a branch without an open PR fast check (compile, format, credo, fast tests) quick feedback, cheap
Push to a branch with an open, non-draft PR nothing beyond the PR run avoids a second run of the same commit
PR opened, updated or marked ready full check (lint, typecheck, all tests), tested on the merge result with main the gate for merging, replaces the promote
Draft PR (WIP:) fast check work in progress
Merge to main build and publish the image; full check only if the merged tree differs from the tree the PR tested main stays green without repeating the check
Tag check the tag, promote the image, draft the release, without compiling the app see fast-release-tag-steps.md
Renovate branches as PRs; automerge only after the full check many branches at once
Cron (optional) full check on main, deps.audit advisories appear without a code change

Questions to settle

  • Push with an open PR: Drone fires both push and pull_request for it. Options: a first step in the push pipeline asks the Forgejo API for an open PR from this branch and ends the run with exit 78 (successful skip); or branches get CI only through PRs.
  • What a PR build tests: does Drone with Forgejo check out the PR head or the merge result? If only the head, add a step that merges origin/main before testing, or require branches to be up to date before merging (Forgejo branch protection).
  • Skipping the full check on main: how the main pipeline learns that the tree was already tested (e.g. the commit status of the PR head, compared via HEAD^{tree}), without trusting the shared cache.
  • Merge fast and full: compare the run times of check-fast and check-full (Drone history) and decide whether one check with all tests is enough for every PR run.
  • Inside a pipeline: run lint, typecheck and tests in parallel steps; compile once per MIX_ENV; decide whether wait_for_postgres and deps.get repeat per step.
  • Not yet in CI: JS bundle and empty plurals (lint-gaps.md), browser tests (browser-test-layer.md), PR images (pr-preview-images.md), signed tags (signed-release-tags.md). Which of them belong in which row.

Acceptance criteria

  • A short CI document in the repository (e.g. docs/ci.md) states the matrix, the reason for each row, and which status checks branch protection requires.
  • Every row of the matrix is implemented in .drone.jsonnet, and every part of today's setup is either in the matrix or removed.
  • A PR needs no manual promote; its full check is the required status for merging.
  • A merge whose tree the PR's full check already passed does not repeat tests and typecheck before the image is built.
  • A push to a branch with an open PR does not start a second full run.
  • The image and the release are still built only from tested trees; no publishing pipeline reads data a branch build can write.
  • CI and production use the same PostgreSQL version.
  • The time from push to green PR, from merge to published main image, and from tag to draft release is measured before and after.
Follows up #561 and #31. Absorbs the earlier idea of skipping the repeated full check after a merge; links `fast-release-tag-steps.md`, `pr-preview-images.md`, `signed-release-tags.md`, `lint-gaps.md`, `browser-test-layer.md`, `timing-bound-tests-async.md`. ## Problem The CI grew step by step, and one change now passes through more runs and gates than it needs: push → `check-fast` → manual promote → `check-full-promote` → merge → `check-full` again → `build-and-publish` (compiles the app for `image-tags`) → tag → `release` (compiles the app again in `release-files`). - The full check runs twice on the same tree when the branch already contains current main. - The promote is a manual gate that only exists to start the full check. - `check-fast` has become fast enough that its split from `check-full` may no longer pay off. - Each pipeline runs `mix deps.get` and compiles several times (`lint` in dev, the tests in test, `image-tags`, `release-files`), and the steps run one after another. - CI tests against PostgreSQL 18.3 (`.drone.jsonnet`), production runs 18.6 (`docker-compose.prod.yml`). - `concurrency: { limit: 1 }` queues every repetition behind Renovate and other branches. ## Goal Write down from scratch what should run on which event, with its reason, and map today's setup to it: keep, change or drop each part. Then implement it. ## Principles (to agree on) 1. Every check runs once per tree that can still reach main. A tree that passed the full check is not checked again. 2. A pull request that is ready to merge needs no extra manual step: opening or updating it runs the full check, and that check is the required status for merging. 3. Fast feedback for work in progress stays cheap. 4. Main is green by construction: it only receives trees that passed the full check, against the main they are merged into. 5. Images are built once, from tested main commits; a release only promotes an image and builds nothing. 6. Nothing a branch build can write (shared cache, files) is trusted by a pipeline that holds the registry or release token. 7. CI runs the same service versions as production. ## Proposed matrix (draft) | Event | Runs | Why | |---|---|---| | Push to a branch without an open PR | fast check (compile, format, credo, fast tests) | quick feedback, cheap | | Push to a branch with an open, non-draft PR | nothing beyond the PR run | avoids a second run of the same commit | | PR opened, updated or marked ready | full check (lint, typecheck, all tests), tested on the merge result with main | the gate for merging, replaces the promote | | Draft PR (`WIP:`) | fast check | work in progress | | Merge to main | build and publish the image; full check only if the merged tree differs from the tree the PR tested | main stays green without repeating the check | | Tag | check the tag, promote the image, draft the release, without compiling the app | see `fast-release-tag-steps.md` | | Renovate branches | as PRs; automerge only after the full check | many branches at once | | Cron (optional) | full check on main, `deps.audit` | advisories appear without a code change | ## Questions to settle - **Push with an open PR**: Drone fires both `push` and `pull_request` for it. Options: a first step in the push pipeline asks the Forgejo API for an open PR from this branch and ends the run with `exit 78` (successful skip); or branches get CI only through PRs. - **What a PR build tests**: does Drone with Forgejo check out the PR head or the merge result? If only the head, add a step that merges `origin/main` before testing, or require branches to be up to date before merging (Forgejo branch protection). - **Skipping the full check on main**: how the main pipeline learns that the tree was already tested (e.g. the commit status of the PR head, compared via `HEAD^{tree}`), without trusting the shared cache. - **Merge fast and full**: compare the run times of `check-fast` and `check-full` (Drone history) and decide whether one check with all tests is enough for every PR run. - **Inside a pipeline**: run lint, typecheck and tests in parallel steps; compile once per `MIX_ENV`; decide whether `wait_for_postgres` and `deps.get` repeat per step. - **Not yet in CI**: JS bundle and empty plurals (`lint-gaps.md`), browser tests (`browser-test-layer.md`), PR images (`pr-preview-images.md`), signed tags (`signed-release-tags.md`). Which of them belong in which row. ## Acceptance criteria - A short CI document in the repository (e.g. `docs/ci.md`) states the matrix, the reason for each row, and which status checks branch protection requires. - Every row of the matrix is implemented in `.drone.jsonnet`, and every part of today's setup is either in the matrix or removed. - A PR needs no manual promote; its full check is the required status for merging. - A merge whose tree the PR's full check already passed does not repeat tests and typecheck before the image is built. - A push to a branch with an open PR does not start a second full run. - The image and the release are still built only from tested trees; no publishing pipeline reads data a branch build can write. - CI and production use the same PostgreSQL version. - The time from push to green PR, from merge to published `main` image, and from tag to draft release is measured before and after.
simon closed this issue 2026-11-13 14:18:14 +01:00
Sign in to join this conversation.
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
local-it/mitgliederverwaltung#663
No description provided.