Skip to content

feat: per-task status surfaces + generic repro/fix framework#1016

Merged
gregmagolan merged 1 commit into
mainfrom
multi_task_status_checks
May 13, 2026
Merged

feat: per-task status surfaces + generic repro/fix framework#1016
gregmagolan merged 1 commit into
mainfrom
multi_task_status_checks

Conversation

@gregmagolan
Copy link
Copy Markdown
Member

@gregmagolan gregmagolan commented Apr 23, 2026

Summary

This PR makes every built-in aspect-cli task (build / test / lint / format / gazelle / delivery) a first-class citizen on every status surface — GitHub check-runs, Buildkite annotations, the per-PR aggregated comment, and PR review comments — and introduces the framework that gets them there.

Before: only build / test had real check-run / annotation rendering; the others fell through to a generic bazel template that didn't reflect what they actually did. There was no Buildkite annotation feature at all.

After: each task has a dedicated render library, contributes its own copy-pasteable repro/fix commands, opens phase-tagged BK sections + Task-timing rows, and aggregates into a single sticky PR comment with kind-aware dedup.

The PR also reworks the AXL ↔ Rust runtime contract around tasks: Environment becomes WorkflowsEnvironment with optional sub-records, phases gain a proper TaskPhase Starlark type with caller-supplied emoji + display name, and a handful of cross-cutting helpers (feature_logger, bail_if_not_public_github, workflows_results_url, patch_download_cmd) replace duplicated patterns across features.

What's new, by area

Built-in tasks (build / test / lint / format / gazelle / delivery)

  • Every task now emits phase-tagged task_update(...) calls via a single Phase(name, description, emoji, display_name) record at the spawn / detect / format / apply / filter / checksum / resolve / record / query / download / deliver transitions. The runtime records each phase + auto-closes the active one on _impl exit (interrupted=True).
  • aspect format and aspect gazelle can now optionally upload their generated patch as a CI artifact (--upload-format-diff / --upload-gazelle-diff). The render libraries surface a clickable patch link plus a host-aware one-line download + git apply command (curl on Buildkite, gh api … | unzip -p … on GitHub Actions).
  • aspect format positional files: aspect format -- file1 file2 formats exactly those paths, bypassing change-detection.
  • aspect delivery off-runner preview: --mode=always --dry-run --track-state=false lets a developer dry-run delivery locally without a backend; --commit-sha is now only required when --track-state=true.
  • aspect lint: LintTrait.findings_destination accepts auto (default) / comments / annotations / both; only_annotate_changed_regions restricts inline annotations to lines visible in Files Changed. Fix-bearing findings post as review comments with <details> suggestion blocks; non-fix findings post as check-run annotations.

Per-kind result libraries (lib/<kind>_results.axl)

Five render libraries with a shared interface (init_data / render_check_output) that both GithubStatusChecks and BuildkiteAnnotations dispatch through:

Library Status What it renders
lib/bazel_results.axl extended Bazel targets, build metrics, invocation, options, metadata, workspace status, Task timing, plus the SHARED_DETAILS_BODY_TEMPLATE every other library splices into its tail
lib/lint_results.axl new By-severity + by-tool tables, lint findings with per-rule grouping, check-run annotation pipeline (compute → chunk → PATCH)
lib/format_results.axl new "N files need formatting" with the patch download block
lib/gazelle_results.axl new Out-of-date BUILD files with the patch download block
lib/delivery_results.axl new Counts-by-outcome (ok / skip / warn / fail / pending) with per-bucket tables and clickable Build URLs

Plus lib/check_dispatch.axl (new) — single RENDERERS registry both surface features dispatch through. Adding a new task kind is one table entry.

Status surfaces

  • GithubStatusChecks (feature/github_status_checks.axl) — pre-existing, significantly extended: per-task check-run dispatched by task_update.kind (was build/test-only). Posts annotations (multi-batch PATCH for >50). Carries the check-run URL on GitHubCheckRunTrait.html_url so sibling features can link to it.
  • BuildkiteAnnotations (feature/buildkite_annotations.axl) — new: parallel feature for BK; previously this code path didn't exist. Leading task pill (:aspect: task <code>...</code>) so a step that runs multiple tasks gets distinguishable annotations. Streaming updates unwrap <details> blocks so BK's auto-collapse doesn't snap them shut between refreshes.
  • GithubStatusComments (feature/github_status_comments.axl) — pre-existing, significantly extended: PR-level sticky comment aggregating every sibling task. New in this PR: severity-sorted h4 task groups, status-emoji bullets, 🔁 Reproduce and 🛠️ Fix sections that dedup commands across siblings (kind-aware), copy-button-friendly fenced code blocks, clickable links.
  • GithubLintComments (feature/github_lint_comments.axl) — pre-existing, significantly extended: rules_lint-driven PR review comments. New in this PR: cross-run reaping (stale comments from prior pushes get deleted), patch-derived suggestion blocks (with Apply buttons) as a second channel alongside the existing SARIF-derived per-finding comments.

Generic repro & fix framework

lib/repro_commands.axl (new) — three primitives every producer uses:

  • build_aspect_command(subcommand, flags, targets) — composes aspect <subcommand> <flags> -- <targets> with explicit flag values (no reliance on config.axl defaults), single-quotes paths with shell-meta characters, wraps long commands across lines.
  • task_cli_path(task)task(group=[...], name=...)"<group> <subgroup> <name>".
  • dedup_and_attribute(entries) — collapses identical (kind, command, description) tuples from sibling tasks into one row attributed to every contributor.
  • patch_download_cmd(patch_url, strip) — host-detects BK vs GHA and produces the right one-liner (curl … | git apply vs gh api … | unzip -p … | git apply).

Every built-in task writes data["repro_commands"] and data["fix_commands"] at terminal-emit time. The PR-comment aggregator and per-task check-run body both render them via the same dedup/attribution path.

Lifecycle + phase infrastructure

lib/lifecycle.axl (new):

  • TaskLifecycleTraittask_started, task_update, task_complete slots that all status surfaces hook.
  • task_update(ctx, lifecycle, status, progress, kind, data, priority, phase) — single emitter that prints the → <Phase> · <progress> CLI line, emits a BK section header on transitions (--- <emoji> <Phase> · …), records the phase boundary on ctx.task, and dispatches TaskUpdate to every handler. Takes a Phase input record (4 fields: name/description/emoji/display_name); paired with the runtime's TaskPhase snapshot value that comes back out of ctx.task.phases() / current_phase().
  • setup_phase() / preflight_phase() helpers — open named phases bracketing CI-platform setup work and pre-health-check hooks so the time shows up labeled in the Task-timing table.
  • ProgressStyles record — per-surface progress strings (body for status-surface "Last update" lines, stream for live terminal output with ANSI) so each surface can show its preferred form.

Cross-cutting AXL helpers

  • feature_logger(feature) in lib/environment.axl — returns struct(info, warn, error, trace) that prepends <Feature name>: to every message. Each feature declares _LOG = feature_logger("Buildkite annotations") once and call sites drop to _LOG.info(ctx.std, msg).
  • bail_if_not_public_github(std, log, skip_label) in lib/github.axl — single helper that logs an actionable "Aspect GitHub App is only on github.com" message and returns True for the caller to early-return when ASPECT_VCS_URL points elsewhere. Used by all three GitHub-touching features.
  • workflows_results_url(std) — single accessor for the Aspect Workflows results-page base URL across the three surface features that render Aspect Web UI links.
  • Severity-prefixed output (info / warn / error) — bold-cyan / yellow / red ANSI prefixes, gated on color_enabled(). Codified hierarchy + capitalization conventions in the env.axl docstring.

WorkflowsEnvironment refactor (was Environment)

lib/environment.axl:

WorkflowsEnvironment = record(
    remote_cache  = field(RemoteCache  | None, default = None),
    build_events  = field(BuildEvents  | None, default = None),
    runner        = field(Runner       | None, default = None),
    ci            = field(CI           | None, default = None),
)

Each sub-record is independently optional — populated by its own _read_* helper that returns None when the relevant ASPECT_WORKFLOWS_* env vars aren't set. get_workflows_environment(std) always returns a WorkflowsEnvironment; callers gate per field. Replaces the old if get_environment() is None short-circuit that prevented Workflows feature configuration from firing in plain GHA / BK jobs without an Aspect Workflows runner.

TaskPhase Starlark type + runtime changes

crates/axl-runtime/src/engine/task_info.rs:

  • New TaskPhase Starlark value with attributes: name, description, duration_ms, interrupted, emoji, display_name.
  • ctx.task.phases() -> list[TaskPhase] for closed phases.
  • ctx.task.current_phase() -> TaskPhase | None for the live one (snapshot computed per call).
  • ctx.task.phase(name, description, emoji, display_name) Starlark method gains emoji= + display_name= kwargs; runtime carries them on PhaseRecord / CurrentPhase.
  • Replaces five parallel current_phase_* attributes (name / elapsed_ms / description / emoji / display_name) with a single method returning a snapshot.

Tests + snapshot scaffolding

  • New _test.axl files alongside each _results.axl library — 17 PR-comment scenarios, 21 BK annotation scenarios, 16+ scenarios per kind (clean / partial / full failure / aborted / large-trim / mixed-status / per-phase live updates).
  • New feature/test_fixtures.axl with shared payload builders.
  • AXL unit tests in .aspect/axl.axl grew to 727 cases covering repro_commands lib, resolve_aspect_url, patch_download_cmd, task_cli_path, dedup_and_attribute, etc.

Documentation

Changes are visible to end-users: yes

  • Searched for relevant documentation and updated as needed: yes
  • Breaking change (forces users to change their own code or config): no
  • Suggested release notes appear below: yes

Suggested release notes

  • Buildkite annotations: new feature that posts per-task annotations to the BK step, dispatched by task kind through the same renderers GH check-runs use.
  • Status checks now cover every built-in task: lint, delivery, format, and gazelle render dedicated check-run summaries (previously they fell through to the generic build template).
  • Sticky PR comment (GithubStatusComments) now aggregates every sibling task's status with kind-aware 🔁 Reproduce and 🛠️ Fix sections that dedup commands across tasks.
  • GithubLintComments gains cross-run reaping and a second posting channel for rules_lint patch suggestions (<details> blocks with Apply buttons).
  • Lint check-run annotations: configurable via LintTrait.findings_destination (comments / annotations / both); annotations filter to changed-region lines by default.
  • Repro & Fix commands on every task: failing build/test/lint/format/gazelle/delivery runs surface copy-pastable reproducer and (where applicable) fix commands in both the per-task check-run body and the PR-comment summary.
  • aspect format positional files: aspect format -- file1 file2 formats exactly those paths, bypassing change-detection.
  • Off-runner delivery dry-run: aspect delivery --mode=always --dry-run --track-state=false -- //... works without a commit SHA when not connected to a delivery state backend.
  • aspect format / aspect gazelle patch upload: --upload-format-diff / --upload-gazelle-diff upload the diff as a CI artifact; the check-run body shows a clickable download link and a host-aware one-line apply command.

Test plan

  • aspect tests axl — 727 AXL unit tests
  • aspect dev test-template-snapshots — bazel_results (16 scenarios)
  • aspect dev test-lint-template-snapshots — render + annotation pipeline (8 scenarios)
  • aspect dev test-delivery-template-snapshots (13 scenarios)
  • aspect dev test-format-template-snapshots (13 scenarios)
  • aspect dev test-gazelle-template-snapshots (11 scenarios)
  • aspect dev test-bk-annotation-snapshots (21 scenarios across all kinds + severity styles)
  • aspect dev test-pr-comment-snapshots (17 scenarios including kind-aware dedup, severity sort, h4 grouping)
  • Live: run the branch against this PR. Confirmed per-task check-run bodies render with emoji headers + Reproduce/Fix sections; PR comment shows kind-grouped Reproduce/Fix with copy-buttons; format/gazelle patch downloads apply cleanly.

@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 15 times, most recently from 61fb57f to aa0c95f Compare April 29, 2026 11:32
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 2 times, most recently from 3905989 to 40aac17 Compare May 5, 2026 14:16
@gregmagolan gregmagolan changed the title multi task status checks feat: per-task status checks with lint annotations May 5, 2026
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 5 times, most recently from 1d77782 to 7bc08fd Compare May 8, 2026 00:47
@gregmagolan gregmagolan changed the title feat: per-task status checks with lint annotations feat: add buildkite annotations and status checks for remaining built-in tasks (lint, delivery, format, gazelle) May 8, 2026
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 6 times, most recently from c0fe2ec to fd64f02 Compare May 9, 2026 04:22
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 3 times, most recently from d0f85c0 to 8d82ed4 Compare May 11, 2026 00:20
gregmagolan added a commit that referenced this pull request May 11, 2026
GitHub's PR Files API occasionally returns files with status=modified
but additions=0/deletions=0/patch=null — rename-only entries, or
degenerate cases where a rebase brought the file's content in sync
with the new base while still listing it as part of the PR. Concrete
example on PR #1016: examples/lint/hello.sh is listed as "modified"
but the API says changes=0 and patch=null, even though a local
`git diff base..head` against the same SHAs shows real adds. (Likely
a stale-rebase artifact on the GitHub side; the API is the authority
for what reviewers see in Files Changed.)

Previous behaviour kept these files in changed_files with empty
`lines` / `hunk_lines`, which caused:
  - hold_the_line (file-level only) passed every finding through to
    the rendered details body
  - line-level diff filter (`_filter_by_diff`,
    `compute_annotations`) rejected every comment / annotation
    because `(line - 1) in []` is always False

Net effect: lint findings rendered in the GHSC / BK details but
silently never annotated or commented on the PR — asymmetric and
confusing.

Skip these files in detect_changed_files so every surface stays
aligned: a file the API reports as zero-changes contributes nothing
to any surface (no findings in rendered details, no annotations, no
PR comments). If the local diff disagrees with the API, that's a
GitHub-side issue (likely fixable by rebasing the PR onto current
base) and falls outside the lint task's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 5 times, most recently from a46ffe7 to 1e2e232 Compare May 11, 2026 22:28
@gregmagolan gregmagolan changed the title feat: add buildkite annotations and status checks for remaining built-in tasks (lint, delivery, format, gazelle) feat: per-task status surfaces + generic repro/fix framework May 11, 2026
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 3 times, most recently from cb345c5 to 32db49a Compare May 11, 2026 23:09
gregmagolan added a commit that referenced this pull request May 12, 2026
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 3 times, most recently from 6d91e82 to d6dc514 Compare May 12, 2026 00:56
gregmagolan added a commit that referenced this pull request May 12, 2026
gregmagolan added a commit that referenced this pull request May 12, 2026
@gregmagolan gregmagolan force-pushed the multi_task_status_checks branch 5 times, most recently from 7b4e955 to 2d0f356 Compare May 12, 2026 13:28
@gregmagolan
Copy link
Copy Markdown
Member Author

Where the lines went

Total: +16 939 / −2 408 across 59 files. The grand-bucket breakdown:

Production AXL              11 847
Tests + scenarios + config   3 418
Rust runtime + cli             804
Docs (README/DEVELOPMENT.md)   830

Built-in task implementations (crates/aspect-cli/src/builtins/aspect/*.axl)

  830  lint.axl
  561  delivery.axl
  462  format.axl
  282  gazelle.axl
   11  build.axl
   11  test.axl
─────
2 157  total

(build/test stay small — they delegate to lib/bazel_runner.axl.)

Cross-cutting features (feature/*.axl, production)

1 056  github_lint_comments.axl    (extended — channels 2+3, cross-run reaping)
  691  github_status_comments.axl  (extended — kind-aware dedup, Reproduce/Fix sections)
  424  buildkite_annotations.axl   (new — entire feature)
  386  github_status_checks.axl    (extended — kind-dispatched check-runs, annotations)
   39  workflows.axl
    8  artifacts.axl
─────
2 604  total

Per-kind & shared libraries (lib/*.axl, production)

1 708  bazel_results.axl          (shared body templates + summary + details renderer)
  972  lint_results.axl           (new)
  595  lifecycle.axl              (new — TaskLifecycleTrait + task_update + Phase record)
  514  environment.axl            (WorkflowsEnvironment + feature_logger + …)
  477  delivery_results.axl       (new)
  457  github.axl                 (API helpers + bail_if_not_public_github)
  335  format_results.axl         (new)
  317  bazel_runner.axl           (shared build/test runner)
  280  repro_commands.axl         (new)
  261  patch.axl                  (new)
  256  gazelle_results.axl        (new)
  198  check_dispatch.axl         (new — single RENDERERS registry)
   98  sarif.axl
   84  buildkite.axl              (artifact upload + URL resolution)
   69  health_check.axl
   45  result_text.axl            (new)
   36  artifacts.axl              (ArtifactsTrait)
   21  checkrun.axl               (new — GitHubCheckRunTrait)
    8  build_metadata.axl
    2  tar.axl
─────
6 733  total

Jinja2 template subtotal (carved out of the libraries / features above)

Among the production AXL above, 585 lines are Jinja2 template bodies (the _DETAILS_TEMPLATE / _SUMMARY_TEMPLATE / _DEFAULT_BODY_TEMPLATE / SHARED_DETAILS_BODY_TEMPLATE / REPRO_COMMANDS_BLOCK / FIX_COMMANDS_BLOCK / BAZEL_TARGETS_TEMPLATE constants — the actual Jinja2 text that produces the rendered check-run / annotation / PR-comment markdown):

358  lib/bazel_results.axl       (SUMMARY + DETAILS + SHARED_DETAILS_BODY + BAZEL_TARGETS
                                  + REPRO_COMMANDS_BLOCK + FIX_COMMANDS_BLOCK)
 60  lib/format_results.axl      (_SUMMARY + _DETAILS w/ patch block)
 48  lib/gazelle_results.axl     (_SUMMARY + _DETAILS w/ patch block)
 46  lib/lint_results.axl        (_SUMMARY + _DETAILS w/ severity tables + linters chart)
 46  feature/github_status_comments.axl  (_DEFAULT_BODY_TEMPLATE)
 27  lib/delivery_results.axl    (_SUMMARY + _DETAILS w/ outcome buckets)
───
585  total (about 5% of production AXL)

Tests + snapshot scenarios + AXL config

 950  feature/github_status_comments_test.axl  (17 PR-comment scenarios — new)
 672  lib/lint_results_test.axl                (8 scenarios + annotation pipeline — new)
 570  .aspect/axl.axl                          (AXL unit tests — 727 cases)
 333  lib/delivery_results_test.axl            (13 scenarios — new)
 284  feature/test_fixtures.axl                (shared payload builders — new)
 243  lib/format_results_test.axl              (13 scenarios — new)
 212  lib/gazelle_results_test.axl             (11 scenarios — new)
 201  lib/bazel_results_test.axl               (16 scenarios)
 181  feature/buildkite_annotations_test.axl   (21 scenarios — new)
  55  .aspect/config.axl                       (test-suite registry)
─────
3 701  total

Rust runtime + aspect-cli

388  crates/axl-runtime/src/engine/task_info.rs       (TaskPhase + phase plumbing)
369  crates/axl-runtime/src/eval/multi_phase.rs       (phase-aware closing line render)
 20  crates/aspect-cli/src/cmd.rs                     (snapshot-test dev tasks)
 10  crates/axl-runtime/src/engine/bazel/build.rs     (BES sinks debug eprintln)
  8  crates/aspect-cli/src/trace.rs
  7  crates/axl-runtime/src/test.rs
  1  crates/axl-runtime/src/eval/mod.rs
  1  crates/aspect-cli/src/main.rs
───
804  total

Documentation

625  crates/aspect-cli/src/builtins/aspect/DEVELOPMENT.md   (contributor guide)
187  crates/aspect-cli/src/builtins/aspect/README.md        (user-facing reference)
 18  DEVELOPMENT.md                                          (root pointer)
───
830  total

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6721835da5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/aspect-cli/src/builtins/aspect/format.axl
Comment thread crates/aspect-cli/src/builtins/aspect/gazelle.axl
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant