Skip to content

chore(verify): add scripts/verify.sh + ruff/basedpyright/import-linter/pytest --cov harness#73

Merged
keli-wen merged 9 commits intomasterfrom
chore/verify-loop
Apr 26, 2026
Merged

chore(verify): add scripts/verify.sh + ruff/basedpyright/import-linter/pytest --cov harness#73
keli-wen merged 9 commits intomasterfrom
chore/verify-loop

Conversation

@keli-wen
Copy link
Copy Markdown
Contributor

@keli-wen keli-wen commented Apr 25, 2026 •
edited
Loading

Part of #71. Closes #72.

Description

  • Adds scripts/verify.sh: a single-command, fast-fail verify loop — ruff format --check → ruff check → basedpyright → lint-imports → pytest --cov. CI runs the same script via .github/workflows/verify.yml, so green local = green PR.
  • Configures the five tools in pyproject.toml so the harness is forward-looking: basedpyright and import-linter only enforce on permanent + post-migration modules (utils/logger, future knowledge/, configs/, preprocess/, flows/, mind/, magic.py); transitional code slated for deletion in PR3-PR5 is excluded.
  • Wires pre-commit (verify hook on pre-push), keeps pre-commit.yml as an orthogonal CI safety net for file-hygiene hooks (whitespace, EOF, yaml, large files, merge conflicts), and updates CLAUDE.md to point at scripts/verify.sh as the canonical local check.

Tool Configuration

Tool Setup Notes
ruff D, E, F, I, W, B, W505 (skips E501; ruff format handles wrapping). Per-file-ignores: tests waive D, B; __init__.py waives D104, F401. Cleanup: ~41 lint fixes (auto + manual E402, F601, E722, F401, F841).
basedpyright typeCheckingMode = "standard", pythonVersion = "3.10". include = quantmind; excludes transitional dirs. Future modules auto-pick-up. One type fix: utils/logger.py use_color annotation.
import-linter root_packages = ["quantmind"]. One Forbidden contract: utils is a permanent leaf. More contracts land per-PR as new modules arrive.
pytest + coverage --cov=quantmind, --cov-fail-under=60, branch coverage on. 60% accommodates branch-coverage on transitional modules (pdf_parser.py 26%, tmp.py 20%). Ratchet to 70%+ in PR3 once they're deleted.
pre-commit Bumps ruff to v0.11.2 (matches runtime). New verify hook on pre-push. Fixes format ping-pong from version mismatch.

CI Workflows (orthogonal)

Workflow Triggers What it runs Why
pre-commit.yml PR + push to master pre-commit run --all-files --hook-stage pre-commit (whitespace, EOF, yaml, large files, merge conflicts, ruff, ruff-format) File hygiene + style — fast safety net for contributors who skip local hooks
verify.yml PR + push to master bash scripts/verify.sh (full 5-step verify loop) Type checks, architecture contracts, tests + coverage — the migration-protection gate

Open Questions Resolved

  • basedpyright vs mypy → basedpyright (faster, stricter, matches llmquant-data-core).
  • Coverage floor starting value → 60% (forced lower than the issue's 70% by branch-coverage on transitional code; ratchet path documented).
  • CI compute → ~1m16s for verify, ~19s for pre-commit; well under budget.

Checklist

Test Plan

  • bash scripts/verify.sh exits 0 locally
  • pre-commit run --all-files --hook-stage pre-commit green
  • pre-commit run --all-files --hook-stage pre-push green
  • git push triggers pre-push verify and succeeds (revealed a PATH issue on first push, fixed in scripts/verify.sh by prepending .venv/bin)
  • Both CI workflows pass (pre-commit ✓ in 19s, verify ✓ in ~1m16s)

🤖 Generated with Claude Code

keli-wen and others added 8 commits April 26, 2026 01:14
@keli-wen @claude
chore: bump pre-commit ruff to v0.11.2 and reformat tmp.py
f5cbfb5 
Pre-commit ruff was pinned at v0.3.3 while the project's runtime ruff is
0.11.2. The two disagree on assert-message wrapping (PEP 8 trailing-comma
behavior changed between releases), causing every push to bounce between
the two formats. Bump pre-commit to match the runtime version, then
reformat the one affected file (quantmind/utils/tmp.py).

Pre-work for landing scripts/verify.sh, which runs ruff format --check
as gate 1 of the verify loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
chore: apply ruff E/F/I/W/B fixes across quantmind/ and tests/
8a59ea5 
Pre-work for tightening the ruff lint config in scripts/verify.sh.
Auto-fixes account for the bulk: unused imports (F401), unsorted imports
(I001), redundant open-modes (UP015), f-strings without placeholders
(F541), pep585 type annotations (UP006/UP007).

Manual fixes:
- config/settings.py: hoist `from dotenv import load_dotenv` to top of
  module (E402)
- parsers/base.py: replace duplicate empty-string keys in the smart-quote
  substitution table with explicit unicode codepoints (F601). The
  original keys had been ASCII-fied at some point in history, leaving
  four duplicate `""` entries that collapsed to a single dead entry.
- parsers/pdf_parser.py: narrow bare `except:` to `except OSError:`
  around os.unlink cleanup (E722)
- sources/__init__.py: add `# noqa: F401` for the conditional re-export
  of ArxivSource (the surrounding try/except + __all__.append already
  documents the intent)
- tests/llm/, tests/config/test_storage.py: drop assignments to
  `_ = block(...)` / `_ = block.method(...)` calls that test side
  effects via mocks (F841)

192 tests still pass. No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
fix(utils/logger): annotate use_color as bool | None
ae39e1c 
ColoredFormatter.__init__ took use_color: bool = None (mismatched type),
which basedpyright at standard mode flags as reportArgumentType. Fix the
annotation to the correct bool | None — the runtime behavior already
handled None via the auto-detect branch, only the type was wrong.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
feat: configure golden-harness in pyproject.toml
37fb484 
Centralize verify-loop tooling configuration so scripts/verify.sh can
run as a single command:

ruff
- Tighten lint rules: D, E, F, I, W, B, W505 (skip E501; ruff format
  handles wrapping). Per-file-ignores: tests waive D and B; __init__
  waives D104/F401.

basedpyright
- typeCheckingMode = "standard", pythonVersion = "3.10"
- Include = quantmind, exclude transitional modules (config/, flow/,
  llm/, parsers/, sources/, models/{analysis,content,paper}.py,
  utils/tmp.py) — they get deleted in PR3-PR5 and aren't worth
  modernizing for type checks. New modules (knowledge/, configs/,
  preprocess/, flows/, mind/, magic.py) automatically get checked at
  standard mode as they land.

import-linter
- root_packages = ["quantmind"], one Forbidden contract: utils is a
  permanent leaf (cannot import config/flow/llm/models/parsers/sources).
  Additional contracts will be added per-PR as new modules land.

pytest + coverage
- testpaths = ["tests"], addopts = "--cov=quantmind --cov-report=term
  -missing --cov-fail-under=60". Floor 60% accommodates branch coverage
  on transitional modules; ratchet to 70%+ once they're deleted.
- coverage.run: source = quantmind, branch = true; coverage.report
  excludes pragma/TYPE_CHECKING/NotImplementedError/__main__ branches.

Dev deps consolidated to ruff, basedpyright, import-linter, pytest,
pytest-cov, pre-commit. Black/isort/mypy dropped — superseded by ruff
and basedpyright respectively.

.gitignore: ignore .coverage / htmlcov/ / coverage.xml artifacts and
docs/superpowers/ (local working notes).

Part of #71. Closes #72.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
feat: add scripts/verify.sh and wire pre-commit to it
101d578 
scripts/verify.sh runs the five-step golden harness in fixed order,
fast-failing on the first error:
  1. ruff format --check
  2. ruff check
  3. basedpyright
  4. lint-imports
  5. pytest --cov

.pre-commit-config.yaml: replace the legacy unittest hook (which only
ran tests on pre-push) with a verify hook that calls scripts/verify.sh.
Pre-commit-stage hooks (ruff, ruff-format, whitespace) are unchanged —
they give fast per-commit feedback; verify.sh runs once per pre-push for
the full discipline.

Removed the now-superseded helpers:
- scripts/lint.sh (running ruff format + ruff check)
- scripts/unittest.sh (running pytest with arg dispatch)

Both are subsumed by scripts/verify.sh.

Part of #71. Closes #72.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
ci: add verify workflow and retire pre-commit workflow
8dc9d13 
.github/workflows/verify.yml runs scripts/verify.sh on every PR to
master and on every push to master, with concurrency cancellation so
later commits supersede earlier in-progress runs. Uses uv to create
the venv and install the project with [dev] extras, mirroring the
local development setup documented in CLAUDE.md.

Removed .github/workflows/pre-commit.yml — superseded by verify.yml.
The legacy workflow ran `pre-commit run --all-files` which only
covered ruff + the local unittest hook; verify.sh now covers ruff +
basedpyright + import-linter + pytest --cov in one place, so running
both would be redundant.

Part of #71. Closes #72.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
docs(claude): document scripts/verify.sh + refresh PR roadmap
30bd7c0 
Replace the ad-hoc "ruff format / ruff check / pytest" instructions with
a pointer to scripts/verify.sh as the single canonical local check.
Document the five-step loop (ruff format/check, basedpyright,
import-linter, pytest --cov), the 60% coverage floor and ratchet plan,
and the pre-commit / pre-push split.

Refresh the roadmap table to reflect the actual sequencing — PR2 is the
Golden Harness (this PR), and the knowledge/configs/preprocess/flows/
mind work moves out by one slot. Refresh the transitional-modules table
to match (PR2->PR3, PR3->PR4, etc.) and add a note that transitional
modules are excluded from basedpyright while new modules auto-pick-up
at standard mode.

Part of #71. Closes #72.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen @claude
fix(verify.sh): prepend .venv/bin to PATH for hook environments
eafd657 
Pre-commit's pre-push stage spawns a fresh shell without an activated
venv, so verify.sh's calls to ruff/basedpyright/lint-imports/pytest
were resolving against the system PATH where the project deps aren't
installed (manifested as "ModuleNotFoundError: No module named 'arxiv'"
during pytest collection on git push).

Prepend $REPO_ROOT/.venv/bin to PATH at the top of the script if the
venv exists. CI provisions its own venv before invoking this script,
so the conditional handles both environments.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen keli-wen self-assigned this Apr 25, 2026
@keli-wen @claude
ci: restore pre-commit workflow as orthogonal safety net
ff6da45 
verify.sh covers ruff + basedpyright + import-linter + pytest, but the
file-hygiene hooks in .pre-commit-config.yaml (trailing-whitespace,
end-of-file-fixer, check-yaml, check-added-large-files,
check-merge-conflict) aren't part of verify.sh. Restoring pre-commit.yml
so those keep running in CI as a safety net for contributors who skip
local hooks.

Refinements vs the original:
- Python 3.10 (matches verify.yml and the project's requires-python)
- Adds push-to-master trigger (matches verify.yml)
- Adds concurrency cancellation
- Restricts to --hook-stage pre-commit (verify.sh runs separately via
  verify.yml, no need to re-run it through pre-commit's pre-push stage)
- Bumps actions/checkout, setup-python, cache to current major versions

The two workflows are orthogonal: pre-commit.yml = file hygiene + ruff,
verify.yml = the full verify loop.

Part of #71. Closes #72.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@keli-wen keli-wen changed the title Golden Harness: scripts/verify.sh + ruff/basedpyright/import-linter/pytest --cov chore(verify): add scripts/verify.sh + ruff/basedpyright/import-linter/pytest --cov harness Apr 26, 2026
@keli-wen keli-wen merged commit c99f00d into master Apr 26, 2026
2 checks passed
@keli-wen keli-wen deleted the chore/verify-loop branch April 26, 2026 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@keli-wen keli-wen

Labels

harness

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Golden Harness: verify loop with ruff + pytest + basedpyright + import-linter

1 participant

@keli-wen