Skip to content

feat: embed live-docs at deployment#207

Merged
JonJagger merged 14 commits into
mainfrom
embed-live-docs-at-deployment
May 5, 2026
Merged

feat: embed live-docs at deployment#207
JonJagger merged 14 commits into
mainfrom
embed-live-docs-at-deployment

Conversation

@JonJagger
Copy link
Copy Markdown
Contributor

@JonJagger JonJagger commented May 5, 2026

Resolve live-docs sections at deployment time instead of dynamically at URL click in page

Moves the live-docs Python logic from the server into the docs repo so that CI workflow URLs and Kosli event links are embedded directly into the CLI reference .md files when docs are deployed, rather than being fetched dynamically by the server on each URL click. Eliminates the server-side live-docs endpoints, removes the gunicorn worker fork caching problem, and makes pages faster. Any existing live-docs section in the .md file is stripped and regenerated unconditionally, so the script works regardless of what the CLI previously wrote into the .md files.

JonJagger and others added 12 commits May 5, 2026 12:00
Replace dynamic server-side URL resolution (app.kosli.com/api/v2/livedocs)
with static pre-generation during the docs deployment pipeline.

scripts/live_docs_modifiers_data.py (copied from the server) is the
source of truth for which commands have live-docs sections and what
CI systems they support.

scripts/resolve_livedocs.py strips any existing live-docs section from
each client_reference .md file and regenerates it with resolved static
URLs, using the same insertion point as the Go CLI doc generator. The
approach is idempotent: safe to run before or after the server endpoints
are removed.

URL resolution is currently hard-coded as placeholder values; real
GitHub/Kosli API resolution is to follow.

live-docs.sh is a temporary local test helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy live_docs.py, live_docs_cache.py, and live_docs_queries_data.py
from the server, adapting only what is needed to make them importable
outside the server context: replace seams/FastAPI/werkzeug/telemetry
imports with shims, fix module paths, and rewrite the match/case
statement in matching_event_query_string for Python 3.9 compatibility.

resolve_livedocs.py now delegates to the real resolution logic:
yaml_url calls the GitHub/GitLab and Kosli snapshot APIs to find the
exact workflow line; event_url finds the matching trail event.
Both fall back to backup URLs from _MODIFIERS on any failure.

Also guard against empty event URLs for commands whose flow exists
in _MODIFIERS but produces no resolvable trail event.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The server's caching was designed for a long-running process handling
concurrent requests. As a one-shot deployment script, all we need is
in-process memoization to avoid redundant HTTP calls within a single
run. Replace the threading/warmup machinery with functools.lru_cache
on the four fetch functions, and rename the file accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move yaml_url, event_url and their helpers into live_docs_fetch.py,
eliminating live_docs.py and live_docs_queries_data.py entirely.
The server shims (BadRequest, HTTPException, record_trace) are gone
with them. resolve_livedocs.py now imports directly from live_docs_fetch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Example is in cyber-dojo/snyk-scanning tests/test_rego_rules.sh
at line 153. GitHub only, no trail event.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The script previously ran inside the server's Docker container to
avoid setting up PYTHONPATH. Now that live_docs_modifiers_data.py
lives in scripts/, it runs directly with no setup required.

Update the usage comment in live_docs_modifiers_data.py accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restore the deleted usage comment in live_docs_modifiers_data.py
pointing to both audit and refresh scripts, now updated to run
directly from the repo root without Docker.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the 10 server-side /api/v2/livedocs/cli endpoint links with
statically resolved content generated by resolve_livedocs.py. Each
## Live Example section now shows a shell block with the command plus
an Accordion containing pretty-printed JSON fetched from the Kosli API
at deployment time. Adds live_docs_queries_data.py (_QUERIES and
_CLI_COMMANDS mappings) and extends live_docs_fetch.py with
fetch_cli_json().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A trailing shell comment on the export line confused copy-paste use.
Moving it to a standalone line above makes the block paste-safe.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Aligns the script name and workflow step name with what the script
actually does: adding live-docs sections to CLI reference pages.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copies the relevant modifier and query tests from the server, adapting
them to use monkeypatch instead of the seam-based stub mechanism.
Adds generate_live_docs_test_data.py to regenerate the JSON stub files
from app.kosli.com whenever _MODIFIERS changes. Run with: pytest tests/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moves the local dev helper into scripts/ alongside the other live-docs scripts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@JonJagger JonJagger requested a review from a team as a code owner May 5, 2026 12:36
@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented May 5, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
kosli 🟢 Ready View Preview May 5, 2026, 12:37 PM

@JonJagger JonJagger changed the title Embed live docs at deployment feat: embed live-docs at deployment May 5, 2026
Copy link
Copy Markdown
Contributor

@pbeckham pbeckham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. I love that we are moving this as a docs concern and we can remove the logic from the server application

Comment thread tests/live_docs_helpers.py
Comment thread .github/workflows/update-cli-docs.yml
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@tooky
Copy link
Copy Markdown

tooky commented May 5, 2026

@JonJagger before we remove anything from the server I want to make sure that as an acceptance test the links on kosli.com/how-it-works continue to work.

These are the live docs links that are used during sales demos.

@tooky Roger that.

@tooky
Copy link
Copy Markdown

tooky commented May 5, 2026

I continue to wonder if finding the live docs links couldn't be a javascript library that was embeddable in docs or www.

@tooky This is the simplest first step. It could be there is mileage in that option. Claude claimed there would be potential cross-site scripting problems.

Comment thread .github/workflows/update-cli-docs.yml Outdated
Tests guard the docs repo's own Python code so should run on every PR
to this repo, not only when a CLI release triggers update-cli-docs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread scripts/add_livedocs.py
Comment thread client_reference/kosli_diff_snapshots.md
@JonJagger JonJagger merged commit fb83004 into main May 5, 2026
5 checks passed
@JonJagger JonJagger deleted the embed-live-docs-at-deployment branch May 5, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants