Skip to content

Flightcheck: add WD-WF-CAT-001 Workday custom-workflow inventory checklist#138

Open
apurvabanka wants to merge 5 commits into
mainfrom
feature/flightcheck-workday-custom-workflow
Open

Flightcheck: add WD-WF-CAT-001 Workday custom-workflow inventory checklist#138
apurvabanka wants to merge 5 commits into
mainfrom
feature/flightcheck-workday-custom-workflow

Conversation

@apurvabanka

Copy link
Copy Markdown
Contributor

Summary

Adds WD-WF-CAT-001 — a MANUAL FlightCheck row that enumerates Workday
scenarios wired up in customer topics that fall outside the 17 OOTB
workflows the kit ships SOAP tests for. The check walks every agent topic
under workspace/agents/*/topics/*.mcs.yml for two wiring patterns,
diffs the discovered scenarios against the tenant-live OOTB catalog
(read from Dataverse msdyn_employeeselfservicetemplateconfigs where
ismanaged=true), and surfaces any unknowns as a bucketed MANUAL row
with a four-item operator checklist plus a gap-discovery loop-back.

A companion WD-WF-CAT-LINK trailer fires from inside _check_workflows
so admins reading a clean 17-row SOAP-test pass don't miss the manual
row below it.

Addresses 6 incidents rooted in this gap pattern (custom workflow
names not in the 41-workflow catalog), notably:

  • 760098889 — Multiday Leave application topic
  • 783902203 — conversation flow specifics

Parent feature: AB#7392268.
Action owner for MANUAL row triage: Workday admin.

Acceptance criteria

  • AC1 — Checklist enumerates: which custom Workday workflows are
    wired up, ISU account used, expected payload shape, test prompt
    (pinned by test_unknown_scenario_emits_manual_with_full_checklist)
  • AC2 — Linked from Test-WorkdayWorkflows output when an unknown
    workflow name is referenced in customer topics
    (pinned by test_trailer_emitted_when_unknowns_exist /
    test_trailer_absent_when_clean)
  • AC3 — Includes a "found a new pattern? log it here" loop back
    to the gap-discovery process
    (pinned by test_checklist_includes_gap_discovery_loopback)

Failure modes covered in the remediation text

Ticket failure mode Checklist item that addresses it
Problem 1 — Workflow Contract/Payload Mismatch (adaptive card submit fails because the request payload doesn't match the Workday SOAP/template contract) Item 2 (Payload shape) — directs operator to open the template config XML and confirm field names, reference types, and required vs. optional elements match the Workday WSDL. Names "the 'Workflow Contract/Payload Mismatch' failure mode" verbatim.
Problem 2 — Intermittent Workday Authentication Failure (expired/revoked OAuth token state on one or more connection references) Item 4 (Auth health) — cross-links to WD-CONN-* connection token-health output and gives the Power Platform Maker → Connections re-authenticate click-path.

What changed

File Purpose
solutions/ess-maker-skills/scripts/flightcheck/checks/workday.py New _check_custom_workflow_inventory (WD-WF-CAT-001) + _append_wd_wf_cat_link_trailer (WD-WF-CAT-LINK) + topic/workflow walkers + Dataverse catalog resolver. Wired into run_workday_checks after the SOAP tests.
solutions/ess-maker-skills/src/reference/ess-docs/flightcheck/validation-matrix.md Adds WD-WF-CAT-001 + WD-WF-CAT-LINK rows under §5 Workday Deep Validation with status-decision matrix prose.
tests/flightcheck/checks/test_workday_custom_inventory.py 15 tests covering every branch (gates, MANUAL, PASS, WARNING, both patterns, bucketing, caching, all three ACs, all 4 Dataverse-catalog resolver legs).

Design notes

Two detection patterns

  • Pattern A — topic calls
    WorkdaySystemGetCommonExecution with a scenarioName: value. Walker
    finds the dialog: ...WorkdaySystemGetCommonExecution line and looks
    backward up to 30 lines for the nearest scenarioName:.
  • Pattern B — topic uses kind: InvokeFlowAction with a flowId that
    resolves to a workflow.json whose connectionReferences bind to
    shared_workdaysoap. Conservative qualification — non-Workday flows
    (e.g. ServiceNow-bound) are ignored so the check doesn't emit false
    positives.

Catalog source — live Dataverse, no fallback

The OOTB scenario catalog is resolved per-invocation from the customer's
own tenant by querying msdyn_employeeselfservicetemplateconfigs and
treating every row with ismanaged=true as OOTB. This auto-detects every
scenario the installed Workday extension pack ships — no kit-side
curation, no stale JSON catalog to maintain. Result is cached on
runner._workday_ootb_catalog_cache so the cross-link trailer can
re-read without re-querying.

Status decision matrix (per AGENTS.md principles #1, #2, #3, #11)

Tenant state Status Why
Simplified Workday install (1 conn ref, OBO) SKIPPED ISU/scenario inventory doesn't apply (gated on runner._workday_package_flavor == "simplified" per principle #11)
workspace/agents/ missing SKIPPED Run /setup first
Zero Workday refs in any topic SKIPPED No Workday wired into this agent
Workday refs exist but no Dataverse token SKIPPED Cannot resolve catalog → can't claim PASS (principle #1)
Dataverse query errored (500, etc.) WARNING Surface error verbatim (principle #3)
Every discovered ref is a managed row PASSED
Any unknown / Pattern-B ref MANUAL 4-item checklist + AC3 loop-back; bucketed to ONE row per principle #7

apurvabanka_microsoft and others added 5 commits June 8, 2026 14:18
The AC3 acceptance criterion (Includes a `found a new pattern? log it
here` loop back to the gap-discovery process) was dropped in commit
0fb2383 when the JSON catalog was replaced with a live Dataverse query.
The rationale in the test comment was that kit-side PRs aren't the
catalog remediation anymore — but AC3 is broader than the catalog:

  * Detection-pattern gaps — a topic shape the walker doesn't catch
    (Pattern C and beyond)
  * OOTB-promotion feedback — scenarios customers routinely build
    custom that should ship in the extension pack
  * Checklist-completeness feedback — the 4-item checklist itself can
    grow as new failure modes emerge

Restore the loop-back paragraph at the bottom of _WD_WF_CAT_CHECKLIST,
reframed for the live-Dataverse world: still points operators at the
kit repo issues page with an actionable title format, but enumerates
the three feedback categories the loop exists to capture instead of
asking for catalog-JSON PRs that no longer exist.

Pin the restored text with test_checklist_includes_gap_discovery_loopback
so a future drive-by edit that re-strips it fails CI. The test file's
docstring already claimed to pin each AC explicitly; this commit makes
that claim true for AC3 alongside AC1/AC2.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant