[delight] UX Analysis Report – 2026-04-23

[github-actions[bot]]

Executive Summary

Today's analysis covered:

• 2 documentation files — engines.md (reference) and web-search.md (guide)
• 2 workflow message configurations — architecture-guardian.md and ci-doctor.md
• 1 validation file — permissions_validation.go

Overall Quality: Mostly professional — one high-priority gap in documentation, one medium-priority message refinement.

Key Finding: The web-search.md guide covers only Tavily MCP integration and omits Codex's native opt-in tools.web-search capability, creating a dead-end for Codex users who follow the cross-reference from engines.md.

---

Quality Highlights ✅

Example 1: engines.md — Model Reference Documentation

• File: docs/src/content/docs/reference/engines.md
• What works well: Every feature and timeout knob is surfaced in a summary table and explained in prose with copy-paste YAML. The "Which engine should I choose?" decision section respects reader expertise while giving concrete recommendations. Per-engine sub-sections prevent information scattering.
• Rating: ✅ Professional

Example 2: permissions_validation.go — Actionable Error Messages

• File: pkg/workflow/permissions_validation.go
• What works well: formatMissingPermissionsMessage presents two fix paths (add permissions vs. reduce toolsets), each with ready-to-paste YAML. ValidateIncludedPermissions names the exact edit location and includes a docs URL. Users can act on the error without searching documentation.
• Rating: ✅ Professional

Example 3: ci-doctor.md — Coherent Persona in Messages

• File: .github/workflows/ci-doctor.md
• What works well: The medical theme is applied consistently across every message — "reporting for duty", "examining the patient", "diagnosis", "Prescription issued 💊". The run-failure message ("Doctor needs assistance") implies an escalation action rather than just stating a failure. Persona coherence is excellent.
• Rating: ✅ Professional

---

Improvement Opportunities 💡

High Priority

Opportunity 1: web-search.md — Missing Native Codex Web-Search Guidance

• File: docs/src/content/docs/guides/web-search.md
• Current State: The guide covers only the Tavily MCP server approach. The introduction acknowledges alternative providers exist ("While alternatives exist (Exa, SerpAPI, Brave Search), this guide focuses on Tavily") but treats provider selection as the only dimension of choice.
• Issue: engines.md feature table notes tools.web-search: ✅ (opt-in) for Codex and directs readers here with "See Using Web Search". A Codex user following that link finds zero guidance for their engine. The native vs. MCP distinction — which drives how web-search is configured — is never explained.
• User Impact: Enterprise users running Codex workflows must search the codebase or community for configuration instructions not present in the official guide. The cross-reference in engines.md actively misdirects them.
• Suggested Change: Add a brief "Native Web Search (Codex)" section before the Tavily section, explaining the tools.web-search: frontmatter entry for Codex and linking back to the engines table. Add a short introductory paragraph clarifying the two approaches (native vs. MCP-based).
• Design Principle: Efficiency and Productivity — eliminate the dead-end for Codex users

Medium Priority

Opportunity 2: architecture-guardian.md — run-success Message Redundancy

• File: .github/workflows/architecture-guardian.md
• Current State (line ~45):

  run-success: "✅ Architecture scan complete! [{workflow_name}]({run_url}) has reviewed code structure. Report delivered! 📋"
  

• Issue: "Architecture scan complete" and "has reviewed code structure" express the same idea twice in 12 words. The trailing 📋 is decorative rather than informative.
• User Impact: Minor — the redundancy adds length without adding meaning, slightly diluting the signal-to-noise ratio in notification feeds.
• Suggested Change:

  run-success: "✅ Architecture scan complete — [{workflow_name}]({run_url}) report delivered."
  

• Design Principle: Clarity and Precision — every word should add distinct information

---

Files Reviewed

Documentation

• docs/src/content/docs/reference/engines.md — Rating: ✅ Professional
• docs/src/content/docs/guides/web-search.md — Rating: ❌ Needs Significant Work

Workflow Messages

• .github/workflows/architecture-guardian.md — Rating: ⚠️ Needs Minor Work
• .github/workflows/ci-doctor.md — Rating: ✅ Professional

Validation Code

• pkg/workflow/permissions_validation.go — Rating: ✅ Professional

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 1
  • ❌ Needs Significant Work: 1

---

🎯 Actionable Tasks

Task 1: Fill Native Codex Web-Search Gap — web-search.md

File to Modify: docs/src/content/docs/guides/web-search.md

Current Experience

The entire guide assumes the reader will use an MCP server for web search. The intro sentence "While alternatives exist (Exa, SerpAPI, Brave Search), this guide focuses on Tavily" positions provider choice as the key variable, but never surfaces the more fundamental distinction: Codex has a built-in web-search tool that requires no MCP server, while all other engines require an MCP server. A Codex user who follows the cross-reference from engines.md finds nothing applicable to their setup.

Quality Issue

Design Principle: Efficiency and Productivity

When a cross-reference links to a guide, users expect that guide to answer their question regardless of which engine they chose. The omission forces a secondary search, breaking the "direct path to outcomes" principle.

Proposed Improvement

Add an introductory paragraph and a "Native Web Search (Codex)" section before the Tavily section:

Before (current opening):

This guide shows how to add web search to workflows using the Tavily Model Context Protocol (MCP) server, an AI-optimized search provider designed for LLM applications. While alternatives exist (Exa, SerpAPI, Brave Search), this guide focuses on Tavily configuration.

After:

GitHub Agentic Workflows support web search in two ways: as a **native tool** built into the Codex engine, or via a **third-party MCP server** for all other engines. Choose the approach that matches your engine.

| Approach | Engines | Configuration |
|----------|---------|---------------|
| Native tool | Codex | `tools: web-search:` in frontmatter |
| MCP server | Copilot, Claude, Gemini, Crush, OpenCode | Add an MCP server (e.g., Tavily) |

## Native Web Search (Codex)

Codex includes web search as an opt-in built-in tool — no MCP server or external API key required:

```aw
---
on: issues
engine: codex
tools:
  web-search:
---

Search for recent information about: $\{\{ github.event.issue.title }}

No additional setup is needed. Web-search results are returned directly to the agent.

MCP-Based Web Search (All Other Engines)

For Copilot, Claude, Gemini, and other engines, web search is provided by a third-party MCP server. This guide covers [Tavily]((tavily.com/redacted), an AI-optimized search provider, but alternatives such as Exa and Brave Search follow the same MCP pattern.

**Why This Matters**
- **User Impact**: Codex users get direct, actionable guidance instead of a dead-end
- **Quality Factor**: Completeness and accuracy — the guide now matches the feature table in `engines.md`
- **Frequency**: Every Codex user looking for web-search setup follows this cross-reference

**Success Criteria**
- [ ] Changes made to `docs/src/content/docs/guides/web-search.md` only
- [ ] Codex native `tools.web-search:` example is present and working
- [ ] Introductory comparison table clarifies native vs. MCP distinction
- [ ] The Tavily MCP section is preserved and reachable from the page

**Scope Constraint**
- **Single file only**: `docs/src/content/docs/guides/web-search.md`
- No changes to `engines.md` or other files required

---

#### Task 2: Tighten `run-success` Message — `architecture-guardian.md`

**File to Modify**: `.github/workflows/architecture-guardian.md`

**Current Experience**

The `run-success` message reads:

"✅ Architecture scan complete! {workflow_name} has reviewed code structure. Report delivered! 📋"

Two phrases — "Architecture scan complete" and "has reviewed code structure" — both describe the same completed action. The trailing `📋` adds visual noise without conveying status or directing action.

**Quality Issue**

**Design Principle**: Clarity and Precision

Enterprise notification feeds surface many automated messages. Concise, non-redundant messages reduce cognitive load and improve signal quality. Each word should contribute distinct information.

**Proposed Improvement**

**Before:**
```yaml
run-success: "✅ Architecture scan complete! [{workflow_name}]({run_url}) has reviewed code structure. Report delivered! 📋"

After:

run-success: "✅ Architecture scan complete — [{workflow_name}]({run_url}) report delivered."

The em dash replaces the repeated subject–verb ("has reviewed code structure") with a flowing summary. The decorative 📋 is removed; the ✅ at the front already signals success.

Why This Matters

• User Impact: Shorter, sharper notification with no lost information
• Quality Factor: Clarity — removes redundant clause and decorative punctuation
• Frequency: Fires on every successful weekday architecture scan (~5×/week)

Success Criteria

• Changes made to .github/workflows/architecture-guardian.md only
• run-success message is ≤ 80 characters (excluding link markup)
• No duplicate meaning; ✅ prefix retained for status signalling

Scope Constraint

• Single file only: .github/workflows/architecture-guardian.md
• Only the run-success value in the messages: block needs updating

---

References:

• §24834413890

📊 User experience analysis by Delight · ● 936K · ◷

• expires on Apr 26, 2026, 12:21 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #28259.