chore(deps): update dependency promptfoo to v0.121.11

[renovate]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
promptfoo	0.121.2 → 0.121.11

---

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

---

Release Notes

promptfoo/promptfoo (promptfoo)

v0.121.11

Compare Source

Features

• quiverai: add Arrow 1.1 models, vectorize endpoint, and GPT Image-2 pipeline (#​9139) (ce2c62d)

Bug Fixes

• redteam: handle MCP target prompt materialization (#​9149) (a050023)
• redteam: keep fresh target type state in sync (#​9145) (770130f)
• redteam: show target type selector immediately (#​9141) (c143fa6)

v0.121.10

Compare Source

Features

• app: humanize per-cell latency in eval results table (#​8998) (509aa54)
• bedrock: add converse mcp support (#​9134) (c1fe183)
• claude-agent-sdk: bump SDK to 0.2.120 and expose new options (#​8946) (350a24c)
• cli: add eval filter range (#​8895) (f049dd3)
• examples: add integration-inspect-osworld wrapper (#​8932) (5d3279a)
• openapi: generate server spec from route DTOs (#​8928) (c92102b)
• output: add junit xml reports (#​9073) (ec8149c)
• providers: add Atlas Cloud provider (#​8980) (90da094)
• providers: add first-class n parameter support for OpenAI chat completions (#​8914) (6e120b1)
• providers: expose OpenAI prompt cache controls (#​9093) (9ec9c1f)
• providers: update mistral support (#​9018) (130f59f)
• xai: refresh provider support (#​9033) (c2c7abf)

Bug Fixes

• api: add DTO validation for core server routes (#​8922) (87f637f)
• api: correct user-visible response regressions (#​8976) (74e7f95)
• api: harden node package contracts (#​9055) (be44d4e)
• api: make modelAudit safeRespond fire-safe under parse failures (#​8977) (524c5e6)
• api: preserve error details and stack traces in shared helpers (#​8975) (ed0b67b)
• api: validate eval route DTOs (#​8924) (84d5c8c)
• api: validate media, blob, user, and trace DTOs (#​8923) (9a313f5)
• api: validate model audit route DTOs (#​8927) (684ae81)
• api: validate provider route DTOs (#​8925) (b761730)
• app: make update banner dismiss button clickable (#​8942) (79a2864)
• assertions: avoid slow XML candidate matching (#​8941) (df029d2)
• assertions: invert not-llm-rubric results (#​8986) (c45831b)
• assertions: normalize shell command arrays in trajectories (#​9060) (a39fac1)
• assertions: support indexed XML paths (#​8970) (bd3969f)
• cache: hash anthropic cache keys (#​8642) (d1f1e5b)
• cache: hash bedrock cache keys (#​8644) (d65fa5e)
• cache: hash vertex and foundry keys (#​8640) (c7b2486)
• cache: stabilize Anthropic cache hashes across runs (#​9124) (8e72bd1)
• cli: avoid hard exits in recoverable flows (#​9118) (d00bddd)
• cli: explain better-sqlite3 ABI mismatches (#​9000) (fc7573a)
• cli: fail closed on modelaudit signal exits (#​9014) (2f29f65)
• code-scan: honor enable-fork-prs (#​8938) (517ec9d)
• codex: isolate default providers by credential (#​8633) (dddd275)
• config: make resolution errors reusable (#​9008) (6b8d3db)
• deps: update dependency ai to ^6.0.168 (#​9007) (95adf3c)
• deps: update dependency fast-xml-parser to ^5.7.1 (#​9010) (72140be)
• deps: update dependency undici to ^7.25.0 (#​9017) (5be6015)
• deps: update openai packages (#​9092) (efe2cd0)
• deps: update opentelemetry (#​9103) (9106ef0)
• eval: delete traces with eval records (#​8973) (a395551)
• eval: harden OpenAI media blob exports (#​8876) (9d4948b)
• eval: honor prompt suggestion counts (#​9105) (3f98e0d)
• eval: honor sharing disable for blob uploads (#​8940) (77fa02a)
• eval: keep package APIs in library mode (#​9111) (3bc3a22)
• eval: preserve filter range on resume (#​8960) (cd16d65)
• eval: stop HuggingFace empty-page loops (#​8939) (56b3c9f)
• google: drop late live messages after resolution (#​9138) (f21272d)
• google: use AIStudioChatProvider for default grading providers (#​9108) (42ab9dd)
• model-audit: fail closed across report flows (#​9050) (f1d0a6c)
• output: strip nested response metadata (#​9038) (ba3ecaa)
• output: strip nested test case metadata (#​9039) (1179413)
• providers: claude-agent-sdk returns main agent result, not first sub-agent result (#​9056) (a694552)
• providers: drop retired groq reasoning model (#​9025) (937e1c9)
• providers: expose OpenAI prompt cache controls (#​9093) (7fbbdca)
• providers: honor disabled tools in model providers (#​9045) (cfdc9f9)
• providers: honor passthrough for OpenAI embeddings (#​9107) (3eeccdb)
• providers: improve openai pricing accuracy (#​9049) (6cb5be9)
• providers: refresh WatsonX model metadata (#​9015) (355fbaf)
• providers: separate OpenCode fork cache keys (#​9102) (9851681)
• providers: structured 429 classification in Azure assistants (#​8896) (d2c64d2)
• providers: support newer bedrock chat models (#​9024) (f3be26f)
• providers: unify coding agent working dirs (#​9043) (bfcac5b)
• providers: use agent_reference for Azure AI Foundry agents (#​8987) (989a17e)
• redteam: honor coding-agent artifact vars (#​9133) (0187a8c)
• redteam: improve SQL injection grading to distinguish from RBAC issues (#​7284) (2665cac)
• redteam: keep results toolbar visible (#​9125) (1a986ee)
• redteam: make email validation failures recoverable (#​9012) (698b9ae)
• redteam: refine report table layout (#​9127) (3e5371f)
• redteam: validate redteam route DTOs (#​8926) (1b8c4fb)
• server: honor remote generation disable for hosted helpers (#​8967) (61baa48)
• test: prevent test-hygiene race with parallel fixture writers (#​8948) (549e2b7)
• tracing: skip orphan otlp trace rows (#​9132) (a303d36)

Performance Improvements

• providers/http: parallelize createHttpsAgent file reads (#​8956) (108c80b)

Reverts

• feat(providers): add first-class n parameter support for OpenAI chat completions (#​9113) (543b5de)

v0.121.9

Compare Source

Features

• providers: add gpt-5.5 model support (#​8884) (8c5dc92)

Bug Fixes

• cli: align command-line reference with CLI (#​8900) (c4ce0d4)
• deps: update openai packages (544d1c6)
• telemetry: mirror person properties on every event (#​8902) (95ad451)
• prevent report table header icon overlap (#​8880) (ad87060)
• trailing slashes on internal routes, Docusaurus Link for internal hrefs, and test hygiene (#​8903) (0828ff7)

v0.121.8

Compare Source

Features

• claude-agent-sdk: bump to 0.2.116 and add title option (#​8858) (9bca53a)
• providers: add GPT-5.5 OpenAI support (#​8873) (6488623)

Bug Fixes

• cache: hash azure assistant keys (#​8646) (7264a73)
• deps: update dependency uuid to v14 [security] (#​8864) (3fa2956)
• providers: canonicalize Azure assistant cache keys (#​8883) (c80f431)
• redteam: support remote input materialization (#​8863) (d89382e)

v0.121.7

Compare Source

Features

• add promptfoo codex skills (#​8790) (d794c0d)
• harmbench plugin filter by category (#​8827) (73533f4)
• openai: add gpt-image-2 support (#​8835) (ac5ff4c)
• providers: add google:embedding: AI Studio embeddings (#​8825) (cb67352)
• redteam: add DOCX input wrapper generation (#​8506) (5bba957)
• webui: virtualize shared data tables (#​8537) (ae756c9)

Bug Fixes

• assertions: support Vercel AI SDK tool-call spans (#​8800) (642e925)
• auth: log file auth refresh lifecycle (#​8833) (0da6c6a)
• cache: hash mistral cache keys (#​8643) (869f98d)
• cache: hash moderation cache inputs (#​8641) (2c36415)
• cache: hash replicate cache keys (#​8647) (495e09e)
• cache: hash watsonx cache keys (#​8649) (1461b6f)
• cache: isolate fetch cache by request identity (e0da9dd)
• deps: update DOMPurify to 3.4.1 (#​8844) (29045ed)
• deps: update opentelemetry (#​8851) (4370216)
• eval: scope result ratings to eval (33be399)
• eval: show rendered assertion values instead of raw templates in UI (#​7976) (dde71c9)
• providers: convert OpenCode permissions to v2 PermissionRuleset (#​8808) (335149a)
• providers: honor OpenRouter config overrides and refresh model references (#​8804) (432d1a8)
• redteam: support named custom policy configs (#​8836) (b035d66)
• server: handle large eval table payloads (#​8745) (92d5929)
• server: redact route debug bodies (#​8639) (d160e4c)
• vertex: align VertexEmbeddingProvider constructor with ProviderOptions (#​8843) (f29cfef)

v0.121.5

Compare Source

Features

• providers: add Abliteration provider (b29fa9a)
• providers: add OpenAI Codex app-server provider (#​8578) (a403dd1)
• providers: let anthropic:messages auth via Claude Code session (#​8692) (642af70)
• webui: add system theme selector (#​8538) (2ed5e5b)

Bug Fixes

• app: chart toggle visibility (#​8439) (f9d448d)
• assertions: avoid jsdom in HTML assertions (979aee4)
• correct rouge assertion reason messages when inverse: true (#​8682) (7d64d8d)
• deps: update anthropic packages (#​8670) (0a4a7e0)
• deps: update dependency openai to ^6.34.0 (#​8701) (73f6ba1)
• deps: update dependency parse5 to v8 (#​8710) (759391d)
• deps: update openai packages (#​8672) (1ae735c)
• providers: close stdin on exec provider to prevent child process hanging (#​8686) (9cd7709)
• providers: resolve settings file path for claude-agent-sdk (#​8606) (d575bd1)
• providers: send reasoning config for Azure responses deployments (#​8255) (67437f3)
• redteam: throw immediately on non-OK HTTP responses in redteam discover (#​8677) (770a557)
• server: prevent circular JSON when Bedrock/Anthropic provider is used as llm-rubric judge from web UI (#​8688) (f26e54b)

v0.121.4

Compare Source

Features

• allow per-test opt-out of defaultTest assertions (5e5959e)
• codex: expand Codex SDK eval controls and docs (#​8433) (80c3f7f)
• eval: group serial grading by provider (#​8509) (d289602)
• examples: add traced openai agents python sdk example (#​8354) (6870717)
• http: support structured multipart requests (#​8533) (5bac47c)
• japan fiea plugin (#​8316) (f330ab3)
• matchers: expose grading provider metadata in GradingResult (#​8330) (03cbac6)
• openai: enable chatgpt login via codex (#​8327) (5a9cb96)
• providers: add Gemma 4 provider support (#​8454) (b0667ed)
• providers: add missing Anthropic SDK features and fix apiKeyRequired bug (#​8351) (3060847)
• providers: expand OpenClaw support (#​8589) (93f29ec)
• providers: support Codex local images and harden SDK (929790b)
• providers: support multimodal openai agents input (#​8397) (4065844)
• redteam: add coding-agent redteam plugins (85a0cfe)
• redteam: add FDA medical plugins (#​8456) (30e4ac3)
• redteam: add next coding-agent plugins (5ab3ba4)
• redteam: add teen safety plugins (#​8308) (61aa057)
• redteam: enforce max chars per message (#​8428) (9af7b7d)
• redteam: use Codex login for default text graders (#​8493) (9a6b61b)

Bug Fixes

• app: clarify attack success rate label (#​8386) (d088eac)
• app: clarify attack success rate label (#​8387) (7482eff)
• app: keep select-all checkbox visible (#​8549) (02f9064)
• assertions: apply weights to named scores in assertion results (#​8206) (01da019)
• assertions: handle inverse flag in finish-reason handler (#​8556) (7155f28)
• assertions: normalize javascript function assertion results (#​8377) (a7fafba)
• assertions: normalize negated script assertions (#​8485) (fe62243)
• assertions: preserve nested metric weights (#​8558) (f3c173d)
• assertions: resolve search-rubric web providers (#​8557) (af751aa)
• assertions: skip simulated-user as implicit grader (#​8429) (9bc5fdf)
• auth: honour org flag in login command (#​8430) (8bdb15d)
• bedrock: preserve zero-valued inference config (#​8272) (1cd397e)
• browser: make stealth plugin install explicit (#​8434) (aa72a34)
• cache: cache repeated evals by repeat index (#​8480) (fbd59a6)
• cli: enforce exact Node engine range (#​8380) (1f8e3b4)
• cli: serialize source-map support initialization (#​8401) (d4a9d4a)
• code-scan: avoid npm before env for MCP npx (#​8515) (7d2eacd)
• code-scan: retry scanner MCP request timeouts (#​8545) (4312797)
• deps: pin claude-agent-sdk lockfile to 0.2.87 (#​8400) (cbfb91a)
• deps: update dependency @​anthropic-ai/sdk to ^0.82.0 (#​8510) (cbc96df)
• deps: update dependency @​modelcontextprotocol/sdk to ^1.28.0 (416f3cd)
• deps: update dependency @​modelcontextprotocol/sdk to ^1.28.0 (5a8a732)
• deps: update dependency @​modelcontextprotocol/sdk to ^1.29.0 (#​8501) (8821383)
• deps: update dependency @​modelcontextprotocol/sdk to ^1.29.0 (#​8502) (84db2aa)
• deps: update dependency @​openai/agents to ^0.8.1 (#​8371) (3f2234c)
• deps: update dependency @​openai/agents to ^0.8.1 (#​8372) (aecdb11)
• deps: update dependency @​openai/agents to ^0.8.2 (#​8500) (2b2ce6c)
• deps: update dependency @​opentelemetry/exporter-trace-otlp-http to ^0.214.0 (#​8379) (462f1e5)
• deps: update dependency ai to ^6.0.138 (#​8403) (855e6b3)
• deps: update dependency ai to ^6.0.138 (#​8410) (4aa56cf)
• deps: update dependency openai to ^6.33.0 (#​8388) (19e04ce)
• deps: update dependency proxy-agent to v8 (#​8503) (52953a6)
• deps: update dependency undici to ^7.24.5 (#​8411) (3d8a24d)
• deps: update example dependencies (#​8367) (977a656)
• deps: update IBM Cloud SDK Core (#​8584) (07aaf97)
• deps: update openai packages (#​8368) (e5b842a)
• elevenlabs: Disable implicit retries for POST calls (#​8541) (34f819a)
• elevenlabs: preserve explicit zero retries (#​8358) (493ef88)
• eval: fix bugs and performance issues in evaluator orchestrator (#​8481) (36a2788)
• G-Eval wrong scoring for negative criteria (#​8259) (e7bba6e)
• handle generated redteam exports during eval (#​8301) (ef3f67f)
• harden object guards in comparison checks (#​8408) (6e81cc9)
• image should preserve aspect ratio in results table (#​8279) (e96d630)
• mcp: preserve falsy values and bound truncation (#​8423) (384249e)
• mcp: stabilize withTimeout timeout handling (#​8399) (df7d8a9)
• openai: Avoid invalid realtime zero output tokens (#​8543) (9cde9b4)
• openai: preserve agents zero token usage (#​8283) (6354f4e)
• openai: preserve assistant temperature zero (#​8271) (311c0a4)
• parse rate-limit reset headers (#​8341) (985ab28)
• preserve configured plugin config in strategy preview (#​8326) (e20458e)
• prevent SSTI in conversation-relevance assertion (#​8258) (49cbe41)
• prompts: skip recursive rendering for undefined vars (#​8394) (8f14e8d)
• providers: align omitDefaults handling across OpenAI and Azure (#​8332) (edb5db6)
• providers: deduplicate token refresh retries (#​8421) (b78ea03)
• providers: include azure moderation config in cache key (#​8349) (c528c5a)
• providers: preserve explicit zero values in Mistral, AI21, and Azure Assistant config (#​8555) (7612368)
• providers: prevent max_tokens leaking via passthrough in multi-provider eval (#​8449) (b07ca39)
• providers: support systemInstruction for Vertex Claude models (#​8522) (ff1cf68)
• providers: support transitive TypeScript provider imports (#​8445) (1bb0f39)
• providers: use apiKeyEnvar in error messages across OpenAI providers (#​8180) (7008087)
• providers: use rendered prompt in simulated user (#​8363) (1f1fa86)
• redteam: avoid nested buttons in transform dialog (#​8520) (50c65b4)
• redteam: clarify remote generation guidance (#​8252) (45870ab)
• redteam: fix --env-file arg parsing and test flakiness (#​8420) (6a920c5)
• redteam: guard missing cross-session leak metadata (#​8238) (9924bae)
• redteam: honour generate concurrency config (#​8463) (e626ea9)
• redteam: normalize code editor imports (#​8306) (057f9e0)
• redteam: pass full conversation history to crescendo grader (#​8239) (01619a8)
• redteam: preserve goat zero max turns (#​8294) (3077acf)
• redteam: preserve voice crescendo zero max turns (#​8310) (5d38b28)
• redteam: preserve zero prompt injection sample (#​8291) (16b3221)
• redteam: preserve zero-valued crescendo config (#​8273) (9ec66b5)
• redteam: preserve zero-valued custom config (#​8274) (a76058f)
• redteam: preserve zero-valued voice crescendo config (#​8270) (7d36044)
• redteam: remove dead stores and preserve metadata (#​8407) (28d3383)
• redteam: skip unblocking feature checks when disabled (#​8266) (c6bdf39)
• redteam: support policy multi-input generation (#​8320) (be14251)
• refresh prompt listings after prompt metadata updates (#​8120) (85694e9)
• replace undeclared tiny-invariant imports in app (#​8309) ([f85704f](https://redirect.github.com/prompt

✂ Note

PR body was truncated to here.

---

Configuration

📅 Schedule: (in timezone Europe/London)

• Branch creation
  • "before 10am on friday"
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[github-actions]

Caution

[High Risk] New API server will be unreachable from customer networks because it only serves port 9090 while the public-facing security group exposes only 443

The new production API server on 540044833068.eu-west-2.ec2-instance.i-025efedc46bef3be1 is being launched with user_data that starts only a health server on port 9090, and the plan registers that instance into the api-health-terraform-example target group on 9090. But the attached customer-facing security group sg-03cf38efd953aa056 only allows inbound 443 from whitelisted client CIDRs, while the internal security group sg-089e5107637083db5 allows 9090 only from 10.0.0.0/8.

When this change deploys, internal health checks can succeed on 9090, but external clients using the associated public endpoint will not be able to connect because nothing in the change exposes the service on 443 and the security groups do not permit customer traffic to 9090. The result is a production API endpoint that appears provisioned and healthy internally but is unavailable to real users.
_{View reasoning tree here.}

Signals

Routine → Multiple AWS compute and supporting resources are showing unusual and infrequent change activity at 1 event/week for the last 3 months and 2 events/week for the last 3 months, which is infrequent compared to typical patterns.
Policies → Multiple infrastructure resources showing unusual policy violations that may need review: an S3 bucket does not have server-side encryption configured and is missing required tags, while a security group allows SSH on port 22 from anywhere (0.0.0.0/0).

_{Additional Change Details:} Items 131 Edges 325 model|risks_v6 ✨Encryption Key State Risk ✨KMS Key Creation

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Auto-blocked: Routine score (-5) is below minimum (-1)

---

📊 Signals Summary

Routine 🔴 -5

---

🔥 Risks Summary

High 0 · Medium 0 · Low 0

---

💥 Blast Radius

Items 23 · Edges 75

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Auto-blocked: Policy signal (-3) is below threshold (-2); Routine score (-5) is below minimum (-1)

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 0 · Medium 0 · Low 0

---

💥 Blast Radius

Items 5 · Edges 20

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Found 2 high risks requiring review

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 2 · Medium 0 · Low 0

---

💥 Blast Radius

Items 107 · Edges 219

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Found 1 high risk requiring review

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 1 · Medium 1 · Low 0

---

💥 Blast Radius

Items 63 · Edges 135

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Found 2 high risks requiring review

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 2 · Medium 0 · Low 0

---

💥 Blast Radius

Items 93 · Edges 217

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Auto-blocked: Policy signal (-3) is below threshold (-2); Routine score (-5) is below minimum (-1)

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 0 · Medium 0 · Low 0

---

💥 Blast Radius

Items 79 · Edges 192

---

View full analysis in Overmind ↗