FEAT: Block GUI sends when target doesn't support the modality#1692
Merged
romanlutz merged 18 commits intomicrosoft:mainfrom May 8, 2026
Merged
FEAT: Block GUI sends when target doesn't support the modality#1692romanlutz merged 18 commits intomicrosoft:mainfrom
romanlutz merged 18 commits intomicrosoft:mainfrom
Conversation
Expose supported_input_data_types from TargetCapabilities through the backend DTO and mapper so the frontend knows which input modalities a target accepts. ChatInputArea now shows a non-blocking warning when the user attaches a file whose type (image, audio, video, file) is not in the target's supported input data types. Changes: - Add supported_input_data_types field to TargetInstance backend model - Flatten input_modalities in target_object_to_instance mapper - Add supported_input_data_types to frontend TargetInstance type - Add file->binary_path mapping in converterTypes - Derive unsupported attachment types in ChatInputArea and show warning - Add backend mapper tests and frontend component tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change the unsupported modality warning from non-blocking to blocking: - Disable the send button when unsupported attachment types are present - Guard handleSend against unsupported types - Update warning text to instruct user to remove the attachment - Update tests to verify send is disabled Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend modality validation to also check converter output data types, not just raw attachment types. When a converter transforms text into an unsupported type (e.g., text-to-image on a text-only target), the send button is now disabled with a warning. Changes: - Add outputDataType to PieceConversion interface - Pass output type from ConverterPanel through ConverterPreview - Pass converterOutputDataTypes from ChatWindow to ChatInputArea - Validate converter outputs against target supported_input_data_types - Update all affected tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Increase textInput and convertedTextarea maxHeight from 96px/80px to 30vh each, allowing each to use up to ~30% of viewport height - Remove JS-side 96px cap on auto-resize so CSS maxHeight controls it - Add matching webkit scrollbar styling to convertedTextarea - Use alignItems flex-start on both rows so they grow naturally Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When a conversion is active, both textareas now cap at 15vh each (half of the 30vh total) via inline style overrides. Without a conversion, the original textarea gets the full 30vh. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Wrap both original and converted textareas in a single scrollable container (textScrollArea) with max-height 60vh and resize: vertical - Textareas grow naturally inside it with overflow hidden (no per-textarea scrollbars), the outer container handles scrolling - Move clear-conversion button to columnRight below send with tooltip - Both textareas share space equally within the scrollable area Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove overflow:hidden from inputWrapper that was blocking scroll - Add clearConversionButton style with circular border (matches send button shape but uses subtle neutral colors for discoverability) - Increase columnRight gap to spacingVerticalS for more breathing room between info icon and send button - Add marginRight to convertedBadge matching originalBadge for consistent indentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Textarea sizing: - Each textarea auto-resizes via JS (height = scrollHeight) and is capped by CSS max-height with overflowY: auto for scrolling - Original textarea: 60vh when solo, 30vh when conversion active (via textInputShared class toggled with mergeClasses) - Converted textarea: 30vh with matching scrollbar styling - Both use identical webkit scrollbar styles - Converted textarea also auto-resizes via its own ref + useLayoutEffect Button styling: - Attach and converter buttons get round border (iconButton) - Clear conversion button has distinct circular style with subtle border - More gap between attach/converter buttons (spacingVerticalS) - More gap in columnRight between info/send/clear (spacingVerticalS) Badge alignment: - convertedBadge now has marginRight matching originalBadge Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the ad-hoc supports_multi_turn and supported_input_data_types top-level fields on TargetInstance with a single nested TargetCapabilitiesInfo DTO that mirrors the full TargetCapabilities domain dataclass: - 6 boolean capability flags (supports_multi_turn, supports_json_*, etc.) - supported_input_data_types: flattened sorted unique input data types - supported_output_data_types: flattened sorted unique output data types Filtering of capability-named identifier params in the mapper now covers the full CapabilityName enum so future flags don't accidentally leak into target_specific_params. Frontend: TargetInstance.capabilities is optional (consumer code uses ?. ) so missing/old responses fail open. All consumers updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
romanlutz
added a commit
to romanlutz/PyRIT
that referenced
this pull request
May 7, 2026
…ndow The frontend was reading the deprecated top-level `supports_multi_turn` field via `capabilities.supports_multi_turn ?? supports_multi_turn`. Since the backend now always populates `capabilities`, the fallback is dead code and creates a two-source-of-truth issue. The legacy field will be removed in microsoft#1692; tightening the read sites here keeps that follow-up cleaner. - ChatInputArea single-turn warning: read only from `capabilities`. - ChatWindow single-turn limit and `isSingleTurn` flag: same. - Tests: replace top-level `supports_multi_turn` fixtures with `capabilities` populated via a small `buildCapabilities` helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve overlapping `TargetCapabilitiesInfo` introduction between this branch (modality blocking refactor) and main's PR microsoft#1691 (config table capability columns): - Use main's field naming: `supported_input_modalities` and `supported_output_modalities` (not `_data_types`) - Drop the now-removed top-level `supports_multi_turn` from `TargetInstance` and its legacy-compat test - Keep `capabilities` required (cleaner contract; main had it optional) - Combine both filter additions in target_mappers.py: filter all `CapabilityName` enum values plus `target_configuration` - Adopt main's `buildCapabilities` test helper everywhere; drop my duplicate `makeCaps` helper - Update `ChatInputArea.tsx` modality check to use the renamed `supported_input_modalities` field Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The TS type ConverterCatalogEntry declares supported_input_types and supported_output_types as required string[]. The backend Pydantic model sets default_factory=list so the field is always present, and the PromptConverter base class enforces non-empty SUPPORTED_INPUT_TYPES / SUPPORTED_OUTPUT_TYPES at class-definition time via __init_subclass__. The "?? []" guards were dead code; the "?? text" fallback is kept only at the call site where selectedConverter itself can be undefined. Removed the "handles converter with empty supported types" test since empty types are impossible per backend invariant - the test was asserting the dead defensive code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the unhelpful "Unsupported modality helper" section banner with "Target modality validation" and add a JSDoc on getUnsupportedDataTypes explaining what it returns, how attachments and converter outputs are combined, and why a non-empty result blocks send. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
behnam-o
approved these changes
May 8, 2026
Contributor
behnam-o
left a comment
behnam-o
left a comment
There was a problem hiding this comment.
looks good to me, just a tiny comment
Hannah: split unsupported-modality reporting by source so users can tell
whether to fix an attachment or change the selected converter.
- Replace single getUnsupportedDataTypes (returning a flat mixed list)
with two focused helpers: getUnsupportedAttachmentTypes and
getUnsupportedConverterOutputTypes.
- Warning banner now renders distinct sentences for each source, e.g.:
"This target does not support image attachments. Remove them to send."
"The selected converter produces audio output, which this target
does not support."
- Both sentences appear together when both lists are non-empty.
- Strip _path suffix from converter output labels for display, matching
ConverterPanel badge formatting.
- Updated existing test assertions for the new wording and added a test
for the both-unsupported case.
Behnam: make target_capabilities_to_info private since only the mapper
in the same file uses it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The GUI previously didn't tell users when a target doesn't support a modality they're sending (e.g., attaching an image to a text-only target, or applying a text-to-image converter against a text-only target). This resulted in cryptic backend errors.
This PR exposes target input modality capabilities to the frontend and blocks sends with a clear warning when there's a mismatch.
Changes
Backend
pyrit/backend/models/targets.py— Addedsupported_input_data_types: list[str]toTargetInstanceDTOpyrit/backend/mappers/target_mappers.py— Flatteninput_modalitiesfromTargetCapabilitiesinto the new fieldFrontend — Modality validation
frontend/src/types/index.ts— Addedsupported_input_data_typestoTargetInstancetypefrontend/src/components/Chat/converterTypes.ts— Addedfile → binary_pathmapping; addedoutputDataTypetoPieceConversionfrontend/src/components/Chat/ChatInputArea.tsx— Validates attachment types AND converter output types against target capabilities; disables send button with warning when mismatchedfrontend/src/components/Chat/ChatWindow.tsx— PassesconverterOutputDataTypesto ChatInputAreafrontend/src/components/Chat/ConverterPanel/ConverterPanel.tsx— Passes selected converter's output type to ConverterPreviewfrontend/src/components/Chat/ConverterPanel/ConverterPreview.tsx— IncludesoutputDataTypein PieceConversionFrontend — Input area styling
Tests
supported_input_data_typesScreenshot
This shows the shared space with two scroll bars, realigned buttons.