Skip to content

feat(integrations): add Resemble Detect + Intelligence block#4925

Open
devshahofficial wants to merge 1 commit into
simstudioai:mainfrom
devshahofficial:plugin/resemble-detect-intelligence
Open

feat(integrations): add Resemble Detect + Intelligence block#4925
devshahofficial wants to merge 1 commit into
simstudioai:mainfrom
devshahofficial:plugin/resemble-detect-intelligence

Conversation

@devshahofficial

@devshahofficial devshahofficial commented Jun 9, 2026

Copy link
Copy Markdown

Resemble Detect + Intelligence block

Adds a Resemble block + tool bundle for Resemble AI media
safety: deepfake detection, media intelligence, and watermarking. Scope is
Detect + Intelligence only (no TTS / voice cloning).

What it adds

  • apps/sim/tools/resemble/ — four ToolConfigs: resemble_detect, resemble_intelligence,
    resemble_watermark_detect, resemble_watermark_apply (+ shared utils.ts, types.ts).
  • apps/sim/blocks/blocks/resemble.ts — block with an operation dropdown and per-op inputs.
  • Registered in tools/registry.ts and blocks/registry.ts; ResembleIcon added to components/icons.tsx.

Implementation

  • API-key auth (AuthMode.ApiKey), Bearer header. Optional baseUrl override.
  • Detection is asynchronous; transformResponse polls /detect/{uuid} to completion
    (bounded by maxWaitSeconds). Watermark uses Prefer: wait.
  • Large inline base64 artifacts (e.g. heatmaps) are stripped from output.

Testing

The HTTP core (Bearer auth, {success,item} envelope, submit→poll, watermark) mirrors a
client verified live against the Resemble API (detection, intelligence, watermark all green).

@vercel

vercel Bot commented Jun 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 9, 2026 7:14pm

Request Review

@cursor

cursor Bot commented Jun 9, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
New third-party integration that sends user-supplied media URLs and API keys on outbound requests; keys are user-only but workflows still handle sensitive media and credentials.

Overview
Adds a Resemble workflow block and four registered tools so workflows can call Resemble AI for deepfake detection, media intelligence, and watermark detect/apply against a public media URL.

The block is API-key auth (IntegrationType.Security), operation-driven UI (detection toggles, intelligence media type, watermark strength/message), and maps each operation to resemble_detect, resemble_intelligence, resemble_watermark_detect, or resemble_watermark_apply. Tool implementations POST to Resemble v2, use Bearer auth with optional baseUrl, poll async jobs for detect/intelligence/apply when a UUID is returned, use Prefer: wait on watermark endpoints, and strip large inline base64 from JSON outputs via sanitize.

Also adds ResembleIcon and wires ResembleBlock into blocks/registry.ts and the four tools into tools/registry.ts.

Reviewed by Cursor Bugbot for commit 49299ad. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c2e3d83. Configure here.

data = await pollResource(baseOf(params), `/intelligence/${it.uuid}`, authHeaders(params), params.maxWaitSeconds || 120)
} catch {
/* poll path may vary; return submit payload */
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong intelligence poll URL

Medium Severity

When media intelligence is still in a non-terminal state, polling uses /intelligence/{uuid} instead of the documented /intelligences/{uuid} GET route. Poll failures are caught and ignored, so the tool can return success: true with the initial submit payload instead of finished analysis.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c2e3d83. Configure here.

let last = await getJson(`${base}${path}`, headers)
while (true) {
const s = (rItem(last).status || '').toString().toLowerCase()
if (!s || TERMINAL.has(s)) return last

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Polling stops without status field

Medium Severity

pollResource treats a missing status as finished (!s), but watermark apply results signal in-progress work via watermarked_media: null without a status field. The fallback poll after apply can exit on the first GET and return success: true while the asset is still processing.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c2e3d83. Configure here.

while (true) {
const s = (rItem(last).status || '').toString().toLowerCase()
if (!s || TERMINAL.has(s)) return last
if (Date.now() >= deadline) return last

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Timeout returns incomplete success

High Severity

After maxWaitSeconds, pollResource returns the last non-terminal API payload without error. Detect and other tools always expose that as success: true, so long-running jobs can look completed while still processing and missing scores or media URLs.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c2e3d83. Configure here.

@greptile-apps

greptile-apps Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a Resemble AI integration block covering deepfake detection, media intelligence, and watermarking. It introduces four ToolConfig definitions, a matching BlockConfig with a per-operation dropdown UI, and registers everything in the tool and block registries.

  • Four tools (resemble_detect, resemble_intelligence, resemble_watermark_detect, resemble_watermark_apply) share utility helpers for auth headers, JSON polling, and base64 sanitization; the detect and intelligence tools use a submit-then-poll pattern while watermark operations use Prefer: wait with an async fallback.
  • intelligence.ts and watermark_apply.ts silently swallow poll errors and return the incomplete submit-phase payload as a success result, whereas detect.ts correctly propagates poll errors; the TERMINAL status set is also duplicated between utils.ts and intelligence.ts rather than being exported and shared.

Confidence Score: 3/5

The registry wiring and block definition are safe, but the intelligence and watermark-apply tools can silently return a pending/incomplete payload when polling fails, making the failure invisible to callers.

When pollResource throws inside intelligence.ts or watermark_apply.ts, the error is caught and discarded, and the function returns the original submit-phase data (e.g., { status: "processing", uuid: "…" }) wrapped as { success: true }. A caller executing the "Media Intelligence" or "Apply Watermark" operation on slow or temporarily unavailable result endpoints will receive what looks like a valid completed result but is actually the pending job descriptor. The same polling helper in detect.ts propagates errors correctly, so the inconsistency is likely unintentional.

apps/sim/tools/resemble/intelligence.ts and apps/sim/tools/resemble/watermark_apply.ts — both contain the silent catch-and-return pattern; apps/sim/tools/resemble/utils.ts for the early-exit on missing status.

Important Files Changed

Filename Overview
apps/sim/tools/resemble/intelligence.ts Poll errors are silently swallowed, returning a pending-state payload as success; also duplicates the TERMINAL set from utils.ts.
apps/sim/tools/resemble/watermark_apply.ts Uses Prefer: wait with an async fallback; poll errors are silently swallowed in the fallback path, same pattern as intelligence.ts.
apps/sim/tools/resemble/utils.ts Core polling/auth utilities; pollResource exits early on responses lacking a status field and the unexported TERMINAL set causes duplication in intelligence.ts.
apps/sim/tools/resemble/detect.ts Submit-then-poll deepfake detection; always polls when a UUID is present but correctly propagates poll errors to the caller.
apps/sim/blocks/blocks/resemble.ts Block definition wires up four operations with correct tool dispatch; structuredJson toggle for the intelligence operation is missing from the UI subBlocks.
apps/sim/tools/resemble/watermark_detect.ts Straightforward synchronous watermark detection; no async polling needed, errors propagate correctly.
apps/sim/tools/resemble/types.ts Clean type definitions for all four operations; no issues.
apps/sim/tools/resemble/index.ts Re-exports all four tools and types; no issues.
apps/sim/blocks/registry.ts Resemble block registered correctly alongside existing blocks.
apps/sim/tools/registry.ts All four Resemble tools registered correctly with distinct IDs.
apps/sim/components/icons.tsx Shield/waveform SVG icon added for Resemble; follows existing icon patterns.

Reviews (1): Last reviewed commit: "feat(integrations): add Resemble Detect ..." | Re-trigger Greptile

Comment on lines +41 to +45
try {
data = await pollResource(baseOf(params), `/intelligence/${it.uuid}`, authHeaders(params), params.maxWaitSeconds || 120)
} catch {
/* poll path may vary; return submit payload */
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Poll failure silently returns a pending payload as success

When pollResource throws (e.g., the result endpoint returns 404, or a network error occurs), the catch block discards the error and the function returns the submit-phase payload — typically { status: "processing", uuid: "..." } — wrapped as { success: true }. Callers have no way to distinguish "analysis complete" from "polling failed and the job is still running."

The same pattern appears in watermark_apply.ts (lines 39–43). Contrast with detect.ts, which intentionally lets the pollResource error propagate so callers can react to it.

import { authHeaders, baseOf, pollResource, rItem, sanitize } from '@/tools/resemble/utils'
import type { ToolConfig } from '@/tools/types'

const TERMINAL = new Set(['completed', 'failed', 'error', 'cancelled', 'success'])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Duplicate TERMINAL set — divergence risk

utils.ts already defines an identical TERMINAL set on line 2, but it is not exported. intelligence.ts re-declares it locally, so if a new terminal status is ever added to utils.ts (e.g., "timeout"), the pre-poll check here will be silently missed. Exporting TERMINAL from utils.ts and importing it here eliminates the drift risk.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +59 to +71
// Intelligence options
{
id: 'mediaType',
title: 'Media Type',
type: 'dropdown',
options: [
{ label: 'Auto', id: 'auto' },
{ label: 'Audio', id: 'audio' },
{ label: 'Video', id: 'video' },
{ label: 'Image', id: 'image' },
],
value: () => 'auto',
condition: { field: 'operation', value: 'resemble_intelligence' },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 structuredJson parameter has no corresponding UI control

intelligenceTool declares a structuredJson boolean param that controls whether the API returns structured JSON fields. Because no subBlock exists for it in the block definition, the value is always undefined in the UI, which causes the body builder to evaluate p.structuredJson !== false as true — permanently fixing it to structured mode. Users who need raw/unstructured output from the Intelligence operation cannot opt out from the block UI.

Comment on lines +55 to +57
while (true) {
const s = (rItem(last).status || '').toString().toLowerCase()
if (!s || TERMINAL.has(s)) return last

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No-status response exits the poll loop immediately

if (!s || TERMINAL.has(s)) return last exits when status is absent. Some intermediate async responses (e.g., queued state before a status key is populated) would cause pollResource to return the still-incomplete payload without retrying. A separate guard like if (TERMINAL.has(s)) return last (and leaving the no-status case to either continue or throw) would be safer.

Four tools (detect / intelligence / watermark detect+apply) + a Resemble block.
Bearer auth; async detection polls to completion. Registered in tool/block
registries with a ResembleIcon.
@devshahofficial devshahofficial force-pushed the plugin/resemble-detect-intelligence branch from c2e3d83 to 49299ad Compare June 9, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant