Skip to content

[bot] HuggingFace Hub: InferenceClient generative media tasks (text_to_image, image_to_text, text_to_speech) not instrumented #487

@braintrust-bot

Description

@braintrust-bot

Summary

The existing huggingface_hub integration instruments four InferenceClient / AsyncInferenceClient task methods:

  • chat_completion
  • text_generation
  • feature_extraction
  • sentence_similarity

Three additional generative AI execution surfaces are available on the same client but produce no Braintrust spans:

Method Task type Available since
InferenceClient.text_to_image() image generation huggingface-hub ≥ 0.15
InferenceClient.image_to_text() vision captioning / VQA huggingface-hub ≥ 0.15
InferenceClient.text_to_speech() audio synthesis huggingface-hub ≥ 0.18

All three have async counterparts on AsyncInferenceClient with identical signatures.

What's missing

  • Wrapper functions (tracing shapes) for text_to_image, image_to_text, and text_to_speech in py/src/braintrust/integrations/huggingface_hub/tracing.py
  • Sync + async FunctionWrapperPatcher classes in py/src/braintrust/integrations/huggingface_hub/patchers.py
  • CompositeFunctionWrapperPatcher groups wired into the integration's setup() / auto_instrument() path
  • Cassette-backed tests under py/src/braintrust/integrations/huggingface_hub/cassettes/<version>/

Upstream sources

  • huggingface_hub.inference._client.InferenceClient.text_to_image — sync surface
  • huggingface_hub.inference._generated._async_client.AsyncInferenceClient.text_to_image — async surface
  • Same pattern for image_to_text and text_to_speech
  • huggingface-hub current matrix version: 1.17.0 ([tool.braintrust.matrix] in py/pyproject.toml)

Braintrust docs

The Braintrust integrations docs describe tracing for LLM and embedding calls via spans with input, output, metadata (model, provider params), and metrics (tokens, timing). Image generation and audio synthesis calls fit this pattern: input = prompt/audio params, output = image bytes or audio bytes, metadata = model + generation params.

Local files inspected

  • py/src/braintrust/integrations/huggingface_hub/patchers.py — current patcher list (4 task methods, no media tasks)
  • py/src/braintrust/integrations/huggingface_hub/tracing.py — wrapper implementations
  • py/pyproject.toml — matrix version huggingface-hub==1.17.0

Metadata

Metadata

Assignees

No one assigned
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions