Summary
The existing huggingface_hub integration instruments four InferenceClient / AsyncInferenceClient task methods:
chat_completion
text_generation
feature_extraction
sentence_similarity
Three additional generative AI execution surfaces are available on the same client but produce no Braintrust spans:
| Method |
Task type |
Available since |
InferenceClient.text_to_image() |
image generation |
huggingface-hub ≥ 0.15 |
InferenceClient.image_to_text() |
vision captioning / VQA |
huggingface-hub ≥ 0.15 |
InferenceClient.text_to_speech() |
audio synthesis |
huggingface-hub ≥ 0.18 |
All three have async counterparts on AsyncInferenceClient with identical signatures.
What's missing
- Wrapper functions (tracing shapes) for
text_to_image, image_to_text, and text_to_speech in py/src/braintrust/integrations/huggingface_hub/tracing.py
- Sync + async
FunctionWrapperPatcher classes in py/src/braintrust/integrations/huggingface_hub/patchers.py
CompositeFunctionWrapperPatcher groups wired into the integration's setup() / auto_instrument() path
- Cassette-backed tests under
py/src/braintrust/integrations/huggingface_hub/cassettes/<version>/
Upstream sources
huggingface_hub.inference._client.InferenceClient.text_to_image — sync surface
huggingface_hub.inference._generated._async_client.AsyncInferenceClient.text_to_image — async surface
- Same pattern for
image_to_text and text_to_speech
- huggingface-hub current matrix version:
1.17.0 ([tool.braintrust.matrix] in py/pyproject.toml)
Braintrust docs
The Braintrust integrations docs describe tracing for LLM and embedding calls via spans with input, output, metadata (model, provider params), and metrics (tokens, timing). Image generation and audio synthesis calls fit this pattern: input = prompt/audio params, output = image bytes or audio bytes, metadata = model + generation params.
Local files inspected
py/src/braintrust/integrations/huggingface_hub/patchers.py — current patcher list (4 task methods, no media tasks)
py/src/braintrust/integrations/huggingface_hub/tracing.py — wrapper implementations
py/pyproject.toml — matrix version huggingface-hub==1.17.0
Summary
The existing
huggingface_hubintegration instruments fourInferenceClient/AsyncInferenceClienttask methods:chat_completiontext_generationfeature_extractionsentence_similarityThree additional generative AI execution surfaces are available on the same client but produce no Braintrust spans:
InferenceClient.text_to_image()InferenceClient.image_to_text()InferenceClient.text_to_speech()All three have async counterparts on
AsyncInferenceClientwith identical signatures.What's missing
text_to_image,image_to_text, andtext_to_speechinpy/src/braintrust/integrations/huggingface_hub/tracing.pyFunctionWrapperPatcherclasses inpy/src/braintrust/integrations/huggingface_hub/patchers.pyCompositeFunctionWrapperPatchergroups wired into the integration'ssetup()/auto_instrument()pathpy/src/braintrust/integrations/huggingface_hub/cassettes/<version>/Upstream sources
huggingface_hub.inference._client.InferenceClient.text_to_image— sync surfacehuggingface_hub.inference._generated._async_client.AsyncInferenceClient.text_to_image— async surfaceimage_to_textandtext_to_speech1.17.0([tool.braintrust.matrix]inpy/pyproject.toml)Braintrust docs
The Braintrust integrations docs describe tracing for LLM and embedding calls via spans with
input,output,metadata(model, provider params), andmetrics(tokens, timing). Image generation and audio synthesis calls fit this pattern:input= prompt/audio params,output= image bytes or audio bytes,metadata= model + generation params.Local files inspected
py/src/braintrust/integrations/huggingface_hub/patchers.py— current patcher list (4 task methods, no media tasks)py/src/braintrust/integrations/huggingface_hub/tracing.py— wrapper implementationspy/pyproject.toml— matrix versionhuggingface-hub==1.17.0