Summary
The Anthropic SDK exposes client.messages.countTokens(), a stable HTTP API (POST /v1/messages/count_tokens) that accepts the same parameters as messages.create() and returns an accurate input_tokens count without executing inference. This call is part of the generative workflow (used for billing estimation and routing decisions before making actual generation calls) and produces an observable API span. The JavaScript SDK instruments messages.create() in detail but has zero instrumentation for messages.countTokens(). Calls to this method produce no Braintrust spans.
What instrumentation is missing
| SDK Method |
HTTP Endpoint |
Description |
client.messages.countTokens({ model, messages, system, tools, ... }) |
POST /v1/messages/count_tokens |
Returns { input_tokens: number } — accepts the same parameters as messages.create() including tools, system prompts, images, documents, and thinking config |
The method makes a real HTTP request to the Anthropic API and is subject to the same latency and API quota considerations as inference calls. It accepts all message content types (text, images, documents), tools, and extended thinking configuration — making it observability-relevant for understanding what was being counted and how long it took.
No coverage in any instrumentation layer:
- Auto-instrumentation config (
js/src/auto-instrumentations/configs/anthropic.ts): Only instruments Messages.create (2 SDK layout variants) and beta.Messages.create / beta.Messages.toolRunner. No entry for Messages.countTokens.
- Channels (
js/src/instrumentation/plugins/anthropic-channels.ts): Only 3 channels defined — messagesCreate, betaMessagesCreate, betaMessagesToolRunner. No messagesCountTokens channel.
- Wrapper (
js/src/wrappers/anthropic.ts): Only proxies messages.create and beta.messages.toolRunner. countTokens calls pass through untraced.
A grep for countTokens across js/src/ returns zero matches.
Braintrust docs status
not_found — The Braintrust Anthropic integration page at https://www.braintrust.dev/docs/integrations/ai-providers/anthropic does not mention countTokens or token counting.
Upstream references
- Anthropic
messages.countTokens() API reference: https://docs.anthropic.com/en/api/messages-count-tokens
- Available in all current
@anthropic-ai/sdk versions (introduced well before v0.27.0)
- Accepts:
model, messages, system, tools, tool_choice, thinking — the full message parameter set
- Returns:
{ input_tokens: number } — accurate token count matching actual billing
Precedent in this repo
messages.create() is fully instrumented including input parameters, output content, token usage metrics, caching metadata, and time-to-first-token- A
countTokens span would capture: model, input content (messages/tools/system), returned input_tokens, and request latency — directly useful for correlating token estimates with actual inference costs
Local files inspected
js/src/auto-instrumentations/configs/anthropic.ts — no countTokens entryjs/src/instrumentation/plugins/anthropic-channels.ts — no messagesCountTokens channeljs/src/wrappers/anthropic.ts — no countTokens proxy handlinge2e/scenarios/anthropic-instrumentation/scenario.impl.mjs — no countTokens test scenarios
Summary
The Anthropic SDK exposes
client.messages.countTokens(), a stable HTTP API (POST /v1/messages/count_tokens) that accepts the same parameters asmessages.create()and returns an accurateinput_tokenscount without executing inference. This call is part of the generative workflow (used for billing estimation and routing decisions before making actual generation calls) and produces an observable API span. The JavaScript SDK instrumentsmessages.create()in detail but has zero instrumentation formessages.countTokens(). Calls to this method produce no Braintrust spans.What instrumentation is missing
client.messages.countTokens({ model, messages, system, tools, ... })POST /v1/messages/count_tokens{ input_tokens: number }— accepts the same parameters asmessages.create()including tools, system prompts, images, documents, and thinking configThe method makes a real HTTP request to the Anthropic API and is subject to the same latency and API quota considerations as inference calls. It accepts all message content types (text, images, documents), tools, and extended thinking configuration — making it observability-relevant for understanding what was being counted and how long it took.
No coverage in any instrumentation layer:
js/src/auto-instrumentations/configs/anthropic.ts): Only instrumentsMessages.create(2 SDK layout variants) andbeta.Messages.create/beta.Messages.toolRunner. No entry forMessages.countTokens.js/src/instrumentation/plugins/anthropic-channels.ts): Only 3 channels defined —messagesCreate,betaMessagesCreate,betaMessagesToolRunner. NomessagesCountTokenschannel.js/src/wrappers/anthropic.ts): Only proxiesmessages.createandbeta.messages.toolRunner.countTokenscalls pass through untraced.A grep for
countTokensacrossjs/src/returns zero matches.Braintrust docs status
not_found— The Braintrust Anthropic integration page at https://www.braintrust.dev/docs/integrations/ai-providers/anthropic does not mentioncountTokensor token counting.Upstream references
messages.countTokens()API reference: https://docs.anthropic.com/en/api/messages-count-tokens@anthropic-ai/sdkversions (introduced well before v0.27.0)model,messages,system,tools,tool_choice,thinking— the full message parameter set{ input_tokens: number }— accurate token count matching actual billingPrecedent in this repo
messages.create()is fully instrumented including input parameters, output content, token usage metrics, caching metadata, and time-to-first-tokencountTokensspan would capture: model, input content (messages/tools/system), returnedinput_tokens, and request latency — directly useful for correlating token estimates with actual inference costsLocal files inspected
js/src/auto-instrumentations/configs/anthropic.ts— nocountTokensentryjs/src/instrumentation/plugins/anthropic-channels.ts— nomessagesCountTokenschanneljs/src/wrappers/anthropic.ts— nocountTokensproxy handlinge2e/scenarios/anthropic-instrumentation/scenario.impl.mjs— nocountTokenstest scenarios