diff --git a/README.md b/README.md index af8afa4..920daa0 100644 --- a/README.md +++ b/README.md @@ -1,380 +1,93 @@ -# Last9 GenAI - Python SDK +# last9-genai -> OpenTelemetry extension for LLM observability: track conversations, workflows, and costs +OpenTelemetry SDK for Python that tracks LLM cost, conversations, and agent workflows โ€” with one-call setup for OpenAI and AutoGen apps. -[![PyPI version](https://img.shields.io/pypi/v/last9-genai.svg)](https://pypi.org/project/last9-genai/) -[![PyPI downloads](https://img.shields.io/pypi/dm/last9-genai.svg)](https://pypi.org/project/last9-genai/) -[![Python 3.10+](https://img.shields.io/pypi/pyversions/last9-genai.svg)](https://pypi.org/project/last9-genai/) -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) - -## Overview - -**Track conversations and workflows in your LLM applications** with automatic context propagation. Built on OpenTelemetry for seamless integration with your existing observability stack. - -**Not a replacement** for OTel auto-instrumentation โ€” works alongside it or standalone. - -**Key Features:** -- ๐ŸŽฏ **Conversation Tracking**: Automatic multi-turn conversation tracking with `conversation_context` -- ๐Ÿค– **Agent Tracking**: First-class agent identity with `agent_context` (OTel `gen_ai.agent.*` semantic conventions) -- ๐Ÿ”„ **Workflow Management**: Track complex multi-step AI workflows with `workflow_context` -- ๐ŸŽจ **Zero-Touch Instrumentation**: `@observe()` decorator for automatic tracking -- ๐Ÿ“Š **Context Propagation**: Thread-safe attribute tracking across nested operations -- ๐Ÿ’ฐ **Optional Cost Tracking**: Bring your own pricing for cost monitoring -- ๐Ÿท๏ธ **Span Classification**: Filter by type (llm/tool/chain/agent/prompt) +[![PyPI version](https://badge.fury.io/py/last9-genai.svg)](https://badge.fury.io/py/last9-genai) +[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/) ## Features -### Core Tracking -- ๐ŸŽฏ **Conversation Tracking**: Multi-turn conversations with `gen_ai.conversation.id` and turn numbers -- ๐Ÿค– **Agent Identity**: Track agents with `gen_ai.agent.id`, `gen_ai.agent.name`, `gen_ai.agent.version` (OTel semantic conventions) -- ๐Ÿ”„ **Workflow Management**: Track multi-step AI operations across LLM calls, tools, and retrievals -- ๐Ÿ“Š **Auto-Context Propagation**: Thread-safe context managers that automatically tag all nested operations -- ๐ŸŽจ **Decorator Pattern**: `@observe()` for zero-touch instrumentation with full input/output/latency tracking -- ๐Ÿ”ง **SpanProcessor**: Automatic context enrichment for all spans in your application - -### Enhanced Observability -- ๐Ÿท๏ธ **Span Classification**: `gen_ai.l9.span.kind` for filtering (llm/tool/chain/agent/prompt) -- ๐Ÿ› ๏ธ **Tool/Function Tracking**: Enhanced attributes for function calls and tool usage -- โšก **Performance Metrics**: Response times, token counts, and quality scores -- ๐ŸŒ **Provider Agnostic**: Works with OpenAI, Anthropic, Google, Cohere, etc. -- ๐Ÿ“ **Standard Attributes**: Full OpenTelemetry `gen_ai.*` semantic conventions - -### Optional Features -- ๐Ÿ’ฐ **Cost Tracking**: Bring your own model pricing for cost monitoring -- ๐Ÿ’ธ **Workflow Costing**: Aggregate costs across multi-step operations - -## Relationship to OpenTelemetry GenAI - -**This is an EXTENSION, not a replacement:** - -| Package | Purpose | Approach | -|---------|---------|----------| -| **OTel GenAI**
`opentelemetry-instrumentation-openai-v2` | Auto-instrument LLM SDKs | Automatic (monkey-patching) | -| **Last9 GenAI**
`last9-genai` | Add conversation/workflow tracking | Context-based enrichment | - -**You can use:** -1. **Last9 GenAI alone** - Full conversation and workflow tracking -2. **Both together** - OTel auto-traces + Last9 adds conversation/workflow context (recommended!) - -See [Working with OTel Auto-Instrumentation](#working-with-otel-auto-instrumentation) for combined usage. - -## Installation - -**Basic:** -```bash -pip install last9-genai -``` - -**With OTLP export (recommended):** -```bash -pip install last9-genai[otlp] -``` - -**Requirements:** -- Python 3.10+ -- `opentelemetry-api>=1.20.0` -- `opentelemetry-sdk>=1.20.0` +- **`install()`** โ€” one call wires TracerProvider, LoggerProvider, processors, and OpenAI instrumentation +- **Conversation tracking** โ€” `gen_ai.conversation.id` across multi-turn sessions +- **Workflow cost aggregation** โ€” group LLM calls by workflow, track total spend +- **Agent identity** โ€” `gen_ai.agent.*` attributes per OTel GenAI semantic conventions +- **Cost calculation** โ€” automatic for 20+ models; bring-your-own pricing for the rest +- **Log-to-span bridge** โ€” promotes `opentelemetry-instrumentation-openai-v2` log events onto spans so the Last9 LLM dashboard renders prompts, completions, and tool calls +- **`@observe` decorator** โ€” manual span creation with tags, metadata, and category +- **Full OTel GenAI v1.28.0 compliance** ## Quick Start -**Note:** The examples below use `client` to represent your LLM client. Initialize your preferred provider: - -```python -# OpenAI -from openai import OpenAI -client = OpenAI() - -# Or Anthropic -from anthropic import Anthropic -anthropic_client = Anthropic() - -# Or any other provider (Google, Cohere, etc.) -``` - -The SDK works with **any LLM provider** - just use your client normally! - -### Track Conversations (Recommended) - -Automatically track multi-turn conversations with zero manual instrumentation: - -```python -from last9_genai import conversation_context, Last9SpanProcessor -from opentelemetry import trace -from opentelemetry.sdk.trace import TracerProvider - -# Setup tracing with Last9 processor -provider = TracerProvider() -trace.set_tracer_provider(provider) -provider.add_span_processor(Last9SpanProcessor()) - -# Track conversations automatically - works with any LLM provider -with conversation_context(conversation_id="session_123", user_id="user_456"): - # OpenAI - response1 = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": "Hello!"}] - ) - - # Anthropic (same context!) - response2 = anthropic_client.messages.create( - model="claude-sonnet-4", - messages=[{"role": "user", "content": "How are you?"}] - ) - # Both calls automatically have conversation_id = "session_123"! -``` - -### Track Workflows - -Track complex multi-step AI operations: - -```python -from last9_genai import workflow_context - -# Track entire workflow with automatic tagging -with workflow_context(workflow_id="rag_search", workflow_type="retrieval"): - # All operations automatically tagged with workflow_id - docs = retrieve_documents(query) # Tagged - context = rerank_documents(docs) # Tagged - response = generate_answer(context) # Tagged - # Full workflow visibility with zero manual instrumentation! - -# Nest workflows and conversations -with conversation_context(conversation_id="support_123"): - with workflow_context(workflow_id="order_lookup"): - # Both conversation AND workflow tracked automatically - result = lookup_and_respond() -``` - -### Track Agents - -Track agent identity using [OTel GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans/) (`gen_ai.agent.*`): - -```python -from last9_genai import agent_context - -# Track agent identity โ€” all child spans get gen_ai.agent.* attributes -with agent_context(agent_id="support_bot_v2", agent_name="Support Bot", agent_version="2.0"): - response = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": "Help me with my order"}] - ) - # Span automatically has gen_ai.agent.id, gen_ai.agent.name, gen_ai.agent.version - -# Nest with conversations for full context -with conversation_context(conversation_id="session_123", user_id="user_456"): - with agent_context(agent_id="router_agent", agent_name="Router"): - route = classify_intent(query) - - with agent_context(agent_id="support_agent", agent_name="Support"): - response = handle_support(query) - # Each agent's spans are tagged separately, both share the conversation -``` - -### Decorator Pattern (Zero-Touch) - -Use `@observe()` for automatic tracking of everything: - -```python -from last9_genai import observe - -@observe() # That's it! -def call_llm(prompt: str): - response = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": prompt}] - ) - return response - -# Automatically tracks: -# - Input (prompt) -# - Output (response) -# - Latency (span duration) -# - Context (conversation_id, workflow_id if set) - -# Works seamlessly with context managers -with conversation_context(conversation_id="session_456"): - response = call_llm("Explain quantum computing") - # Span automatically has conversation_id! +```bash +pip install last9-genai opentelemetry-exporter-otlp-proto-grpc ``` -### Optional: Cost Tracking - -Add cost monitoring by providing model pricing: - ```python -from last9_genai import ModelPricing - -# Add pricing when creating processor -processor = Last9SpanProcessor(custom_pricing={ - "gpt-4o": ModelPricing(input=2.50, output=10.0), - "claude-sonnet-4-5": ModelPricing(input=3.0, output=15.0), -}) - -# Or with decorator -pricing = {"gpt-4o": ModelPricing(input=2.50, output=10.0)} +from last9_genai import install +from opentelemetry.sdk.trace.export import BatchSpanProcessor +from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter -@observe(pricing=pricing) -def call_llm(prompt: str): - # Now also tracks cost automatically - return client.chat.completions.create(...) +handle = install() +handle.tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) ``` -### Decorator Pattern (Zero-Touch) - -Use `@observe()` decorator for automatic tracking of input/output, latency, and cost: +That's it. All providers, processors, and OpenAI instrumentation are wired automatically. -```python -from last9_genai import observe, ModelPricing - -pricing = {"gpt-4o": ModelPricing(input=2.50, output=10.0)} +Set these environment variables before running: -@observe(pricing=pricing) -def call_openai(prompt: str): - """Automatically tracks everything!""" - response = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": prompt}] - ) - return response - -# That's it! Automatically tracks: -# - Input (prompt) -# - Output (response) -# - Latency (span duration) -# - Cost (calculated from usage) -# - Metadata (from context) - -# Works with context too: -with conversation_context(conversation_id="session_123"): - response = call_openai("Hello!") - # Span automatically has conversation_id! +```bash +OTEL_SERVICE_NAME=my-llm-app +OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.last9.io +OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic " ``` -### Tags and Categories +> **Python 3.14 + openai-v2**: pin `wrapt<2`. A kwarg rename in wrapt 2.0 breaks `opentelemetry-instrumentation-openai-v2` instrumentation silently. -Add tags and categories for better filtering and organization in your observability platform: +## Conversation & Workflow Tracking ```python -from last9_genai import observe +from last9_genai import install, conversation_context, workflow_context -@observe( - tags=["production", "customer_support"], - metadata={ - "category": "customer_support", # Appears in Last9 dashboard Category column - "version": "1.0.0", - "priority": "high" - } -) -def handle_support_query(query: str): - """Categorized LLM call with metadata""" - response = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": query}] - ) - return response - -# Categories automatically appear in Last9 dashboard: -# - Category column in traces table -# - Category filter dropdown -# - Enhanced trace details - -# Use underscores for multi-word categories: -@observe(metadata={"category": "data_analysis"}) # Shows as "data analysis" -def analyze_data(data: str): - return client.chat.completions.create(...) +handle = install() +# ... wire OTLP exporter ... + +with conversation_context(conversation_id="thread-123", user_id="user-456"): + with workflow_context(workflow_id="support-flow", workflow_type="chat"): + response = client.chat.completions.create( + model="gpt-4o-mini", + messages=[{"role": "user", "content": "Hello!"}] + ) + # Span has gen_ai.conversation.id, workflow.id, user.id automatically ``` -**Common categories:** -- `customer_support`, `conversational_ai`, `code_assistant` -- `data_analysis`, `content_generation`, `summarization` -- `translation`, `research`, `qa_automation` +Contexts nest โ€” `conversation_context` wraps multiple `workflow_context` calls, all spans in scope get tagged. -## Working with OTel Auto-Instrumentation - -**Recommended**: Combine OTel auto-instrumentation with Last9 extensions: +## Agent Identity ```python -# Step 1: Auto-instrument with OpenTelemetry (standard attributes) -from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor -OpenAIInstrumentor().instrument() - -# Step 2: Add Last9 extensions (cost, workflows) -from last9_genai import Last9GenAI, ModelPricing - -l9 = Last9GenAI(custom_pricing={ - "gpt-4o": ModelPricing(input=2.50, output=10.0), -}) - -# Now make LLM calls -from openai import OpenAI -client = OpenAI() - -# OTel automatically traces this call (standard attributes) -response = client.chat.completions.create( - model="gpt-4o", - messages=[{"role": "user", "content": "Hello!"}] -) +from last9_genai import agent_context -# Last9 adds cost on top of auto-traced span -from opentelemetry import trace -span = trace.get_current_span() -usage = { - "input_tokens": response.usage.prompt_tokens, - "output_tokens": response.usage.completion_tokens, -} -cost = l9.add_llm_cost_attributes(span, "gpt-4o", usage) -print(f"Cost: ${cost.total:.6f}") +with conversation_context(conversation_id="thread-1"): + with agent_context(agent_name="support-bot", agent_id="bot-001"): + # All spans: gen_ai.agent.name, gen_ai.agent.id + response = client.chat.completions.create(...) ``` -**Result**: You get standard OTel attributes (automatic) + Last9 cost/workflow (manual). - -### Capturing Prompts, Completions, and Tool Calls - -`opentelemetry-instrumentation-openai-v2` (v2.x) follows the new OpenTelemetry -GenAI semantic conventions and emits message content, tool calls, and -completions as **OTel log events**, not as span attributes. The Last9 LLM -dashboard reads span attributes / events, so without a bridge those payloads -never reach the dashboard. - -`Last9LogToSpanProcessor` listens to those log events and promotes their -payloads onto the currently active span: - -- `gen_ai.prompt` (JSON array of prompt messages) -- `gen_ai.completion` (JSON array of completion choices) -- span events `gen_ai.content.prompt` / `gen_ai.content.completion` -- indexed `gen_ai.prompt.{i}.*` / `gen_ai.completion.{i}.*` (AgentOps / - Traceloop compatible) - -**Recommended (one call):** - -```python -from last9_genai import install - -handle = install() - -# add your OTLP exporter to the returned provider -from opentelemetry.sdk.trace.export import BatchSpanProcessor -from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter -handle.tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) -``` +`agent_context` composes with `conversation_context` and `workflow_context`. Use it for multi-agent handoffs โ€” each agent sets its own identity on its spans. -`install()` creates the TracerProvider + LoggerProvider if they don't exist -(pass your own via `tracer_provider=` / `logger_provider=`), wires -`Last9SpanProcessor` and `Last9LogToSpanProcessor` together, sets -`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true`, and calls -`OpenAIInstrumentor().instrument(logger_provider=...)` if openai-v2 is -installed. Pass `instrument_openai=False` if you want to wire instrumentation -yourself. +## Manual Instrumentation -**Manual wiring (same result, when you need more control):** +When `install()` isn't enough โ€” bring your own providers: ```python from opentelemetry import trace, _logs from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk._logs import LoggerProvider from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor - from last9_genai import Last9SpanProcessor, Last9LogToSpanProcessor +import os + +os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "true" log_bridge = Last9LogToSpanProcessor() @@ -386,312 +99,178 @@ logger_provider = LoggerProvider() logger_provider.add_log_record_processor(log_bridge) _logs.set_logger_provider(logger_provider) -import os -os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "true" OpenAIInstrumentor().instrument(logger_provider=logger_provider) ``` -Either path: every LLM call instrumented by `openai-v2` now has its full -prompt and completion content available on the span. - -> **Python 3.14 users**: pin `wrapt<2`. `opentelemetry-instrumentation-openai-v2` -> 2.3b0 calls `wrap_function_wrapper(module=..., name=..., wrapper=...)` and -> wrapt 2.0 renamed the first kwarg to `target=`. Without the pin, -> instrumentation fails silently and no log events are emitted. - -## Usage Examples - -### Multi-Turn Conversations - -Track conversations across multiple turns automatically: +## @observe Decorator ```python -from last9_genai import conversation_context +from last9_genai import observe +from openai import OpenAI -# Track a complete conversation session -with conversation_context(conversation_id="support_session_456", user_id="user_456"): - # Turn 1 - response1 = client.chat.completions.create( - messages=[{"role": "user", "content": "I need help with my order"}] - ) +client = OpenAI() - # Turn 2 - response2 = client.chat.completions.create( - messages=[ - {"role": "user", "content": "I need help with my order"}, - {"role": "assistant", "content": response1.choices[0].message.content}, - {"role": "user", "content": "Order #12345"} - ] +@observe( + tags=["production"], + metadata={"category": "customer_support"} +) +def handle_query(query: str): + return client.chat.completions.create( + model="gpt-4o", + messages=[{"role": "user", "content": query}] ) - - # Both calls automatically tagged with: - # - conversation_id = "support_session_456" - # - user_id = "user_456" - # All turns linked together for analysis! ``` -### Complex Workflows - -Track multi-step AI workflows with automatic tagging: - -```python -from last9_genai import workflow_context +`category` appears in the Last9 LLM dashboard Category column and filter dropdown. Use underscores for multi-word categories (`data_analysis` โ†’ "data analysis"). -# RAG workflow example -with workflow_context(workflow_id="rag_pipeline", workflow_type="retrieval"): - # Step 1: Query expansion (automatically tagged) - expanded_query = expand_query(user_question) +## Configuration - # Step 2: Retrieval (automatically tagged) - documents = vector_search(expanded_query) +### Environment variables - # Step 3: Reranking (automatically tagged) - relevant_docs = rerank(documents, user_question) +| Variable | Description | Default | +|----------|-------------|---------| +| `OTEL_SERVICE_NAME` | Service name in traces | `unknown-service` | +| `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP endpoint URL | required | +| `OTEL_EXPORTER_OTLP_HEADERS` | Auth headers (`key=value`) | required | +| `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | Capture prompt/completion bodies | `false` | +| `OTEL_RESOURCE_ATTRIBUTES` | Additional resource attributes | โ€” | - # Step 4: Generation (automatically tagged) - response = generate_answer(relevant_docs, user_question) +### `install()` kwargs -# All 4 steps automatically have: -# - workflow_id = "rag_pipeline" -# - workflow_type = "retrieval" -# Perfect for analyzing bottlenecks and performance! +| Kwarg | Description | Default | +|-------|-------------|---------| +| `capture_content` | Sets `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` | `True` | +| `instrument_openai` | Call `OpenAIInstrumentor().instrument(logger_provider=...)` | `True` | +| `set_global` | Register providers as OTel globals | `True` | +| `tracer_provider` | Provide an existing `TracerProvider` | `None` | +| `logger_provider` | Provide an existing `LoggerProvider` | `None` | +| `**span_processor_kwargs` | Forwarded to `Last9SpanProcessor` (e.g. `custom_pricing`) | โ€” | -### Nested Workflows and Conversations +`install()` returns an `InstallHandle` with `.tracer_provider`, `.logger_provider`, and `.shutdown()`. -Combine conversation and workflow tracking: +## Cost Tracking ```python -# Track conversation -with conversation_context(conversation_id="user_session_789", user_id="user_789"): - - # Inside conversation, track a specific workflow - with workflow_context(workflow_id="product_search", workflow_type="search"): - # Search workflow steps - results = search_products(query) - recommendations = rank_results(results) - - # Outside workflow, still in conversation - followup = handle_followup_question() - -# Result: -# - search_products and rank_results: both conversation_id AND workflow_id -# - handle_followup_question: only conversation_id -# Perfect granularity for analysis! +from last9_genai import install, ModelPricing + +handle = install( + capture_content=True, + custom_pricing={ + "gpt-4o": ModelPricing(input=2.50, output=10.0), + "gpt-4o-mini": ModelPricing(input=0.15, output=0.60), + "claude-sonnet-4-5": ModelPricing(input=3.0, output=15.0), + } +) ``` -### Tool/Function Tracking +Prices are **USD per million tokens**. Conversion: per-token `$0.000003` โ†’ `3.0`; per-1K `$0.003` โ†’ `3.0`. -Track tool calls: +### Common models -```python -with tracer.start_span("gen_ai.tool.search") as span: - l9.add_tool_attributes( - span, - tool_name="web_search", - tool_type="search", - arguments={"query": "weather"}, - result={"temp": 72}, - duration_ms=150 - ) -``` - -## OpenTelemetry Integration +| Provider | Model | Input $/1M | Output $/1M | +|----------|-------|-----------|------------| +| Anthropic | claude-opus-4-6 | 15.0 | 75.0 | +| Anthropic | claude-sonnet-4-5 | 3.0 | 15.0 | +| Anthropic | claude-haiku-4-5 | 0.8 | 4.0 | +| OpenAI | gpt-4o | 2.50 | 10.0 | +| OpenAI | gpt-4o-mini | 0.15 | 0.60 | +| OpenAI | o1 | 15.0 | 60.0 | +| Google | gemini-1.5-pro | 1.25 | 10.0 | +| Google | gemini-2.0-flash | 0.075 | 0.30 | -### Export to Last9 +For current pricing: [Anthropic](https://www.anthropic.com/pricing) ยท [OpenAI](https://openai.com/api/pricing/) ยท [Google](https://ai.google.dev/pricing) ยท [llm-prices.com](https://www.llm-prices.com/) -```bash -export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp.last9.io:443" -export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic YOUR_KEY" -``` +Use `"azure/gpt-4o"` for Azure, `"ollama/llama3.1"` with `input=0.0, output=0.0` for self-hosted. -```python -from opentelemetry import trace -from opentelemetry.sdk.trace import TracerProvider -from opentelemetry.sdk.trace.export import BatchSpanProcessor -from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter +## Architecture -# Setup -trace.set_tracer_provider(TracerProvider()) -otlp_exporter = OTLPSpanExporter() -trace.get_tracer_provider().add_span_processor( - BatchSpanProcessor(otlp_exporter) -) ``` - -### Export to Console (Development) - -```python -from opentelemetry.sdk.trace.export import ConsoleSpanExporter - -console_exporter = ConsoleSpanExporter() -trace.get_tracer_provider().add_span_processor( - BatchSpanProcessor(console_exporter) -) +your app + โ””โ”€โ”€ install() + โ”œโ”€โ”€ TracerProvider + โ”‚ โ””โ”€โ”€ Last9SpanProcessor โ† enriches spans with cost, conversation, + โ”‚ โ”‚ workflow, agent attrs + โ”‚ โ””โ”€โ”€ (your OTLP exporter) + โ””โ”€โ”€ LoggerProvider + โ””โ”€โ”€ Last9LogToSpanProcessor โ† bridges openai-v2 log events + onto the active span ``` -## Configuration +**How `Last9LogToSpanProcessor` works:** `opentelemetry-instrumentation-openai-v2` emits prompt/completion content as OTel log records (new GenAI semconv), not span attributes. The bridge listens to those log records and writes `gen_ai.prompt`, `gen_ai.completion`, span events, and indexed `gen_ai.prompt.{i}.*` / `gen_ai.completion.{i}.*` onto the active span โ€” so the Last9 LLM dashboard can render them. -### Disable Cost Tracking +## Troubleshooting -```python -# Track tokens only, skip cost calculation -l9 = Last9GenAI(enable_cost_tracking=False) -``` +### `gen_ai.prompt` / `gen_ai.completion` missing on spans -### Custom Workflow Tracker +Two likely causes: -```python -from last9_genai import WorkflowCostTracker +1. `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` is not `true`. `install(capture_content=True)` sets this automatically. +2. `OpenAIInstrumentor().instrument()` was called without `logger_provider=`. The bridge only receives events if openai-v2 routes logs to the same `LoggerProvider`. `install()` handles this automatically. -tracker = WorkflowCostTracker() -l9 = Last9GenAI(workflow_tracker=tracker) -``` - -## Attributes Reference +### No traces appearing in Last9 -### Standard OpenTelemetry (Always Set) +`install()` does **not** add an exporter โ€” you must wire one: ```python -gen_ai.system = "openai" -gen_ai.request.model = "gpt-4o" -gen_ai.usage.input_tokens = 150 -gen_ai.usage.output_tokens = 250 -``` - -### Last9 Extensions (Optional) +from opentelemetry.sdk.trace.export import BatchSpanProcessor +from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter -```python -# Cost (when pricing provided) -gen_ai.usage.cost_usd = 0.00225 -gen_ai.usage.cost_input_usd = 0.000375 -gen_ai.usage.cost_output_usd = 0.0025 - -# Classification -gen_ai.l9.span.kind = "llm" # or "tool", "prompt" - -# Workflow -workflow.id = "customer_support" -workflow.total_cost_usd = 0.015 -workflow.llm_calls = 3 - -# Conversation -gen_ai.conversation.id = "session_123" -gen_ai.conversation.turn_number = 2 - -# Agent (OTel GenAI semantic conventions) -gen_ai.agent.id = "support_bot_v2" -gen_ai.agent.name = "Support Bot" -gen_ai.agent.version = "2.0" +handle.tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) ``` -## Model Pricing - -**No default pricing included.** You provide pricing for models you use. - -### Finding Pricing - -- **Anthropic**: https://www.anthropic.com/pricing -- **OpenAI**: https://openai.com/api/pricing/ -- **Google**: https://ai.google.dev/pricing -- **Community**: https://www.llm-prices.com/ +`OTLPSpanExporter` reads `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` at instantiation time. -### Pricing Format +### Python 3.14 + wrapt error -All prices in **USD per million tokens**: - -```python -ModelPricing( - input=3.0, # $3 per 1M input tokens - output=15.0 # $15 per 1M output tokens -) ``` - -**Conversion:** -- Per-token: `$0.000003` โ†’ `3.0` -- Per-1K: `$0.003` โ†’ `3.0` - -### Common Models (February 2026) - -```python -custom_pricing = { - # Anthropic - "claude-opus-4-6": ModelPricing(input=15.0, output=75.0), - "claude-sonnet-4-5": ModelPricing(input=3.0, output=15.0), - "claude-haiku-4-5": ModelPricing(input=0.8, output=4.0), - - # OpenAI - "gpt-4o": ModelPricing(input=2.50, output=10.0), - "gpt-4o-mini": ModelPricing(input=0.15, output=0.60), - "o1": ModelPricing(input=15.0, output=60.0), - - # Google - "gemini-1.5-pro": ModelPricing(input=1.25, output=10.0), - "gemini-2.0-flash": ModelPricing(input=0.075, output=0.30), -} +TypeError: wrap_function_wrapper() got an unexpected keyword argument 'module' ``` -### Special Cases +Pin `wrapt<2` โ€” wrapt 2.0 renamed the kwarg and `opentelemetry-instrumentation-openai-v2` 2.3b0 hasn't caught up yet. -**Azure OpenAI:** -```python -custom_pricing = { - "azure/gpt-4o": ModelPricing(input=2.50, output=10.0), -} -``` +### Tool call spans missing message content -**Self-hosted (free):** -```python -custom_pricing = { - "ollama/llama3.1": ModelPricing(input=0.0, output=0.0), -} -``` +`execute_tool` span content capture (tool arguments and results) is Phase 2 work โ€” not yet implemented. Tracked in the project issues. -**Fine-tuned:** -```python -custom_pricing = { - "ft:gpt-3.5-turbo:org:model:id": ModelPricing(input=12.0, output=16.0), -} -``` +## Span Attributes Reference +| Attribute | Description | +|-----------|-------------| +| `gen_ai.conversation.id` | Thread / session identifier | +| `gen_ai.prompt` | JSON array of prompt messages | +| `gen_ai.completion` | JSON array of completion choices | +| `gen_ai.prompt.{i}.role` / `.content` | Indexed prompt messages | +| `gen_ai.completion.{i}.role` / `.content` | Indexed completion choices | +| `workflow.id` | Workflow identifier | +| `workflow.type` | Workflow type | +| `user.id` | User identifier | +| `gen_ai.agent.id` | Agent identifier | +| `gen_ai.agent.name` | Agent name | +| `gen_ai.usage.cost` | Computed cost in USD | +| `gen_ai.l9.span.kind` | `llm` / `tool` / `prompt` | ## Examples See [`examples/`](./examples/) directory: -**Basic Usage:** -- [`basic_usage.py`](./examples/basic_usage.py) - Simple LLM tracking -- [`openai_integration.py`](./examples/openai_integration.py) - OpenAI SDK -- [`anthropic_integration.py`](./examples/anthropic_integration.py) - Anthropic SDK -- [`langchain_integration.py`](./examples/langchain_integration.py) - LangChain -- [`fastapi_app.py`](./examples/fastapi_app.py) - FastAPI web app -- [`tool_integration.py`](./examples/tool_integration.py) - Function calls - -**Auto-Tracking (Recommended):** -- [`context_tracking.py`](./examples/context_tracking.py) - Context managers for automatic tracking -- [`decorator_tracking.py`](./examples/decorator_tracking.py) - @observe() decorator pattern - -**Advanced:** -- [`conversation_tracking.py`](./examples/conversation_tracking.py) - Multi-turn conversations -- [`agent_tracking.py`](./examples/agent_tracking.py) - Agent identity tracking with OTel semantic conventions +- [`basic_usage.py`](./examples/basic_usage.py) โ€” Simple LLM tracking +- [`openai_integration.py`](./examples/openai_integration.py) โ€” OpenAI SDK +- [`anthropic_integration.py`](./examples/anthropic_integration.py) โ€” Anthropic SDK +- [`langchain_integration.py`](./examples/langchain_integration.py) โ€” LangChain +- [`fastapi_app.py`](./examples/fastapi_app.py) โ€” FastAPI web app ## Contributing -Contributions welcome! Please: -1. Fork the repo +1. Fork the repository 2. Create a feature branch -3. Add tests -4. Submit a PR +3. Run tests: `uv run pytest` +4. Submit a pull request ## License -MIT License - see [LICENSE](./LICENSE) +MIT ## Support -- **Issues**: https://github.com/last9/python-ai-sdk/issues -- **Documentation**: https://github.com/last9/python-ai-sdk -- **Last9**: https://last9.io - ---- - -**Built with โค๏ธ by [Last9](https://last9.io)** +- Issues: [GitHub Issues](https://github.com/last9/python-ai-sdk/issues) +- Email: hello@last9.io