feat: add FastAPI middleware for per-request emissions tracking by davidberenstein1957 · Pull Request #1203 · mlco2/codecarbon

davidberenstein1957 · 2026-05-19T19:33:42Z

Summary

Adds optional codecarbon[fastapi] integration with CodeCarbonMiddleware to measure CO₂ emissions per HTTP request across all FastAPI/Starlette routes
Supports configurable response headers (presets, field lists, custom maps, or header_formatter callback) so clients can read emissions, energy, duration, and more from response headers
Includes create_codecarbon_lifespan for shared app-level tracking, route-based task naming, path exclusions, and two tracking modes (request for concurrency-safe per-request trackers, app for lower-overhead serialized measurement)

Test plan

uv run pytest tests/integrations/ -v — 17 tests pass
Manual smoke: uv run --extra fastapi uvicorn examples.fastapi_middleware:app --reload then curl -i localhost:8000/predict
Verify docs build: uv run task docs

Usage

from fastapi import FastAPI
from codecarbon.integrations.fastapi import add_codecarbon_middleware

app = FastAPI()
add_codecarbon_middleware(app, project_name="my-api", response_headers="default")

See docs/how-to/fastapi.md for full configuration options.

Made with Cursor

codecov · 2026-05-19T19:35:40Z

Codecov Report

❌ Patch coverage is 94.68599% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.17%. Comparing base (d2ce064) to head (bb6995c).

Files with missing lines	Patch %	Lines
codecarbon/integrations/fastapi/middleware.py	91.89%	9 Missing ⚠️
codecarbon/integrations/fastapi/_routing.py	95.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1203      +/-   ##
==========================================
+ Coverage   88.88%   89.17%   +0.28%     
==========================================
  Files          45       50       +5     
  Lines        4302     4509     +207     
==========================================
+ Hits         3824     4021     +197     
- Misses        478      488      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

benoit-cty · 2026-05-20T09:25:15Z

Hello, thanks for this. There is a problem with your branch : there are many changes that are already merged. Can you do a rebase ?

davidberenstein1957 · 2026-05-20T09:38:29Z

@benoit-cty I have reached out to some people at FastAPI, if they would be interested in a quick review :) For visibility, we could also consider deploying it as a standalone integration, but let's see if people like it.

Ship optional codecarbon[fastapi] integration with CodeCarbonMiddleware, configurable response headers, route-based task naming, and lifespan helper for shared app-level tracking. Co-authored-by: Cursor <cursoragent@cursor.com>

Add tests for app tracking mode, deprecated include_emissions_header, on_request_complete callbacks, header preset edge cases, and routing formatters so Codecov patch coverage meets the PR threshold. Co-authored-by: Cursor <cursoragent@cursor.com>

…iltering Update the FastAPI middleware to support include and exclude patterns for request tracking, allowing users to specify which endpoints to measure. Refactor routing helpers for improved clarity and add support for deferred measurement. Update documentation to reflect new features and usage examples. Co-authored-by: Cursor <cursoragent@cursor.com>

Refactor FastAPI middleware and routing code for improved readability by adjusting line breaks and indentation. Remove the product telemetry link from the documentation navigation. Update test cases for consistency in formatting and structure.

Refactor the test for FastAPI middleware to handle deferred task execution using a thread pool. This change enhances the test's reliability by ensuring that asynchronous tasks are properly awaited in a separate thread, improving overall test coverage and stability.

Refactor the exception handling in gpu_amd.py to specifically catch AttributeError when importing amdsmi. Update the warning message to provide clearer guidance on ensuring proper configuration of amdsmi for AMD GPU metrics.

Document latency for sync vs deferred logging on a MiniLM embedder workload and add a reproducible scripts/benchmark_fastapi_middleware.py runner. Co-authored-by: Cursor <cursoragent@cursor.com>

SaboniAmine

Thanks David this a great PR! Left a few questions, and saw that you have already prepared a benchmark script, do you have any numbers / graph already computed to share ?

SaboniAmine · 2026-06-09T06:01:57Z

+HeaderConfig = Union[bool, str, Sequence[str], Mapping[str, str], None]
+HeaderFormatter = Callable[[EmissionsData, Request], Mapping[str, str]]
+
+FIELD_UNITS: dict[str, str] = {


Can this be transformed in an Enum ? I have the feeling it could be leveraged elsewhere for labels and it could be maintained from the core package, not only the fastapi integration. Thoughts ?

SaboniAmine · 2026-06-09T06:02:19Z

+    "wue": "l-per-kwh",
+}
+
+HEADER_PRESETS: dict[str, dict[str, str]] = {


Same, a collection of Enums here would make sense no ?

SaboniAmine · 2026-06-09T07:37:12Z

+    return f"X-CodeCarbon-{title}{suffix}"
+
+
+def resolve_header_mapping(config: HeaderConfig) -> dict[str, str]:


Pretty sure this defensive method could be converted in a way more straightforward (and clear) one, if the unit tests demonstrate resilience to each corner case :)

SaboniAmine · 2026-06-09T07:38:05Z

+    }
+)
+
+HTTP_METHODS = frozenset(


Same, this could benefit from being an Enum

SaboniAmine · 2026-06-09T07:40:58Z

+    request: "Request",
+    formatter: Callable[["Request"], str] | None = None,
+) -> str:
+    """Derive a stable label like ``GET /items/{item_id}`` for task-scoped tracking.


This could lead to task override if resource is being accessed by 2 parallel or sequential requests. Maybe we could add some random here if the goal of this method is to provide a name for the codecarbon task concept

SaboniAmine · 2026-06-09T07:43:39Z

+from collections.abc import Awaitable, Callable, Iterable
+from typing import Any
+
+try:


This defensive import isn't needed, Starlette is installed with fastapi as a requirement for this plugin to work, you can assume that it will be present when end user will execute this code.

SaboniAmine · 2026-06-09T07:45:43Z

+    async def _start_request_tracker(self) -> EmissionsTracker:
+        return await asyncio.to_thread(self._create_and_start_tracker)
+
+    async def _stop_request_tracker(


Stopping the tracker asynchronously will have an impact on measured value. Indeed, reading the value before emitting the OutputData object is made synchronously. Lets discuss this live I faced the same issue when dealing with the vLLM integration

SaboniAmine · 2026-06-09T07:47:09Z

+        response_headers: HeaderConfig | None = None,
+        include_emissions_header: bool = False,
+        header_formatter: HeaderFormatter | None = None,
+        task_name_formatter: Callable[[Request], str] | None = None,


This is a good idea!

SaboniAmine · 2026-06-09T07:49:15Z

+
+Use **`request`** unless you have measured a need for a shared tracker. For production APIs, prefer **`app`** mode with a lifespan handler and `save_to_file=False` to avoid per-request tracker startup cost.
+
+## Performance


Could we have a benchmark vs raw FastAPI, and with another telemetry tool ?
Yes I'm thinking about Logfire but could be any other package injected as a middleware

SaboniAmine · 2026-06-09T07:52:44Z

+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):


I'm not sure about explicit lifespan explicit declaration, could it be merged with the add_codecarbon_middleware method ?
My understanding is that the developer implementing the webserver leveraging the FastAPI would like to keep ownership of this lifespan function, and I'm not sure how to stack multiple constraints with this formalism. Do we have examples about how it's managed for similar libraries ?

davidberenstein1957 force-pushed the feat/add-fastapi-middleware branch from f44740f to 15c0ccf Compare May 20, 2026 09:31

davidberenstein1957 marked this pull request as ready for review May 20, 2026 09:33

davidberenstein1957 requested a review from a team as a code owner May 20, 2026 09:33

davidberenstein1957 and others added 6 commits May 20, 2026 17:59

fix: improve error handling for AMD GPU metrics

bb6995c

Refactor the exception handling in gpu_amd.py to specifically catch AttributeError when importing amdsmi. Update the warning message to provide clearer guidance on ensuring proper configuration of amdsmi for AMD GPU metrics.

davidberenstein1957 force-pushed the feat/add-fastapi-middleware branch 2 times, most recently from eabd526 to bb6995c Compare May 20, 2026 16:03

docs: add HF embedder middleware benchmarks and benchmark script

c938690

Document latency for sync vs deferred logging on a MiniLM embedder workload and add a reproducible scripts/benchmark_fastapi_middleware.py runner. Co-authored-by: Cursor <cursoragent@cursor.com>

SaboniAmine reviewed Jun 9, 2026

View reviewed changes

		return f"X-CodeCarbon-{title}{suffix}"


		def resolve_header_mapping(config: HeaderConfig) -> dict[str, str]:


		Use `request` unless you have measured a need for a shared tracker. For production APIs, prefer `app` mode with a lifespan handler and `save_to_file=False` to avoid per-request tracker startup cost.

		## Performance

Uh oh!

Conversation

davidberenstein1957 commented May 19, 2026

Summary

Test plan

Usage

Uh oh!

codecov Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

benoit-cty commented May 20, 2026

Uh oh!

davidberenstein1957 commented May 20, 2026

Uh oh!

SaboniAmine left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented May 19, 2026 •

edited

Loading