From a708480f60a0a03abb378c16be328fe9598f4a02 Mon Sep 17 00:00:00 2001 From: k1chik <107162115+k1chik@users.noreply.github.com> Date: Fri, 24 Apr 2026 11:04:19 -0400 Subject: [PATCH 1/2] docs: add API reference for CollectorRegistry and custom collector classes Closes #1163 collector/custom.md: Collector protocol section (collect/describe), value vs labels mutual exclusivity note, full constructor and add_metric tables for GaugeMetricFamily, CounterMetricFamily, SummaryMetricFamily, HistogramMetricFamily, and InfoMetricFamily, plus a runnable real-world example. collector/_index.md: constructor parameter tables for ProcessCollector, PlatformCollector, and GCCollector, with exported metrics listed for each. registry/_index.md (new): CollectorRegistry constructor and all public methods (register, unregister, collect, restricted_registry, get_sample_value, set_target_info, get_target_info), the global REGISTRY instance, and examples for isolated registry usage and registry=None. All code examples verified by running them in Python. Signed-off-by: k1chik <107162115+k1chik@users.noreply.github.com> --- docs/content/collector/_index.md | 77 +++++++++- docs/content/collector/custom.md | 250 ++++++++++++++++++++++++++++++- docs/content/registry/_index.md | 141 +++++++++++++++++ 3 files changed, 464 insertions(+), 4 deletions(-) create mode 100644 docs/content/registry/_index.md diff --git a/docs/content/collector/_index.md b/docs/content/collector/_index.md index 957c8ba9..899b7ed5 100644 --- a/docs/content/collector/_index.md +++ b/docs/content/collector/_index.md @@ -18,8 +18,8 @@ ProcessCollector(namespace='mydaemon', pid=lambda: open('/var/run/daemon.pid').r # Platform Collector The client also automatically exports some metadata about Python. If using Jython, -metadata about the JVM in use is also included. This information is available as -labels on the `python_info` metric. The value of the metric is 1, since it is the +metadata about the JVM in use is also included. This information is available as +labels on the `python_info` metric. The value of the metric is 1, since it is the labels that carry information. # Disabling Default Collector metrics @@ -33,4 +33,75 @@ import prometheus_client prometheus_client.REGISTRY.unregister(prometheus_client.GC_COLLECTOR) prometheus_client.REGISTRY.unregister(prometheus_client.PLATFORM_COLLECTOR) prometheus_client.REGISTRY.unregister(prometheus_client.PROCESS_COLLECTOR) -``` \ No newline at end of file +``` + +## API Reference + +### ProcessCollector + +```python +ProcessCollector(namespace='', pid=lambda: 'self', proc='/proc', registry=REGISTRY) +``` + +Collects process metrics from `/proc`. Only available on Linux. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `namespace` | `str` | `''` | Prefix added to all metric names, e.g. `'mydaemon'` produces `mydaemon_process_cpu_seconds_total`. | +| `pid` | `Callable[[], int or str]` | `lambda: 'self'` | Callable that returns the PID to monitor. `'self'` monitors the current process. | +| `proc` | `str` | `'/proc'` | Path to the proc filesystem. Useful for testing or containerised environments with a non-standard mount point. | +| `registry` | `CollectorRegistry` | `REGISTRY` | Registry to register with. Pass `None` to skip registration. | + +Metrics exported: + +| Metric | Description | +|--------|-------------| +| `process_cpu_seconds_total` | Total user and system CPU time in seconds. | +| `process_virtual_memory_bytes` | Virtual memory size in bytes. | +| `process_resident_memory_bytes` | Resident memory size in bytes. | +| `process_start_time_seconds` | Start time since Unix epoch in seconds. | +| `process_open_fds` | Number of open file descriptors. | +| `process_max_fds` | Maximum number of open file descriptors. | + +The module-level `PROCESS_COLLECTOR` is the default instance registered with `REGISTRY`. + +### PlatformCollector + +```python +PlatformCollector(registry=REGISTRY, platform=None) +``` + +Exports Python runtime metadata as a `python_info` gauge metric with labels. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `registry` | `CollectorRegistry` | `REGISTRY` | Registry to register with. Pass `None` to skip registration. | +| `platform` | module | `None` | Override the `platform` module. Intended for testing. | + +Labels on `python_info`: `version`, `implementation`, `major`, `minor`, `patchlevel`. +On Jython, additional labels are added: `jvm_version`, `jvm_release`, `jvm_vendor`, `jvm_name`. + +The module-level `PLATFORM_COLLECTOR` is the default instance registered with `REGISTRY`. + +### GCCollector + +```python +GCCollector(registry=REGISTRY) +``` + +Exports Python garbage collector statistics. Only active on CPython (skipped silently on +other implementations). + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `registry` | `CollectorRegistry` | `REGISTRY` | Registry to register with. Pass `None` to skip registration. | + +Metrics exported: + +| Metric | Description | +|--------|-------------| +| `python_gc_objects_collected_total` | Objects collected during GC, by generation. | +| `python_gc_objects_uncollectable_total` | Uncollectable objects found during GC, by generation. | +| `python_gc_collections_total` | Number of times each generation was collected. | + +The module-level `GC_COLLECTOR` is the default instance registered with `REGISTRY`. diff --git a/docs/content/collector/custom.md b/docs/content/collector/custom.md index bc6a021c..40060e45 100644 --- a/docs/content/collector/custom.md +++ b/docs/content/collector/custom.md @@ -35,4 +35,252 @@ not implemented and the CollectorRegistry was created with `auto_describe=True` (which is the case for the default registry) then `collect` will be called at registration time instead of `describe`. If this could cause problems, either implement a proper `describe`, or if that's not practical have `describe` -return an empty list. \ No newline at end of file +return an empty list. + +## Collector protocol + +A collector is any object that implements a `collect` method. Optionally it +can also implement `describe`. + +### `collect()` + +Returns an iterable of metric family objects (`GaugeMetricFamily`, +`CounterMetricFamily`, etc.). Called every time the registry is scraped. + +### `describe()` + +Returns an iterable of metric family objects used only to determine the metric +names the collector produces. Samples on the returned objects are ignored. If +not implemented and the registry has `auto_describe=True`, `collect` is called +at registration time instead. + +## value vs labels + +Every metric family constructor accepts either `value` or `labels`, but not +both. + +- Pass `value` (and for counters, optionally `created`) to emit a single + unlabelled metric directly from the constructor. +- Pass `labels` (a sequence of label names) and then call `add_metric` one or + more times to emit labelled metrics. + +```python +# single unlabelled value +GaugeMetricFamily('my_gauge', 'Help text', value=7) + +# labelled metrics via add_metric +g = GaugeMetricFamily('my_gauge', 'Help text', labels=['region']) +g.add_metric(['us-east-1'], 3) +g.add_metric(['eu-west-1'], 5) +``` + +## API Reference + +### GaugeMetricFamily + +```python +GaugeMetricFamily(name, documentation, value=None, labels=None, unit='') +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | `str` | required | Metric name. | +| `documentation` | `str` | required | Help text shown in the `/metrics` output. | +| `value` | `float` | `None` | Emit a single unlabelled sample with this value. Mutually exclusive with `labels`. | +| `labels` | `Sequence[str]` | `None` | Label names. Use with `add_metric`. Mutually exclusive with `value`. | +| `unit` | `str` | `''` | Optional unit suffix appended to the metric name. | + +#### `add_metric(labels, value, timestamp=None)` + +Add a labelled sample to the metric family. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `labels` | `Sequence[str]` | Label values in the same order as the `labels` constructor argument. | +| `value` | `float` | The gauge value. | +| `timestamp` | `float` or `Timestamp` | Optional Unix timestamp for the sample. | + +```python +g = GaugeMetricFamily('temperature_celsius', 'Temperature by location', labels=['location']) +g.add_metric(['living_room'], 21.5) +g.add_metric(['basement'], 18.0) +yield g +``` + +### CounterMetricFamily + +```python +CounterMetricFamily(name, documentation, value=None, labels=None, created=None, unit='', exemplar=None) +``` + +If `name` ends with `_total`, the suffix is stripped automatically so the +metric is stored without it and the `_total` suffix is added on exposition. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | `str` | required | Metric name. A trailing `_total` is stripped and re-added on exposition. | +| `documentation` | `str` | required | Help text. | +| `value` | `float` | `None` | Emit a single unlabelled sample. Mutually exclusive with `labels`. | +| `labels` | `Sequence[str]` | `None` | Label names. Use with `add_metric`. Mutually exclusive with `value`. | +| `created` | `float` | `None` | Unix timestamp the counter was created at. Only used when `value` is set. | +| `unit` | `str` | `''` | Optional unit suffix. | +| `exemplar` | `Exemplar` | `None` | Exemplar for the single-value form. Only used when `value` is set. | + +#### `add_metric(labels, value, created=None, timestamp=None, exemplar=None)` + +| Parameter | Type | Description | +|-----------|------|-------------| +| `labels` | `Sequence[str]` | Label values. | +| `value` | `float` | The counter value. | +| `created` | `float` | Optional Unix timestamp the counter was created at. | +| `timestamp` | `float` or `Timestamp` | Optional Unix timestamp for the sample. | +| `exemplar` | `Exemplar` | Optional exemplar. See [Exemplars](../../instrumenting/exemplars/). | + +```python +c = CounterMetricFamily('http_requests_total', 'HTTP requests by status', labels=['status']) +c.add_metric(['200'], 1200) +c.add_metric(['404'], 43) +c.add_metric(['500'], 7) +yield c +``` + +### SummaryMetricFamily + +```python +SummaryMetricFamily(name, documentation, count_value=None, sum_value=None, labels=None, unit='') +``` + +`count_value` and `sum_value` must always be provided together or not at all. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | `str` | required | Metric name. | +| `documentation` | `str` | required | Help text. | +| `count_value` | `int` | `None` | Observation count for a single unlabelled metric. Must be paired with `sum_value`. | +| `sum_value` | `float` | `None` | Observation sum for a single unlabelled metric. Must be paired with `count_value`. | +| `labels` | `Sequence[str]` | `None` | Label names. Use with `add_metric`. Mutually exclusive with `count_value`/`sum_value`. | +| `unit` | `str` | `''` | Optional unit suffix. | + +#### `add_metric(labels, count_value, sum_value, timestamp=None)` + +| Parameter | Type | Description | +|-----------|------|-------------| +| `labels` | `Sequence[str]` | Label values. | +| `count_value` | `int` | The number of observations. | +| `sum_value` | `float` | The sum of all observed values. | +| `timestamp` | `float` or `Timestamp` | Optional Unix timestamp for the sample. | + +```python +s = SummaryMetricFamily('rpc_duration_seconds', 'RPC duration', labels=['method']) +s.add_metric(['get'], count_value=1000, sum_value=53.2) +s.add_metric(['put'], count_value=400, sum_value=28.7) +yield s +``` + +### HistogramMetricFamily + +```python +HistogramMetricFamily(name, documentation, buckets=None, sum_value=None, labels=None, unit='') +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | `str` | required | Metric name. | +| `documentation` | `str` | required | Help text. | +| `buckets` | `Sequence` | `None` | Bucket data for a single unlabelled metric. Each entry is a `(le, value)` pair or `(le, value, exemplar)` triple. Must include a `+Inf` bucket. Mutually exclusive with `labels`. | +| `sum_value` | `float` | `None` | Observation sum. Cannot be set without `buckets`. Omitted for histograms with negative buckets. | +| `labels` | `Sequence[str]` | `None` | Label names. Use with `add_metric`. Mutually exclusive with `buckets`. | +| `unit` | `str` | `''` | Optional unit suffix. | + +#### `add_metric(labels, buckets, sum_value, timestamp=None)` + +| Parameter | Type | Description | +|-----------|------|-------------| +| `labels` | `Sequence[str]` | Label values. | +| `buckets` | `Sequence` | Bucket data. Each entry is a `(le, value)` pair or `(le, value, exemplar)` triple. Must be sorted and include `+Inf`. | +| `sum_value` | `float` or `None` | The sum of all observed values. Pass `None` for histograms with negative buckets. | +| `timestamp` | `float` or `Timestamp` | Optional Unix timestamp. | + +```python +h = HistogramMetricFamily('request_size_bytes', 'Request sizes', labels=['handler']) +h.add_metric( + ['api'], + buckets=[('100', 5), ('1000', 42), ('+Inf', 50)], + sum_value=18350.0, +) +yield h +``` + +### InfoMetricFamily + +```python +InfoMetricFamily(name, documentation, value=None, labels=None) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | `str` | required | Metric name. The `_info` suffix is added automatically on exposition. | +| `documentation` | `str` | required | Help text. | +| `value` | `Dict[str, str]` | `None` | Key-value label pairs for a single unlabelled info metric. Mutually exclusive with `labels`. | +| `labels` | `Sequence[str]` | `None` | Label names for the outer grouping. Use with `add_metric`. Mutually exclusive with `value`. | + +#### `add_metric(labels, value, timestamp=None)` + +| Parameter | Type | Description | +|-----------|------|-------------| +| `labels` | `Sequence[str]` | Outer label values (from the `labels` constructor argument). | +| `value` | `Dict[str, str]` | Key-value label pairs that form the info payload. | +| `timestamp` | `float` or `Timestamp` | Optional Unix timestamp. | + +```python +# single unlabelled info metric +yield InfoMetricFamily('build', 'Build metadata', value={'version': '1.2.3', 'commit': 'abc123'}) + +# labelled: one info metric per service +i = InfoMetricFamily('service_build', 'Per-service build info', labels=['service']) +i.add_metric(['auth'], {'version': '2.0.1', 'commit': 'def456'}) +i.add_metric(['api'], {'version': '1.9.0', 'commit': 'ghi789'}) +yield i +``` + +## Real-world example + +Proxying metrics from an external source: + +```python +from prometheus_client.core import CounterMetricFamily, GaugeMetricFamily, REGISTRY +from prometheus_client.registry import Collector +from prometheus_client import start_http_server + +# Simulated external data source +_QUEUE_STATS = { + 'orders': {'depth': 14, 'processed': 9821}, + 'notifications': {'depth': 3, 'processed': 45210}, +} + +class QueueCollector(Collector): + def collect(self): + depth = GaugeMetricFamily( + 'queue_depth', + 'Current number of messages waiting in the queue', + labels=['queue'], + ) + processed = CounterMetricFamily( + 'queue_messages_processed_total', + 'Total messages processed from the queue', + labels=['queue'], + ) + for name, stats in _QUEUE_STATS.items(): + depth.add_metric([name], stats['depth']) + processed.add_metric([name], stats['processed']) + yield depth + yield processed + +REGISTRY.register(QueueCollector()) + +if __name__ == '__main__': + start_http_server(8000) + import time + while True: + time.sleep(1) +``` diff --git a/docs/content/registry/_index.md b/docs/content/registry/_index.md new file mode 100644 index 00000000..1a81259f --- /dev/null +++ b/docs/content/registry/_index.md @@ -0,0 +1,141 @@ +--- +title: Registry +weight: 8 +--- + +A `CollectorRegistry` holds all the collectors whose metrics are exposed when +the registry is scraped. The global default registry is `REGISTRY`, which all +metric constructors register with automatically unless told otherwise. + +```python +from prometheus_client import REGISTRY, CollectorRegistry + +# Use the default global registry +from prometheus_client import Counter +c = Counter('my_counter', 'A counter') # registered with REGISTRY automatically + +# Create an isolated registry, e.g. for testing +r = CollectorRegistry() +c2 = Counter('my_counter', 'A counter', registry=r) +``` + +## Constructor + +```python +CollectorRegistry(auto_describe=False, target_info=None, support_collectors_without_names=False) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `auto_describe` | `bool` | `False` | If `True`, calls `collect()` on a collector at registration time if the collector does not implement `describe()`. Used to detect duplicate metric names. The default `REGISTRY` is created with `auto_describe=True`. | +| `target_info` | `Dict[str, str]` | `None` | Key-value labels to attach as a `target_info` metric. Equivalent to calling `set_target_info` after construction. | +| `support_collectors_without_names` | `bool` | `False` | If `True`, allows registering collectors that produce no named metrics (i.e. whose `describe()` returns an empty list). | + +## Methods + +### `register(collector)` + +Register a collector with this registry. Raises `ValueError` if any of the +metric names the collector produces are already registered. + +```python +from prometheus_client.registry import Collector + +class MyCollector(Collector): + def collect(self): + ... + +REGISTRY.register(MyCollector()) +``` + +### `unregister(collector)` + +Remove a previously registered collector. + +```python +from prometheus_client import GC_COLLECTOR +REGISTRY.unregister(GC_COLLECTOR) +``` + +### `collect()` + +Yield all metrics from every registered collector. Also yields the +`target_info` metric if one has been set. + +```python +for metric in REGISTRY.collect(): + print(metric.name, metric.type) +``` + +### `restricted_registry(names)` + +Return a view of this registry that only exposes the named metrics. Useful +for partial scrapes. See [Restricted registry](../restricted-registry/) for +usage with `generate_latest` and the built-in HTTP server. + +```python +from prometheus_client import generate_latest, CONTENT_TYPE_LATEST + +subset = REGISTRY.restricted_registry(['python_info', 'process_cpu_seconds_total']) +output = generate_latest(subset) +``` + +### `get_sample_value(name, labels=None)` + +Return the current value of a single sample, or `None` if not found. Intended +for use in unit tests; not efficient for production use. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `name` | `str` | required | Full sample name including any suffix (e.g. `'my_counter_total'`). | +| `labels` | `Dict[str, str]` | `{}` | Label key-value pairs to match. An empty dict matches an unlabelled sample. | + +```python +from prometheus_client import Counter, CollectorRegistry + +r = CollectorRegistry() +c = Counter('requests_total', 'Total requests', registry=r) +c.inc(3) + +assert r.get_sample_value('requests_total') == 3.0 +``` + +### `set_target_info(labels)` + +Set or replace the target metadata labels exposed as a `target_info` metric. +Pass `None` to remove the target info metric. + +```python +REGISTRY.set_target_info({'env': 'production', 'region': 'us-east-1'}) +``` + +### `get_target_info()` + +Return the current target info labels as a `Dict[str, str]`, or `None` if not set. + +```python +info = REGISTRY.get_target_info() +``` + +## The global REGISTRY + +`REGISTRY` is the module-level default instance, created as: + +```python +REGISTRY = CollectorRegistry(auto_describe=True) +``` + +All metric constructors (`Counter`, `Gauge`, etc.) register with `REGISTRY` +by default. Pass `registry=None` to skip registration, or pass a different +`CollectorRegistry` instance to use a custom registry. + +```python +from prometheus_client import Counter, CollectorRegistry + +# skip global registration — useful in tests +c = Counter('my_counter', 'A counter', registry=None) + +# register with a custom registry +r = CollectorRegistry() +c2 = Counter('my_counter', 'A counter', registry=r) +``` From d0a113c8706b364971af10a2224474abccac9ae8 Mon Sep 17 00:00:00 2001 From: k1chik <107162115+k1chik@users.noreply.github.com> Date: Fri, 24 Apr 2026 11:12:33 -0400 Subject: [PATCH 2/2] docs: fix review issues in registry and collector docs - GCCollector does not support registry=None on CPython; remove that claim - Remove unused CONTENT_TYPE_LATEST import from registry example - Fix 'value vs labels' section to correctly describe Summary (count_value/sum_value) and Histogram (buckets) Signed-off-by: k1chik <107162115+k1chik@users.noreply.github.com> --- docs/content/collector/_index.md | 2 +- docs/content/collector/custom.md | 8 ++++---- docs/content/registry/_index.md | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/content/collector/_index.md b/docs/content/collector/_index.md index 899b7ed5..85c6f12f 100644 --- a/docs/content/collector/_index.md +++ b/docs/content/collector/_index.md @@ -94,7 +94,7 @@ other implementations). | Parameter | Type | Default | Description | |-----------|------|---------|-------------| -| `registry` | `CollectorRegistry` | `REGISTRY` | Registry to register with. Pass `None` to skip registration. | +| `registry` | `CollectorRegistry` | `REGISTRY` | Registry to register with. | Metrics exported: diff --git a/docs/content/collector/custom.md b/docs/content/collector/custom.md index 40060e45..62c0180a 100644 --- a/docs/content/collector/custom.md +++ b/docs/content/collector/custom.md @@ -56,11 +56,11 @@ at registration time instead. ## value vs labels -Every metric family constructor accepts either `value` or `labels`, but not -both. +Every metric family constructor accepts either inline data or `labels`, but not +both. The inline data parameter name varies by type: `value` for Gauge, Counter, +and Info; `count_value`/`sum_value` for Summary; `buckets` for Histogram. -- Pass `value` (and for counters, optionally `created`) to emit a single - unlabelled metric directly from the constructor. +- Pass inline data to emit a single unlabelled metric directly from the constructor. - Pass `labels` (a sequence of label names) and then call `add_metric` one or more times to emit labelled metrics. diff --git a/docs/content/registry/_index.md b/docs/content/registry/_index.md index 1a81259f..0d554535 100644 --- a/docs/content/registry/_index.md +++ b/docs/content/registry/_index.md @@ -74,7 +74,7 @@ for partial scrapes. See [Restricted registry](../restricted-registry/) for usage with `generate_latest` and the built-in HTTP server. ```python -from prometheus_client import generate_latest, CONTENT_TYPE_LATEST +from prometheus_client import generate_latest subset = REGISTRY.restricted_registry(['python_info', 'process_cpu_seconds_total']) output = generate_latest(subset)