diff --git a/browsers/telemetry/categories.mdx b/browsers/telemetry/categories.mdx new file mode 100644 index 0000000..88352f1 --- /dev/null +++ b/browsers/telemetry/categories.mdx @@ -0,0 +1,62 @@ +--- +title: "Telemetry Categories" +description: "The categories a browser session can capture, what each contains, and their cost" +--- + +A category groups related telemetry events and is the unit you enable or disable. Selection is opt-in: a session captures a category only when you turn it on. + +For the full payload schema of any event type, see the [Stream telemetry events](https://kernel.sh/docs/api-reference/browser-telemetry/stream-telemetry-events-via-sse) endpoint in the API reference. + +## Operational + +These categories report on the session itself rather than page content. + +| Category | Captures | Event types | +| --- | --- | --- | +| `control` | Computer-control API calls against the session | `api_call` | +| `connection` | CDP and live view connect/disconnect activity | `cdp_connect`, `cdp_disconnect`, `live_view_connect`, `live_view_disconnect` | +| `system` | VM-level failures | `system_oom_kill`, `service_crashed` | +| `captcha` | Results of automated captcha solves | `captcha_solve_result` | + +## Browser activity + +These categories report what's happening in the page. Capturing any of them attaches a Chrome DevTools Protocol (CDP) collector to the session and produces highly granular page-level events. Capturing them adds overhead, so enable only the ones you need. + +| Category | Captures | Event types | +| --- | --- | --- | +| `console` | Console output from the page | `console_log`, `console_error` | +| `network` | Network requests, responses, and failures | `network_request`, `network_response`, `network_loading_failed`, `network_idle` | +| `page` | Navigation and page lifecycle, including performance signals | `page_navigation`, `page_dom_content_loaded`, `page_load`, `page_tab_opened`, `page_layout_shift`, `page_lcp`, `page_layout_settled`, `page_navigation_settled` | +| `interaction` | Browser-native input in the page (clicks, keys, scroll) | `interaction_click`, `interaction_key`, `interaction_scroll_settled` | +| `screenshot` | Periodic screenshots of the session | `monitor_screenshot` | + + +`interaction` events are browser-native DOM events observed in the page, not calls to the [computer-control](/browsers/computer-controls) API (those are reported by the `control` category). + + +### The monitor category + +`monitor` reports the health of the CDP collector itself: `monitor_disconnected`, `monitor_reconnected`, `monitor_reconnect_failed`, and `monitor_init_failed`. + +It isn't directly settable. It flows automatically whenever any of the browser-activity categories are captured. You can still [filter the stream](/browsers/telemetry/streaming) by `monitor` to isolate these events. + +## Data sensitivity + +Telemetry is off by default and the default set carries operational metadata only. The browser-activity categories are different: they capture what actually flows through the session, which is your own browser's data and can include credentials and personal information. + +| Category | Can contain sensitive data | +| --- | --- | +| `network` | Request and response headers (including `Authorization` and `Cookie`), request bodies, and truncated response bodies, plus full URLs. A common place for session tokens, credentials, and personal data. | +| `console` | Anything the page logs. Applications often log access tokens, request or response bodies, and personal data through `console.log`. | +| `page` | Page URLs and titles, which can embed tokens or identifiers in query strings or fragments. | +| `interaction` | Text of clicked elements and typed keys, which can include personal data entered into forms. | +| `screenshot` | A full rendered image of the page - the broadest exposure, capturing anything visible on screen. | +| `control`, `connection`, `system`, `captcha`, `monitor` | Session metadata only (control calls, connection and health events). No page content. | + +Captured events are persisted and can be replayed by [resuming the stream](/browsers/telemetry/streaming#resuming-after-a-disconnect), so this sensitivity applies to the data at rest, not just the live stream. Treat captured telemetry - and anywhere you forward or store it - with the same care as the underlying content. For how Kernel encrypts, retains, and processes data overall, see [Security](/security) and the [Data Processing Addendum](/dpa). + +Some exposure is reduced for you automatically: input into sensitive fields such as passwords is suppressed (`interaction_key` isn't emitted for them, and `interaction_click` omits the element text). Beyond that, because selection is opt-in, the most effective control is to capture only the categories you need - enable `network`, `console`, `page`, `interaction`, or `screenshot` deliberately, and prefer the operational categories when you only need session health. + + +If you operate under HIPAA, GDPR, or similar obligations, be deliberate about the browser-activity categories: pointing them at a site that handles regulated data captures that data into storage. If you have compliance requirements around what Kernel may process, [contact us](mailto:security@kernel.sh) before enabling them. + diff --git a/browsers/telemetry/overview.mdx b/browsers/telemetry/overview.mdx new file mode 100644 index 0000000..7cb2371 --- /dev/null +++ b/browsers/telemetry/overview.mdx @@ -0,0 +1,111 @@ +--- +title: "Telemetry Overview" +description: "Capture what happens inside a browser session" +--- + +Telemetry captures events from inside a browser session - console output, network activity, page lifecycle, user interactions, captcha solves, and operational signals like crashes or connection changes. Once enabled, you can [stream them](/browsers/telemetry/streaming) live or pull them later for analysis. + +Events are grouped into categories (`console`, `network`, `page`, and so on), and categories are the unit of control. Selection is opt-in: a session captures only the categories you turn on, and anything you don't stays off. See [Categories](/browsers/telemetry/categories) for the full list and what each one captures. + +## Enabling telemetry + +You configure telemetry when you [create a browser](/introduction/create) (and can change it later on update). There are three ways to configure it. + +### Enable the default set + +Set `enabled: true` with no per-category settings to capture the default set - a lightweight bundle of operational signals (`control`, `connection`, `system`, `captcha`) that's cheap to leave on: + + +```typescript Typescript/Javascript +import Kernel from '@onkernel/sdk'; + +const kernel = new Kernel(); + +const browser = await kernel.browsers.create({ + telemetry: { enabled: true }, +}); +``` + +```python Python +from kernel import Kernel + +kernel = Kernel() + +browser = kernel.browsers.create( + telemetry={"enabled": True}, +) +``` + +```bash CLI +kernel browsers create --telemetry=all +``` + + +### Capture specific categories + +List the categories you want under `telemetry.browser`. For example, this session captures `console` and `network` only: + + +```typescript Typescript/Javascript +const browser = await kernel.browsers.create({ + telemetry: { + browser: { + console: { enabled: true }, + network: { enabled: true }, + }, + }, +}); +``` + +```python Python +browser = kernel.browsers.create( + telemetry={ + "browser": { + "console": {"enabled": True}, + "network": {"enabled": True}, + }, + }, +) +``` + +```bash CLI +kernel browsers create --telemetry=console,network +``` + + +### Disable telemetry + + +Telemetry is disabled by default. Use this only when updating a session to turn previously enabled telemetry back off. + + +Set `enabled: false` on an existing session to turn telemetry off: + + +```typescript Typescript/Javascript +await kernel.browsers.update(browser.session_id, { + telemetry: { enabled: false }, +}); +``` + +```python Python +kernel.browsers.update( + browser.session_id, + telemetry={"enabled": False}, +) +``` + +```bash CLI +kernel browsers update --telemetry=off +``` + + + +On update, a category list patches the current selection - categories you don't include keep their current state. To reset the selection instead, send `enabled: true` (it replaces the selection with the categories you provide, or the default set if you provide none); send `enabled: false` to turn telemetry off. + + +## What's next + +- [Categories](/browsers/telemetry/categories) - every category, what it captures, the default set, and cost characteristics. +- [Stream telemetry](/browsers/telemetry/streaming) - consume the live stream from the SDK, CLI, or raw SSE, with filtering and reconnection. +- For event payload schemas, see the [Stream telemetry events](https://kernel.sh/docs/api-reference/browser-telemetry/stream-telemetry-events-via-sse) endpoint in the API reference. diff --git a/browsers/telemetry/streaming.mdx b/browsers/telemetry/streaming.mdx new file mode 100644 index 0000000..5f064ad --- /dev/null +++ b/browsers/telemetry/streaming.mdx @@ -0,0 +1,69 @@ +--- +title: "Stream Telemetry" +description: "Consume a session's live telemetry stream from the SDK or CLI" +--- + +Once a session has telemetry [enabled](/browsers/telemetry/overview), you can stream its events in real time. The stream stays open until the session terminates. + +## Via SDK + +Open the stream and iterate over the envelopes: + + +```typescript Typescript/Javascript +import Kernel from '@onkernel/sdk'; + +const kernel = new Kernel(); + +const stream = await kernel.browsers.telemetry.stream(sessionId); + +for await (const { seq, event } of stream) { + console.log(`#${seq} [${event.category}] ${event.type}`); +} +``` + +```python Python +from kernel import Kernel + +kernel = Kernel() + +with kernel.browsers.telemetry.stream(session_id) as stream: + for envelope in stream: + event = envelope.event + print(f"#{envelope.seq} [{event.category}] {event.type}") +``` + + +To filter, check `event.category` and `event.type` in your loop. If the stream drops, re-open it with the last `seq` you processed as `last_event_id` to resume without gaps. + +## Via CLI + +Stream events to your terminal. The command runs until the session ends or you interrupt it: + +```bash +kernel browsers telemetry stream +``` + +### Filtering by category or event type + +```bash +# Only network and console events +kernel browsers telemetry stream --categories=network,console + +# Only specific event types +kernel browsers telemetry stream --types=network_response,console_error + +# Machine-readable output +# -o json emits newline-delimited JSON envelopes for piping: +kernel browsers telemetry stream -o json +``` + +### Resuming after a disconnect + +The stream is a single connection; it does not reconnect on its own. Each event carries a monotonic `seq`, so to resume without gaps you re-open the stream and pass the last `seq` you processed. + +```bash +kernel browsers telemetry stream --seq 1024 +``` + +The server then replays events after that sequence number. diff --git a/docs.json b/docs.json index 4fa66f3..048e147 100644 --- a/docs.json +++ b/docs.json @@ -147,6 +147,14 @@ ] }, "browsers/extensions", + { + "group": "Telemetry", + "pages": [ + "browsers/telemetry/overview", + "browsers/telemetry/categories", + "browsers/telemetry/streaming" + ] + }, { "group": "Reserved Browsers", "pages": [ diff --git a/introduction/observe.mdx b/introduction/observe.mdx index 4f22a6e..1ae528f 100644 --- a/introduction/observe.mdx +++ b/introduction/observe.mdx @@ -3,7 +3,7 @@ title: "Observe" description: "Watch your agent work, debug what went wrong" --- -Browser agents fail in ways that don't show up in logs. Kernel gives you four ways to see what's actually happening — live, after the fact, frame by frame, and line by line. +Browser agents fail in ways that don't show up in logs. Kernel gives you several ways to see what's actually happening: live, after the fact, frame by frame, line by line, and event by event. ## Live view @@ -60,7 +60,7 @@ Add `?readOnly=true` for a non-interactive view, or enable [kiosk mode](/browser ## Replays -Replays are MP4 recordings you start and stop on demand — capture as many clips per session as you need. They're the right tool for post-hoc debugging: a failed run gives you one or more videos to scrub through, share, or attach to a bug report. +Replays are MP4 recordings you start and stop on demand - capture as many clips per session as you need. They're the right tool for post-hoc debugging: a failed run gives you one or more videos to scrub through, share, or attach to a bug report. Replays can also be enabled on managed auth sessions, so you can [debug failed logins](https://www.kernel.sh/docs/auth/configuration#record-sessions-for-debugging) the same way. @@ -119,7 +119,7 @@ Full reference: [Replays](/browsers/replays). ## Screenshots -Pull a frame at any moment with computer controls — useful for snapshotting state at decision points, attaching to traces, or feeding back into a vision model. +Pull a frame at any moment with computer controls - useful for snapshotting state at decision points, attaching to traces, or feeding back into a vision model. ```typescript Typescript/Javascript @@ -192,9 +192,55 @@ if err := stream.Err(); err != nil { Full reference: [Logs](/apps/logs). +## Telemetry + +Telemetry is a real-time, structured stream of what happens inside a session: console output, network activity, page lifecycle, interactions, and operational signals like crashes. Unlike a video or screenshot, it's machine-readable, so it's the right tool for feeding session activity into your own observability pipeline or reacting to events programmatically. Enable it at creation, then stream the events: + + +```typescript Typescript/Javascript +const browser = await kernel.browsers.create({ telemetry: { enabled: true } }); + +const stream = await kernel.browsers.telemetry.stream(browser.session_id); +for await (const { seq, event } of stream) { + console.log(`#${seq} [${event.category}] ${event.type}`); +} +``` + +```python Python +browser = kernel.browsers.create(telemetry={"enabled": True}) + +with kernel.browsers.telemetry.stream(browser.session_id) as stream: + for envelope in stream: + print(f"#{envelope.seq} [{envelope.event.category}] {envelope.event.type}") +``` + +```go Go +stream := client.Browsers.Telemetry.StreamStreaming( + ctx, + kernelBrowser.SessionID, + kernel.BrowserTelemetryStreamParams{}, +) +defer stream.Close() + +for stream.Next() { + fmt.Println(stream.Current()) +} +if err := stream.Err(); err != nil { + panic(err) +} +``` + +```bash CLI +kernel browsers telemetry stream +``` + + +Full reference: [Telemetry](/browsers/telemetry/overview). + ## Picking the right tool - **Building the agent?** Keep a [live view](/browsers/live-view) tab open while you iterate. - **Debugging a failure?** Capture a [replay](/browsers/replays) for the run, then watch the video. - **Instrumenting the agent itself?** Drop [screenshots](/browsers/computer-controls#take-screenshots) and [logs](/apps/logs) into your traces at the points that matter. +- **Feeding an observability pipeline?** Stream [telemetry](/browsers/telemetry/overview) events and route them wherever you collect signals. - **Putting a human in the loop?** Embed the [live view](/browsers/live-view#embedding-in-an-iframe) in your own UI. diff --git a/reference/cli/browsers.mdx b/reference/cli/browsers.mdx index 5d65cdc..61db5ba 100644 --- a/reference/cli/browsers.mdx +++ b/reference/cli/browsers.mdx @@ -22,6 +22,7 @@ Create a new browser session. | `--headless` | Launch without GUI/VNC access. | | `--kiosk` | Launch in Chrome kiosk mode. | | `--start-url ` | Initial page to open on launch. | +| `--telemetry ` | Configure [telemetry](/browsers/telemetry/overview) (opt-in): `all` for the default set, `off` to disable, or a comma-separated category list like `console,network`. | | `--output json`, `-o json` | Output raw JSON object. | ### `kernel browsers delete ` @@ -57,7 +58,7 @@ Update a running browser session — change or remove its proxy, load a profile, | `--save-changes` | Save changes back to the profile when the session ends. | | `--viewport ` | Set viewport size (e.g. `1920x1080@25`). | | `--force` | Force a viewport resize even during an active live view or recording. | -| `--telemetry ` | Toggle telemetry: `all` to enable, `off` to disable, or per-category like `network=on,page=off`. | +| `--telemetry ` | Update [telemetry](/browsers/telemetry/overview): `all` resets to the default set, `off` disables, or a comma-separated list like `console,network` to merge those categories into the current selection. | | `--output json`, `-o json` | Output raw API response as JSON. | ### `kernel browsers curl ` @@ -105,11 +106,11 @@ Stream browser logs from the supervisor or a file path. ## Telemetry ### `kernel browsers telemetry stream ` -Stream live telemetry events (network, console, interaction, page, and system) from a browser session. +Stream live [telemetry](/browsers/telemetry/overview) events from a browser session. Filters are applied client-side and don't change what the session captures. | Flag | Description | |------|-------------| -| `--categories ` | Filter by category: `api`, `console`, `interaction`, `network`, `page`, `system`. | +| `--categories ` | Filter by category: `console`, `network`, `page`, `interaction`, `control`, `connection`, `system`, `screenshot`, `captcha`, `monitor`. | | `--types ` | Filter by event type (e.g. `network_response,console_error`). | | `--seq ` | Resume after sequence number N; replays events with `seq > N` (default: `-1`, stream from now). | | `--output json`, `-o json` | Output newline-delimited JSON envelopes. |