Signals page + MQL query engine + ClickHouse refactor#201
Open
BK1031 wants to merge 3 commits into
Open
Conversation
CreateVehicle previously caught duplicate-key errors with strings.Contains of "Duplicate entry" — MySQL's wording. The repo runs Postgres, which reports "duplicate key value violates unique constraint" (SQLSTATE 23505), so the upsert fallback never fired and every edit surfaced the constraint violation to the user. Replaced the catch-and-retry with clause.OnConflict on the primary key — same pattern already used in vehicle/service/config.go.
Telemetry now lives in ClickHouse (DateTime64-keyed signal table); the
query service still owns the Postgres-backed signal_definition / token
tables but reads bulk data through clickhouse-connect.
Backend layout:
- database/clickhouse.py initializes the HTTP client alongside Postgres
- service/signals.py wraps the signal table with distinct-name listing
and bucketed counts using toStartOfInterval + WITH FILL for zero-filled
axes
- service/query_lang.py + service/query_exec.py implement MQL v0.2 — a
method-chain query language:
avg(value).where(name in ("a", "b")).by(name).every(10s)
Aggregators: count/sum/avg/min/max/last/p50/p95/p99/stddev. Filters
collapse same-column equalities to OR; wildcard `*` in values becomes
LIKE. Rollup intervals from 1s to 1d.
- routes/signals.py: GET /query/signals + /query/signals/counts
- routes/query_run.py: POST /query/run for the new MQL endpoint
- service/auth_guard.py: shared bearer-token dependency
Removed: old query/log/trip/vehicle/model files that were tied to the
prior Postgres-only design. Dropped pandas/numpy/pyarrow from deps in
favor of ClickHouse-side aggregation; added clickhouse-connect.
Wire-format timestamps are always UTC-tagged ISO 8601 (`...Z` suffix) so
the browser doesn't parse them as local time.
New Signals page (/signals) is the chart-driven analog of Datadog's metrics explorer: - TimeframePicker — pill displays the absolute range with the preset label as muted annotation, click opens an editable input that accepts shortcuts (`1h`, `30m`, `2d`) or absolute ranges in the form `YYYY-MM-DD HH:MM - YYYY-MM-DD HH:MM` (local time). - QueryBuilder — Datadog-style chips for aggregator / field / filters / group-by / rollup. Filter values support `*` wildcards (LIKE under the hood) and same-column chips show an OR separator since the semantics union. Serialized live preview emits MQL v0.2 method-chain syntax. - QueryChart — recharts-backed bar/line/area with chart-type toggle, top-K + "other" rollup so 400-series queries don't blow up the legend, click-and-drag brush selection to set custom timeframes, and a safety-rail confirmation panel when bucketCount × seriesCount exceeds 20k (SVG render budget on most browsers). - Collected-signals table (deliberately unbounded by the timeframe) with fuzzy search via fuse.js, sortable columns, hover tooltips for absolute timestamps. Sidebar reorganized: Debug moved into the bottom group above Settings, Signals added under Dashboard, Query and Trips entries removed. Removed: pages/query, pages/trips, components/query, components/trips, models/trip, dead useTrip store slot.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
avg(value).where(name in ("a", "b")).by(name).every(10s)with aggregators, wildcard filters, multi-value OR, and rollups/signalsdashboard page: Datadog-style query builder (chips for aggregator/field/filters/group/rollup), bar/line/area chart-type toggle, click-and-drag brush selection, render-safety guard, collected-signals table with fuzzy search and sortable columnsON CONFLICTclause instead of MySQL's "Duplicate entry" string match...Z) so the browser doesn't parse them as local time