diff --git a/Makefile b/Makefile index 72ea582d..f21e8d34 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,16 @@ SHELL := /bin/bash VENV_NAME := .env + +# Python 3.11 is required to run mkdocs-material, which is the theme used by this project. +# sudo apt update +# sudo apt install software-properties-common +# sudo add-apt-repository ppa:deadsnakes/ppa -y +# sudo apt update +# sudo apt install python3.11 python3.11-venv python3.11-dev +# -------------------------- +# python3.11 -m pip install virtualenv + PYTHON := python3 ################################################################################ @@ -10,8 +20,8 @@ PYTHON := python3 .PHONY: virtenv_create ## Create virtualenv virtenv_create: - @python3 -c "import virtualenv" >/dev/null 2>&1 || pip install --break-system-packages --user virtualenv - python3 -m virtualenv $(VENV_NAME) + @$(PYTHON) -c "import virtualenv" >/dev/null 2>&1 || pip install --break-system-packages --user virtualenv + $(PYTHON) -m virtualenv $(VENV_NAME) source $(VENV_NAME)/bin/activate && pip install -r requirements.txt .PHONY: run_mkdocs diff --git a/docs/documentation/mv_data_space/eclipse/architecture.md b/docs/documentation/mv_data_space/eclipse/architecture.md new file mode 100644 index 00000000..9f507f5a --- /dev/null +++ b/docs/documentation/mv_data_space/eclipse/architecture.md @@ -0,0 +1,374 @@ +# CitCom.ai — Architecture + +## 1. Scope and positioning + +CitCom.ai is a **multi-tenant data space** that lets organisations +publish data assets, discover assets published by other participants, +negotiate machine-readable usage policies, and exchange the data under +those agreements. + +It is built on the **Eclipse Dataspace Components (EDC)** open-source +project (Eclipse Foundation), which we deploy through the +[Sovity Community Edition 16.4.2](https://github.com/sovity/edc-ce) +distribution (`ghcr.io/sovity/edc-ce:16.4.2`). On top of EDC, we layer: + +* **DS-Manager** — backend (FastAPI) + frontend (React) by Hopu + (`registry.hopu.eu/dsr-backend:1.3.4-generic-ds`, + `registry.hopu.eu/dsr-frontend:1.3.3-generic-ds`). DS-Manager is the + human-facing management plane: identity, asset metadata, policy + drafting, federated catalogue browsing, audit. It speaks to EDC over + EDC's Management API. Container names and Keycloak identifiers carry + the legacy `dsr-*` prefix (Hopu's original "Data Space Ready" brand); + we keep those verbatim everywhere they're a runtime literal. +* **An onboarding portal** (`onboarding-api`, Flask) for tenant + registration, the participant registry and SMTP delivery of + activation emails. +* **A federated catalogue**. Two interchangeable backends are + supported and the choice is per–data-space (`CATALOG_BACKEND` env): + DCAT in Apache Jena Fuseki for SPARQL access, or TMForum/DOME + product offerings in Scorpio NGSI-LD for a marketplace-style browse + UX. **CitCom is wired to the DOME stack** (`CATALOG_BACKEND=tmforum`). + Both pipelines are populated by sync workers that poll every + connector's EDC catalogue every 30–60 s. +* **Object storage** (MinIO, S3-compatible) with a per-tenant bucket + served behind an authenticated portal. +* **Identity** anchored on Keycloak as the OIDC issuer, plus DS-Manager-level + DIDs (`did:web:.:did:`) for participants and + per-tenant LEAR designations. +* **Reverse proxy + TLS** via Traefik, with per-host Let's Encrypt + certificates issued through the Cloudflare DNS-01 challenge. + +The result is a deployment where each tenant gets its **own subdomain**, +its **own EDC connector**, its **own DS-Manager instance** (so policy +state and audit are tenant-isolated), and its **own MinIO bucket**, but +shares the Keycloak realm, the catalogue index and the onboarding +landing page. + +## 2. Component model + +```mermaid +flowchart LR + subgraph user["End user (browser)"] + direction TB + U[Operator/
Asset owner] + end + + subgraph traefik["Traefik (TLS termination, host routing)"] + direction TB + T[Traefik
+ LE DNS-01] + end + + subgraph onboarding["Main domain — onboarding"] + direction TB + NG_main[nginx-main
portal + DS-Manager fallback] + OA[onboarding-api
Flask + SMTP] + KC[Keycloak
realm=DSR] + REG[Participant
registry.json] + SYNC1[sync-fuseki] + SYNC2[sync-dome] + FUSEKI[Apache Jena
Fuseki SPARQL] + NGSI[Scorpio
NGSI-LD] + BAE[BAE/DOME
marketplace UI] + TMFP[TMForum
productCatalog] + TMFR[TMForum
resourceCatalog] + TMFS[TMForum
serviceCatalog] + end + + subgraph admin["admin.citcom.dataspaceready.eu"] + direction TB + NG_admin[nginx-admin] + DSR_admin[dsr-backend-dev
shared compliance] + EDC_admin[cp-admin
EDC connector] + end + + subgraph tenant["{tenant}.citcom.dataspaceready.eu (× N)"] + direction TB + NG_t[nginx-tenant
+ tenant-validator] + DSR_t[dsr-backend-{t}
+ frontend] + EDC_t[cp-{t}
EDC connector
Sovity-CE 16.4.2] + PG_t[(Postgres
per tenant)] + end + + subgraph storage["Storage (shared, X-Participant gated)"] + direction TB + MINIO[MinIO] + SP[storage-portal] + end + + U -->|HTTPS| T + T --> NG_main + T --> NG_admin + T --> NG_t + NG_main --> OA + NG_main --> KC + NG_main --> BAE + NG_main --> FUSEKI + NG_main --> TMFP + NG_main --> TMFR + OA -.writes.-> REG + NG_t --> DSR_t + NG_t --> SP + NG_t -->|auth_request| OA + NG_admin --> DSR_admin + DSR_t -->|management API| EDC_t + DSR_admin -->|management API| EDC_admin + DSR_t -->|OIDC validate| KC + EDC_t -.DSP federation.- EDC_admin + EDC_t -.DSP federation.- EDC_t + SYNC1 -->|poll catalog| EDC_admin + SYNC1 -->|poll catalog| EDC_t + SYNC1 -->|TURTLE upsert| FUSEKI + SYNC2 -->|poll catalog| EDC_admin + SYNC2 -->|poll catalog| EDC_t + SYNC2 -->|TMF620| TMFP + SP --> MINIO +``` + +> See [`diagrams/architecture-overview.excalidraw`](diagrams/architecture-overview.excalidraw) +> for an editable version of the same picture. + +## 3. The Eclipse EDC role + +For each tenant we run **one EDC connector** (`cp-{tenant}`), based on +`ghcr.io/sovity/edc-ce:16.4.2`. The connector exposes four HTTP +endpoint groups, each on its own port: + +| Group | Port | Purpose | Who calls it | +|---|---|---|---| +| Management | 8181 | Asset / policy / contract definition / catalogue / negotiation / transfer CRUD | DS-Manager backend, sync workers, deploy scripts | +| Protocol (DSP) | 9084 | Eclipse Dataspace Protocol — federated catalogue, contract negotiation, transfer authorisation | Other connectors | +| Public (data plane) | 9006 | Bearer-protected data egress for `HttpData-PULL` transfers | Consumer once it has an EDR | +| Control | 9241 | Internal control plane (data-plane-management) | Connector internals only | + +The connector is configured in pure code (no UI of its own); we drive +it from DS-Manager. + +### What runs inside an EDC connector + +* **DSP catalog** — answers `dcat:Catalog` queries with the connector's + visible offerings. Each `dcat:dataset` carries a `dcat:distribution` + list (HttpData-PULL, HttpData-PUSH, S3-PUSH, AzureStorage-PUSH) and + an `odrl:hasPolicy` referring to the associated ContractDefinition. +* **Contract negotiation** — implements the Dataspace Protocol state + machine (REQUESTED → AGREED/VERIFIED → FINALIZED). Validates that + the offer the consumer requests matches what's in the catalogue + (offer policy, target asset, assigner). +* **Transfer process** — once a contract is finalized, allocates an + EDR (Endpoint Data Reference) — a one-time bearer token + URL that + the consumer uses to pull the data plane. +* **Data plane** — for `HttpData-PULL`, fronts the asset's + `dataAddress` (HTTP, S3, etc.) so the consumer never sees the raw + storage credentials. + +### Data addresses we currently support + +`AssetCreate` requests in DS-Manager map straight to EDC `dataAddress` +fields. The most common shapes: + +* `HttpData` (open or with `bearer`/`apiKey` auth) — the asset is + served from any HTTP endpoint reachable by the provider connector. +* `AmazonS3` — the asset lives in an S3-compatible bucket. We use + `endpointOverride: http://minio:9000` to target the in-cluster + MinIO. +* The internal `http://storage/edc/{tenant}/{object}` URL — a + shorter-form HttpData address served by the storage portal's + internal `/edc//...` route. Reachable only from the docker + network so it never leaks outside the data plane. + +## 4. Federated catalogue + +The platform supports **two interchangeable catalogue backends**, both +populated by polling each connector's EDC catalogue every 30–60 s. The +choice is per–data-space, set with `CATALOG_BACKEND` in the project's +`env.` file: + +| Value | Index | Browsable from | Use when | +|---|---|---|---| +| `sparql` | Apache Jena Fuseki + DCAT (one named graph per tenant) | `/catalog/` connector-ui (SPARQL UI) | Audience is data-engineering / RDF-native; you want raw SPARQL access to the federated metadata. | +| `tmforum` | Scorpio NGSI-LD + TMForum APIs (TMF620 productCatalog, TMF634 resourceCatalog, TMF638 serviceCatalog) | `/dome-catalog/` BAE marketplace UI + `/catalog/` connector-ui in TMForum mode | Audience is a marketplace / business user; you want a richer browse + filter UX and TMForum-shaped product offerings. | + +The two are functionally redundant — every dataset that reaches one +also reaches the other if both are wired up — but in practice each +deployment picks the one that matches its audience. + +> **CitCom is configured with `CATALOG_BACKEND=tmforum`**, so the +> active federated catalogue is the DOME stack (Scorpio NGSI-LD + the +> TMForum API gateway in front of it, with the BAE marketplace UI as +> the human-facing browser at +> `https://citcom.dataspaceready.eu/dome-catalog/`). +> +> The Fuseki/SPARQL pipeline (4.1 below) still runs in the cluster as +> a fallback / engineering surface — it's cheap and useful for +> debugging — but it is **not** what citcom users see. + +### 4.1 TMForum / DOME (active in CitCom) + +A worker (`sync-dome`, Node.js, every 60 s by default) walks each +connector listed in the participant registry, fetches its DCAT +catalogue via DSP, maps each dataset to a TMForum triplet +**ResourceSpecification → ProductSpecification → ProductOffering**, +and upserts the result into the **TMForum API gateway** in front of +the **Scorpio NGSI-LD** broker. The same worker also reports its +liveness to `onboarding-api`'s `/sync-status` so the catalogue page +shows "last updated s ago" to the user. + +```mermaid +sequenceDiagram + participant SD as sync-dome
(Node.js, every 60 s) + participant Reg as Registry
(onboarding-api) + participant E_a as EDC admin
(requester) + participant E_t as EDC tenant
(target) + participant TMF as TMForum API
(Scorpio NGSI-LD) + participant BAE as BAE marketplace
UI + SD->>Reg: GET /registry + Reg-->>SD: participants[] + loop for each participant + SD->>E_a: POST /catalog/request
{counterPartyAddress: target DSP} + E_a->>E_t: DSP catalog request + E_t-->>E_a: dcat:Catalog + E_a-->>SD: dcat:Catalog + SD->>TMF: upsert ResourceSpec → ProductSpec → ProductOffering
(TMF634 / TMF620) + end + BAE->>TMF: GET /productOffering
(browser query) + TMF-->>BAE: filtered list +``` + +Human-facing browser: the **BAE / DOME marketplace** at +`https://citcom.dataspaceready.eu/dome-catalog/` (and a TMForum-mode +build of `connector-ui` at `/catalog/` that hits the same TMForum API +endpoints rather than SPARQL). + +### 4.2 DCAT / SPARQL (Apache Jena Fuseki — alternative backend) + +This is the path you'd pick if your audience is RDF-native or you want +a programmable SPARQL surface across the whole data space. CitCom +doesn't expose this to end users, but the pipeline runs in the cluster +as a parallel index — useful for debugging and as a fallback. + +```mermaid +sequenceDiagram + participant SS as sync-service
(Node.js, every 30 s) + participant Reg as Registry
(onboarding-api) + participant E_a as EDC admin
(requester) + participant E_t as EDC tenant
(target) + participant F as Fuseki
(named graphs) + SS->>Reg: GET /registry + Reg-->>SS: participants[] + loop for each participant + SS->>E_a: POST /catalog/request
{counterPartyAddress: target DSP} + E_a->>E_t: DSP catalog request + E_t-->>E_a: dcat:Catalog + E_a-->>SS: dcat:Catalog + SS->>F: SPARQL UPDATE
graph http://dataspace/{tenant}/catalog + end +``` + +When a project sets `CATALOG_BACKEND=sparql`, the `connector-ui` at +`/catalog/` switches to a SPARQL-mode build that issues federated +queries directly against Fuseki for the union of all tenant graphs. + +## 5. Identity model + +```mermaid +flowchart TB + subgraph kc[Keycloak realm DSR] + direction TB + DSRAPP[client: dsr-app
shared template] + DSRTLIB[client: dsr-app-libelium
confidential] + DSRTVAL[client: dsr-app-valencia
confidential] + DSRTUPV[client: dsr-app-upv
confidential] + PORTAL[client: ds-portal
onboarding portal] + USERS[Users in groups
libelium, valencia, upv
roles: dsr:asset-owner, dsr:operator] + end + + subgraph dsr_t[DS-Manager per tenant] + direction TB + DID[/api/v1/dids/
did:web:<subdomain>.<domain>:did:<uuid>] + ORG[/api/v1/orgs/
org_id, legal_name, headquarter_address] + LEAR[/api/v1/orgs/<id>/lear
designates user_did → org] + end + + USERS -->|password grant| DSRTLIB + DSRTLIB -->|access_token| dsr_t + dsr_t --> DID + dsr_t --> ORG + dsr_t --> LEAR +``` + +* **Keycloak** is the identity issuer. It hosts a single realm + (`DSR`), one shared template client (`dsr-app`) used by the portal, + one confidential client per tenant (`dsr-app-{tenant}`) used by that + tenant's DS-Manager backend, and a `ds-portal` client for the main-domain + portal. +* The per-tenant client is **provisioned automatically by + `provision-tenant.py`** and kept in sync (mappers, scopes, flags) + with `dsr-app` on every re-run. +* **DS-Manager DIDs** are filesystem-style `did:web:` documents served from + `/did//x509CertificateChain.pem`. Every user gets a personal + DID; every Org gets one too. The LEAR (Legal Entity Appointed + Representative) link maps a user DID to an org DID, which is what a + consumer's connector verifies during contract negotiation. + +## 6. Transfer flow (the path Albufera CSV took) + +```mermaid +sequenceDiagram + participant L as Libelium DS-Manager
(consumer) + participant EL as EDC libelium + participant EV as EDC valencia + participant V as Valencia DS-Manager
(provider) + participant M as MinIO + L->>EL: POST /catalog/request
(target: valencia DSP) + EL->>EV: DSP catalog request + EV-->>EL: dcat:Catalog
{Albufera water level (15-min)} + EL-->>L: catalogue with offer policy + L->>EL: POST /contractnegotiations
{policy verbatim from offer} + EL->>EV: DSP contract request + EV-->>EL: AGREED → VERIFIED → FINALIZED + L->>EL: POST /transferprocesses
{contractId, HttpData-PULL} + EL->>EV: DSP transfer request + EV->>EL: EDR (endpoint, bearer, expiry) + L->>EL: POST /edrs/request → GET /edrs/{tp}/dataaddress + EL-->>L: {endpoint: http://cp-valencia:9006/api/public, authorization: …} + L->>EV: GET /api/public + Authorization: bearer + EV->>M: read object via dataAddress + M-->>EV: stream + EV-->>L: stream (CSV bytes) +``` + +> See [`diagrams/transfer-flow.excalidraw`](diagrams/transfer-flow.excalidraw) +> for an editable version of the same sequence. + +Two notes from the implementation: + +1. The `policy` block sent in `POST /contractnegotiations` MUST + include `odrl:` JSON-LD prefixes (`@type: odrl:Offer`, `odrl:target`, + `odrl:assigner`). If you ship it under the EDC default `@vocab`, EDC + validates `@type` as `https://w3id.org/edc/v0.0.1/ns/Offer` and + rejects. +2. The provider uses the consumer's verified DID/Org as the input to + policy evaluation. With our open `albufera-open-use` policy + (USE-only, no constraints) any verified consumer of any participant + passes. With ABAC policies you can restrict per-attribute (LEAR, + organisation country, etc.). + +## 7. Per-tenant data isolation + +| Layer | Mechanism | +|---|---| +| HTTP routing | One `nginx-{tenant}` per subdomain; `tenant-validator` calls strict mode for the storage portal so a session for tenant A cannot read tenant B's bucket. | +| Storage | `nginx-tenant.conf` injects `X-Participant: {tenant}` so the storage-portal serves only that bucket. Internal `/edc//` routes are blocked at the public gateway and only reachable from the docker network. | +| Identity | DS-Manager backend authenticates against `dsr-app-{tenant}` confidential client; `resource_access.dsr-app-{tenant}.roles` is the only role path it trusts. | +| Catalog | Sync workers write each tenant's catalogue to its own SPARQL named graph and its own TMForum tenant header. | +| Database | One `dsr-postgres-{tenant}` and one `db-{tenant}` (EDC). Per-tenant compose generated by `provision-tenant.py`. | + +## 8. References + +* Eclipse EDC project: +* Sovity Community Edition (the distribution we run): +* DSP (Dataspace Protocol) spec: +* ODRL (used for policy syntax): +* DCAT (catalogue vocabulary): +* TMForum APIs (the marketplace catalogue): +* CitCom.ai (the Testing & Experimentation Facility funding the + pilot): diff --git a/docs/documentation/mv_data_space/eclipse/deployment.md b/docs/documentation/mv_data_space/eclipse/deployment.md new file mode 100644 index 00000000..73ff329c --- /dev/null +++ b/docs/documentation/mv_data_space/eclipse/deployment.md @@ -0,0 +1,324 @@ +# CitCom.ai — Deployment + +This is the operational guide for the CitCom data space at +`https://citcom.dataspaceready.eu`. It assumes you've read +[`architecture.md`](architecture.md) for the *what*; this doc focuses +on the *how*. + +## 1. Topology + +```mermaid +flowchart LR + CF[Cloudflare DNS
*.dataspaceready.eu] + subgraph host[citcom-ds host — 145.239.127.76] + direction TB + Traefik["Traefik :443
LE DNS-01"] + NM[nginx-main] + NA[nginx-admin] + Nt1[nginx-libelium] + Nt2[nginx-valencia] + Nt3[nginx-upv] + NMP[nginx-mailpit] + OA[onboarding-api] + KC[Keycloak
+ Postgres] + DSRA[admin DS-Manager
+ EDC + Postgres] + T1[libelium DS-Manager
+ EDC + Postgres] + T2[valencia DS-Manager
+ EDC + Postgres] + T3[upv DS-Manager
+ EDC + Postgres] + MIN[MinIO + storage-portal] + Sync[sync-fuseki / sync-dome] + Loki[Loki + Alloy] + Mailpit[Mailpit web UI
SMTP relays to buzondecorreo] + end + CF -->|443| Traefik + Traefik --> NM & NA & Nt1 & Nt2 & Nt3 & NMP +``` + +- **Server:** `145.239.127.76` (`citcom-ds`), Ubuntu, user `ubuntu`. +- **Deploy directory:** `~/citcom/` (rsync target — not a git + worktree). +- **Single Docker network:** `edc-net` (external; created on first + deploy). +- **Persistent state:** Docker volumes for each Postgres and MinIO + instance + the Traefik ACME store. +- **DNS:** Wildcard `*.citcom.dataspaceready.eu` in Cloudflare → host + IP. The `traefik-cloudflare-companion` sidecar registers any new + Traefik host label automatically, so a new tenant subdomain + resolves without manual DNS work. +- **TLS:** Per-host Let's Encrypt certs via the Cloudflare DNS-01 + challenge (no port-80 challenge → wildcard-friendly + does not + break HTTP→HTTPS redirect). + +## 2. The deploy command + +A full deploy from a developer machine is one line: + +```bash +make deploy-prod-remote PROJECT=citcom +``` + +What it does: + +1. **rsync** the repo (excluding `.git/`, `active/`, secrets) to + `ubuntu@citcom-ds:~/citcom/`. +2. **ssh** into the host and run `make deploy-prod PROJECT=citcom`, + which is a thin wrapper for `make deploy ENV=production + PROJECT=citcom SSL=skip`. +3. **smoke test** with `curl https://citcom.dataspaceready.eu/`. + +The `make deploy` target itself does, in order: + +1. `make activate` — render `active/` from `platform/` + `projects/citcom/` + + `env.production` + `tenants.yml`. This regenerates + `active/keycloak/realm-export.json`, every tenant compose under + `active/tenants/`, the Keycloak theme, the navbar config, etc. +2. Fix file permissions (`chmod -R o+rX active/`) so containers can + read bind-mounted assets. +3. **Pull all images** referenced by every compose file. Hopu + re-publishes `1.3.x-generic-ds` regularly with the same tag, so a + blind `docker compose up -d` would silently keep the old digest; + we explicitly pull first. +4. Bring up the platform: Traefik (with `docker-compose.cloudflare.yml` + and `docker-compose.traefik-le.yml` overlays when + `CLOUDFLARE_ENABLED=true`), then `docker-compose.yml`, + then MinIO (`docker-compose-minio.yml`), then the admin connector + (`docker-compose-tenant-admin.yml`), then every per-tenant compose + under `active/tenants/`. +5. **Force-recreate every nginx gateway**. Their config is bind-mounted + from `active/nginx/*.conf.template` so a vanilla `up -d` doesn't + pick up template changes. +6. **Force-recreate Keycloak**. Same bind-mount problem (the + themes directory and `realm-export.json` are mounted; `make + activate` removes and recreates those files, so the inode under the + old container is stale). +7. Bring up the audit pipeline (Loki + Alloy). +8. **Run `keycloak_sync.py`** (self-heal). Reconciles realm-level + fields, mappers and per-client redirect URIs against + `active/keycloak/realm-export.json`. Self-heals admin password + drift via fallback-and-rotate. Idempotent — second run is a no-op. +9. **Run `register-admin-connector.py`** to ensure the `admin` + connector exists in the admin DS-Manager's `connector_registry.ds_connector`. +10. Print URLs and exit. + +## 3. Provisioning a new tenant + +Tenants are declared in [`tenants.yml`](../tenants.yml). Each entry +covers identity, EDC, contact, users and (optional but recommended) +`org_details` for the DS-Manager Organisation registry: + +```yaml +- name: libelium # used as connector / db / k8s-style identifier + subdomain: libelium # https://libelium.citcom.dataspaceready.eu + org: Libelium # display name + sector: Technology + contact: + name: Mateo Ferri + email: m.ferri@libelium.com + role: Data Engineer + edc: + api_key: api-key-libelium # X-Api-Key for the connector's management API + public_port: 9005 # internal port for the connector's data plane + org_details: + vat: ESB99065815 + street: Calle Bari 19, Edificio CEEI + city: Zaragoza + postal: "50197" + subdivision: ES-Z # ISO-3166-2 — the DS-Manager backend rejects anything else + country: ES + users: + - email: m.ferri@libelium.com + password: ... + first_name: Mateo + last_name: Ferri + roles: [dsr:asset-owner, dsr:operator] +``` + +Then: + +```bash +# After editing tenants.yml on a deployed host: +cd ~/citcom +ln -sf active/env env # provision-tenant.py reads cwd/env +ln -sf active/tenants.yml tenants.yml # ... and cwd/tenants.yml +python3 platform/scripts/provision-tenant.py --skip-verify +# or do all at once: +python3 platform/scripts/provision-tenant.py --all --skip-verify +``` + +`provision-tenant.py` performs **9 idempotent steps** per tenant: + +1. Generate `active/tenants/docker-compose-tenant-.yml`. +2. Add the tenant subdomain to the `dsr-app` redirect URIs. +3. Create or **self-heal** the per-tenant OIDC client (`dsr-app-`) + — re-applies all 5 protocol mappers and copies the 8 default + 1 + optional client scopes from `dsr-app` even when the client already + exists, fixing accumulated drift. +4. Start the tenant containers (or `--skip-containers` if they're + already up after a `make deploy`). +5. Create the Keycloak group + each declared user, attach roles. +6. Register the EDC connector in the tenant DS-Manager's + `connector_registry.ds_connector` (HTTP API first, falls back to + direct Postgres insert). +7. Identity setup (DID + Organisation + LEAR designation). **This step + runs inside the `onboarding-api` container** so all the HTTP calls + resolve through the internal Docker DNS — running on the host + silently fails because the public domain doesn't loop back. +8. Add the new connector's catalogue endpoints to the sync-service + compose so the federated catalogue picks it up. +9. Endpoint smoke check (skipped with `--skip-verify`). + +## 4. Self-healing + +Two self-healers run on every deploy: + +* **`platform/scripts/keycloak_sync.py`** — reconciles the running + Keycloak realm with the rendered `active/keycloak/realm-export.json` + (themes, SMTP, locale, mappers, redirect URIs). Auto-rotates the + admin password if the value in `env.production` no longer + authenticates and a known fallback (e.g. `admin`) does. +* **`platform/onboarding-api/register-admin-connector.py`** — ensures + the admin EDC connector is registered in the admin DS-Manager's + `connector_registry.ds_connector`. SQL-direct (the previous + `/test-auth/create-session` endpoint was removed in dsr-backend + 1.3.4). + +Both are idempotent and non-fatal: a deploy succeeds even if either +fails. + +## 5. Service inventory + +| Container | Image | Purpose | +|---|---|---| +| `traefik` | `traefik:latest` | TLS termination, host-based routing, ACME (LE) via Cloudflare DNS-01 | +| `cloudflare-companion` | `tiredofit/traefik-cloudflare-companion:latest` | Auto-register DNS records for every Traefik host label | +| `nginx-main` | `nginx:1.25-alpine` | Onboarding portal, `/catalog/`, `/registry/`, DS-Manager fallback | +| `nginx-admin` | `nginx:1.25-alpine` | `admin.` — admin DS-Manager proxy | +| `nginx-mailpit` | `nginx:1.25-alpine` | `mailpit.` (debug catcher; SMTP egress is real) | +| `nginx-{tenant}` | `nginx:1.25-alpine` | One per tenant subdomain | +| `tenant-validator` | custom (FastAPI) | nginx `auth_request` subhandler — validates session against DS-Manager backend, gates `/storage/` strictly | +| `onboarding-api` | custom (Flask) | Participant registry, `/auth/*` portal, activation emails (SMTP), provision API | +| `dsr-keycloak-dev` + `dsr-keycloak-db-dev` | `quay.io/keycloak/keycloak:26.4` + `postgres:15-alpine` | OIDC issuer | +| `dsr-frontend-{tenant}` | `registry.hopu.eu/dsr-frontend:1.3.3-generic-ds` | Per-tenant DS-Manager UI | +| `dsr-backend-{tenant}` | `registry.hopu.eu/dsr-backend:1.3.4-generic-ds` | Per-tenant DS-Manager API (assets, policies, identity) | +| `cp-{tenant}` | `ghcr.io/sovity/edc-ce:16.4.2` | Eclipse EDC connector per tenant | +| `dsr-postgres-{tenant}` + `db-{tenant}` | `postgres:15-alpine` | DS-Manager + EDC databases per tenant | +| `dsr-redis-{tenant}` | `redis:7-alpine` | DS-Manager session store per tenant | +| `dsr-did-init-{tenant}` | one-shot | Generates DID material on first start | +| `cp-admin` + `db-admin` | EDC + postgres | Shared admin connector | +| `dsr-backend-dev` + `dsr-postgres-dev` + `dsr-redis-dev` | DS-Manager | Admin DS-Manager (compliance / sync centre) | +| `dsr-frontend-dev` | `registry.hopu.eu/dsr-frontend:1.3.3-generic-ds` | Admin DS-Manager UI | +| `minio` + `minio-init` | MinIO | Object storage with per-bucket per-tenant separation | +| `storage-portal` | custom (Flask + minio-py) | Authenticated upload/list UI per tenant; X-Participant header gates the bucket | +| `http-source`, `http-sink` | nginx + node | Test-data servers used in transfer demos | +| `fuseki` | Apache Jena | DCAT catalogue index | +| `scorpio` + `dome-postgres` + `tmforum-{product,resource,service}-catalog` | DOME | TMForum API stack | +| `bae-frontend` | DOME marketplace UI | Browseable product catalogue | +| `sync-service` (a.k.a. `sync-fuseki`) | `node:20-alpine` | Polls every connector's DCAT catalogue → Fuseki SPARQL | +| `sync-dome` | `node:20-alpine` | Polls every connector → TMForum | +| `mailpit` | `axllent/mailpit:latest` | Local SMTP catcher (debug) | +| `dsr-loki-dev` + `dsr-alloy-dev` | Loki + Grafana Alloy | Audit log aggregation | + +## 6. Self-issued certificates + +The Traefik service is configured to issue per-host certs through the +Cloudflare DNS-01 challenge. The flag chain: + +``` +--certificatesresolvers.le.acme.email=... +--certificatesresolvers.le.acme.storage=/acme/acme.json +--certificatesresolvers.le.acme.dnschallenge=true +--certificatesresolvers.le.acme.dnschallenge.provider=cloudflare +--certificatesresolvers.le.acme.dnschallenge.resolvers=1.1.1.1:53,8.8.8.8:53 +``` + +Each routed service ships a label `traefik.http.routers..tls.certresolver=${TRAEFIK_CERTRESOLVER:-}`, +which evaluates to `le` in production (where the env sets it) and to +empty in local dev (where Traefik falls back to the file provider's +self-signed wildcard). Adding a new tenant therefore needs no manual +cert work — Traefik issues one on the first request to its host. + +## 7. SMTP / email + +`onboarding-api` sends activation emails directly through SMTP (it does +**not** delegate to Keycloak's mailing). Production points at +`smtp.buzondecorreo.com:465` SSL, authenticated as +`info@geospace.es` — recipients see the message as +`From: CitCom.ai `. The HTML template lives at +[`platform/onboarding-api/templates/activation-email.html`](../../../platform/onboarding-api/templates/activation-email.html) +and is parametrised entirely from env (`EMAIL_PRIMARY_COLOR`, +`EMAIL_ACCENT_COLOR`, `EMAIL_LOGO_FILE`, …) so each project's emails +match its login + portal palette. + +> **Caveat** — sender domain (`geospace.es`) does not match the data- +> space domain (`citcom.dataspaceready.eu`), so a strict DMARC policy +> at the recipient's MX may quarantine. Long-term we'll set up +> `noreply@citcom.dataspaceready.eu` with proper SPF/DKIM in +> Cloudflare; short-term Mailpit on `mailpit.citcom.dataspaceready.eu` +> remains as a debug surface. + +## 8. Common operational tasks + +### Add a tenant + +1. Append entry to `projects/citcom/tenants.yml`. +2. `make deploy-prod-remote PROJECT=citcom` (or, if you only want the + tenant change without a full redeploy, ssh in and + `python3 platform/scripts/provision-tenant.py `). +3. Activation emails are NOT sent automatically by provisioning — they + are sent by an explicit call to the onboarding-api's `send_activation` + helper. We dispatch them in batch after provisioning completes (see + the in-session ad-hoc script we used the day of go-live). + +### Rotate the Keycloak admin password + +1. Edit `KEYCLOAK_ADMIN_PASSWORD` in `projects/citcom/env.production`. +2. `make deploy-prod-remote PROJECT=citcom`. +3. `keycloak_sync.py` detects the env doesn't authenticate, falls back + to the prior known password, resets the master admin user to the + new env value. +4. Save the new password to a secrets store (memory, vault, …) and + update `memory/citcom_keycloak_admin.md`. + +### Refresh DS-Manager images + +Because Hopu re-tags `1.3.x-generic-ds`, a redeploy is enough — the +deploy target now does an explicit `docker compose pull` before +`up -d`. Force-recreate of the relevant containers happens +automatically. + +### Diagnose a failing transfer + +1. Provider EDC log: `docker logs cp- --tail 200` + — look for `ContractNegotiation: ID … Fatal error` lines. The + most common one is the JSON-LD policy mismatch on + `POST /contractnegotiations`. +2. Consumer side: `GET /api/management/v3/contractnegotiations/` + shows the state machine; `state=TERMINATED` plus `errorDetail` + tells you what the consumer rejected. +3. EDR phase: `GET /api/management/v3/edrs/{transferProcessId}/dataaddress` + returns the bearer token + endpoint. If that endpoint 401s, the + token expired or the data-plane URL is wrong. + +## 9. Known runtime drift / things to watch + +* The realm-import lifecycle in Keycloak applies once. Without + `keycloak_sync.py`, every theme / SMTP / mapper change after first + import is silently ignored. +* Tag-pinned images can drift under the same tag. The deploy target + now `docker compose pull`s explicitly to defeat this. +* nginx + Keycloak both bind-mount files that `make activate` + regenerates → `--force-recreate` is on by default for those + services. +* `org_details.subdivision` MUST match `^[A-Z]{2}-[A-Z0-9]{1,3}$` or + the DS-Manager Organisation API rejects with a cryptic 422. + +## 10. Evidence the data space is operational (last verified 2026-04-29) + +* Federated DCAT catalogue from libelium → valencia DSP returns the + Albufera dataset. +* Contract negotiation `019dd641-…` reaches `FINALIZED`. +* Transfer process `019dd641-…` reaches `STARTED`; EDR endpoint + `http://cp-valencia:9006/api/public` issued with a 1-time bearer. +* Consumer `GET` of that endpoint returns the 2 022 698-byte CSV + (`dateObserved,numValue,…`) — i.e. the actual asset bytes flow + end-to-end through Eclipse EDC's DSP + data plane. diff --git a/docs/documentation/mv_data_space/eclipse/diagrams/architecture-overview.excalidraw b/docs/documentation/mv_data_space/eclipse/diagrams/architecture-overview.excalidraw new file mode 100644 index 00000000..111f1593 --- /dev/null +++ b/docs/documentation/mv_data_space/eclipse/diagrams/architecture-overview.excalidraw @@ -0,0 +1,52 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + {"type":"text","version":1,"versionNonce":1,"isDeleted":false,"id":"title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":20,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":700,"height":40,"seed":1,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":28,"fontFamily":1,"text":"CitCom.ai — Architecture overview","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"CitCom.ai — Architecture overview","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":2,"isDeleted":false,"id":"user-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":100,"strokeColor":"#1e1e1e","backgroundColor":"#e0f2fe","width":140,"height":80,"seed":2,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":3,"isDeleted":false,"id":"user-label","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":55,"y":120,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":110,"height":40,"seed":3,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"End user\n(browser)","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"End user\n(browser)","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":4,"isDeleted":false,"id":"traefik-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":250,"y":100,"strokeColor":"#1e1e1e","backgroundColor":"#fef3c7","width":160,"height":80,"seed":4,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":5,"isDeleted":false,"id":"traefik-label","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":265,"y":115,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":130,"height":50,"seed":5,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":14,"fontFamily":1,"text":"Traefik :443\nTLS termination\nLE DNS-01 (Cloudflare)","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"Traefik :443\nTLS termination\nLE DNS-01 (Cloudflare)","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":6,"isDeleted":false,"id":"main-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":480,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#dcfce7","width":380,"height":260,"seed":6,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":7,"isDeleted":false,"id":"main-title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":495,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":350,"height":24,"seed":7,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"Main domain — citcom.dataspaceready.eu","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Main domain — citcom.dataspaceready.eu","lineHeight":1.25}, + {"type":"text","version":1,"versionNonce":8,"isDeleted":false,"id":"main-content","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":500,"y":130,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":340,"height":200,"seed":8,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":14,"fontFamily":1,"text":"• nginx-main\n• onboarding-api (Flask + SMTP)\n• Keycloak (realm DSR)\n• Participant registry.json\n• sync-fuseki / sync-dome\n• Apache Jena Fuseki (DCAT/SPARQL)\n• Scorpio + TMForum API stack\n• BAE/DOME marketplace UI","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"• nginx-main\n• onboarding-api (Flask + SMTP)\n• Keycloak (realm DSR)\n• Participant registry.json\n• sync-fuseki / sync-dome\n• Apache Jena Fuseki (DCAT/SPARQL)\n• Scorpio + TMForum API stack\n• BAE/DOME marketplace UI","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":9,"isDeleted":false,"id":"admin-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":480,"y":380,"strokeColor":"#1e1e1e","backgroundColor":"#ffe4e6","width":380,"height":120,"seed":9,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":10,"isDeleted":false,"id":"admin-title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":495,"y":392,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":350,"height":24,"seed":10,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"admin.citcom.dataspaceready.eu","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"admin.citcom.dataspaceready.eu","lineHeight":1.25}, + {"type":"text","version":1,"versionNonce":11,"isDeleted":false,"id":"admin-content","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":500,"y":425,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":340,"height":80,"seed":11,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":14,"fontFamily":1,"text":"• nginx-admin\n• admin DS-Manager backend (compliance / sync centre)\n• cp-admin (EDC connector — federated requester)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"• nginx-admin\n• admin DS-Manager backend (compliance / sync centre)\n• cp-admin (EDC connector — federated requester)","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":12,"isDeleted":false,"id":"tenant-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":380,"strokeColor":"#1e1e1e","backgroundColor":"#dbeafe","width":380,"height":260,"seed":12,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":13,"isDeleted":false,"id":"tenant-title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":55,"y":392,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":350,"height":24,"seed":13,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"{tenant}.citcom.dataspaceready.eu (× N)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"{tenant}.citcom.dataspaceready.eu (× N)","lineHeight":1.25}, + {"type":"text","version":1,"versionNonce":14,"isDeleted":false,"id":"tenant-content","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":60,"y":425,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":340,"height":210,"seed":14,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":14,"fontFamily":1,"text":"• nginx-{tenant} + tenant-validator\n• dsr-frontend-{tenant} (DS-Manager UI by Hopu)\n• dsr-backend-{tenant} (FastAPI)\n• cp-{tenant}\n ┗ Eclipse EDC — Sovity-CE 16.4.2\n DSP :9084 · Mgmt :8181\n Public/data plane :9006\n• dsr-postgres-{tenant}\n• db-{tenant} (EDC Postgres)\n• dsr-redis-{tenant}","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"• nginx-{tenant} + tenant-validator\n• dsr-frontend-{tenant} (DS-Manager UI by Hopu)\n• dsr-backend-{tenant} (FastAPI)\n• cp-{tenant}\n ┗ Eclipse EDC — Sovity-CE 16.4.2\n DSP :9084 · Mgmt :8181\n Public/data plane :9006\n• dsr-postgres-{tenant}\n• db-{tenant} (EDC Postgres)\n• dsr-redis-{tenant}","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":15,"isDeleted":false,"id":"storage-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":900,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#f3e8ff","width":260,"height":140,"seed":15,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":16,"isDeleted":false,"id":"storage-title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":915,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":230,"height":24,"seed":16,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"Storage (shared)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Storage (shared)","lineHeight":1.25}, + {"type":"text","version":1,"versionNonce":17,"isDeleted":false,"id":"storage-content","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":920,"y":125,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":230,"height":90,"seed":17,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":14,"fontFamily":1,"text":"• MinIO (S3-compatible)\n• storage-portal\n (Flask + minio-py)\n• X-Participant header\n gates the bucket per tenant","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"• MinIO (S3-compatible)\n• storage-portal\n (Flask + minio-py)\n• X-Participant header\n gates the bucket per tenant","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":18,"isDeleted":false,"id":"federation-box","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":900,"y":260,"strokeColor":"#1e1e1e","backgroundColor":"#fef9c3","width":260,"height":120,"seed":18,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":19,"isDeleted":false,"id":"federation-title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":915,"y":272,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":230,"height":24,"seed":19,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"Federation","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Federation","lineHeight":1.25}, + {"type":"text","version":1,"versionNonce":20,"isDeleted":false,"id":"federation-content","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":920,"y":305,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":230,"height":70,"seed":20,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":14,"fontFamily":1,"text":"DSP — Eclipse Dataspace\nProtocol over HTTP\n(every cp-{tenant} ↔ cp-{any})","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"DSP — Eclipse Dataspace Protocol over HTTP","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":30,"isDeleted":false,"id":"a1","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":180,"y":140,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":68,"height":0,"seed":30,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startBinding":null,"endBinding":null,"lastCommittedPoint":null,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[68,0]]}, + {"type":"text","version":1,"versionNonce":31,"isDeleted":false,"id":"a1-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":190,"y":148,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":50,"height":18,"seed":31,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"HTTPS","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"HTTPS","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":32,"isDeleted":false,"id":"a2","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":410,"y":140,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":70,"height":0,"seed":32,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startBinding":null,"endBinding":null,"lastCommittedPoint":null,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[70,0]]}, + + {"type":"arrow","version":1,"versionNonce":33,"isDeleted":false,"id":"a3","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":330,"y":180,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":50,"height":200,"seed":33,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startBinding":null,"endBinding":null,"lastCommittedPoint":null,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[-100,200]]}, + {"type":"arrow","version":1,"versionNonce":34,"isDeleted":false,"id":"a4","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":330,"y":180,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":150,"height":200,"seed":34,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startBinding":null,"endBinding":null,"lastCommittedPoint":null,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[270,200]]}, + + {"type":"arrow","version":1,"versionNonce":35,"isDeleted":false,"id":"a5","fillStyle":"solid","strokeWidth":2,"strokeStyle":"dashed","roughness":1,"opacity":100,"angle":0,"x":420,"y":510,"strokeColor":"#7c3aed","backgroundColor":"transparent","width":60,"height":0,"seed":35,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startBinding":null,"endBinding":null,"lastCommittedPoint":null,"startArrowhead":"arrow","endArrowhead":"arrow","points":[[0,0],[60,-70]]}, + {"type":"text","version":1,"versionNonce":36,"isDeleted":false,"id":"a5-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":425,"y":485,"strokeColor":"#7c3aed","backgroundColor":"transparent","width":80,"height":18,"seed":36,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"DSP","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"DSP","lineHeight":1.25}, + + {"type":"text","version":1,"versionNonce":50,"isDeleted":false,"id":"legend","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":680,"strokeColor":"#525252","backgroundColor":"transparent","width":1100,"height":60,"seed":50,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Solid arrows = HTTP request inside the cluster. Dashed purple = DSP federation between EDC connectors (peer-to-peer over HTTP).\nThe Eclipse EDC stack lives inside each cp-{tenant} container. Sovity-CE 16.4.2 is the distribution; DS-Manager (Hopu) wraps the management UI.","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Solid arrows = HTTP request inside the cluster. Dashed purple = DSP federation between EDC connectors.\nThe Eclipse EDC stack lives inside each cp-{tenant} container.","lineHeight":1.25} + ], + "appState": { + "gridSize": null, + "viewBackgroundColor": "#ffffff" + }, + "files": {} +} diff --git a/docs/documentation/mv_data_space/eclipse/diagrams/deployment-topology.excalidraw b/docs/documentation/mv_data_space/eclipse/diagrams/deployment-topology.excalidraw new file mode 100644 index 00000000..3ce2c0f1 --- /dev/null +++ b/docs/documentation/mv_data_space/eclipse/diagrams/deployment-topology.excalidraw @@ -0,0 +1,57 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + {"type":"text","version":1,"versionNonce":1,"isDeleted":false,"id":"title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":20,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":900,"height":40,"seed":1,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":24,"fontFamily":1,"text":"CitCom — Deployment topology","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"CitCom — Deployment topology","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":2,"isDeleted":false,"id":"cf","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#fef3c7","width":260,"height":80,"seed":2,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":3,"isDeleted":false,"id":"cf-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":50,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":240,"height":60,"seed":3,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":13,"fontFamily":1,"text":"Cloudflare\n• Wildcard DNS *.dataspaceready.eu\n• Companion auto-creates DNS\n records from Traefik labels\n• DNS-01 ACME challenge for LE","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Cloudflare","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":4,"isDeleted":false,"id":"host","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":200,"strokeColor":"#1e1e1e","backgroundColor":"#f1f5f9","width":1100,"height":520,"seed":4,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":5,"isDeleted":false,"id":"host-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":55,"y":210,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":700,"height":24,"seed":5,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":16,"fontFamily":1,"text":"Host • citcom-ds • 145.239.127.76 • Ubuntu • ubuntu@~/citcom","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Host","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":6,"isDeleted":false,"id":"traefik","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":80,"y":250,"strokeColor":"#1e1e1e","backgroundColor":"#fef3c7","width":200,"height":70,"seed":6,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":7,"isDeleted":false,"id":"traefik-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":90,"y":262,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":180,"height":50,"seed":7,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":13,"fontFamily":1,"text":"Traefik :443\n+ cloudflare-companion\n+ LE DNS-01","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"Traefik","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":8,"isDeleted":false,"id":"nginx-main","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":320,"y":250,"strokeColor":"#1e1e1e","backgroundColor":"#dcfce7","width":110,"height":40,"seed":8,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":9,"isDeleted":false,"id":"nginx-main-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":325,"y":260,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":100,"height":20,"seed":9,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"nginx-main","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"nginx-main","lineHeight":1.25}, + {"type":"rectangle","version":1,"versionNonce":10,"isDeleted":false,"id":"nginx-admin","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":320,"y":300,"strokeColor":"#1e1e1e","backgroundColor":"#fee2e2","width":110,"height":40,"seed":10,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":11,"isDeleted":false,"id":"nginx-admin-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":325,"y":310,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":100,"height":20,"seed":11,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"nginx-admin","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"nginx-admin","lineHeight":1.25}, + {"type":"rectangle","version":1,"versionNonce":12,"isDeleted":false,"id":"nginx-mp","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":320,"y":350,"strokeColor":"#1e1e1e","backgroundColor":"#f3f4f6","width":110,"height":40,"seed":12,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":13,"isDeleted":false,"id":"nginx-mp-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":325,"y":360,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":100,"height":20,"seed":13,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"nginx-mailpit","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"nginx-mailpit","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":14,"isDeleted":false,"id":"nginx-lib","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":320,"y":410,"strokeColor":"#1e1e1e","backgroundColor":"#dbeafe","width":110,"height":40,"seed":14,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":15,"isDeleted":false,"id":"nginx-lib-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":325,"y":420,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":100,"height":20,"seed":15,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"nginx-libelium","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"nginx-libelium","lineHeight":1.25}, + {"type":"rectangle","version":1,"versionNonce":16,"isDeleted":false,"id":"nginx-val","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":320,"y":460,"strokeColor":"#1e1e1e","backgroundColor":"#dbeafe","width":110,"height":40,"seed":16,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":17,"isDeleted":false,"id":"nginx-val-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":325,"y":470,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":100,"height":20,"seed":17,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"nginx-valencia","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"nginx-valencia","lineHeight":1.25}, + {"type":"rectangle","version":1,"versionNonce":18,"isDeleted":false,"id":"nginx-upv","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":320,"y":510,"strokeColor":"#1e1e1e","backgroundColor":"#dbeafe","width":110,"height":40,"seed":18,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":19,"isDeleted":false,"id":"nginx-upv-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":325,"y":520,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":100,"height":20,"seed":19,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"nginx-upv","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"nginx-upv","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":20,"isDeleted":false,"id":"main-svcs","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":480,"y":250,"strokeColor":"#1e1e1e","backgroundColor":"#dcfce7","width":260,"height":140,"seed":20,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":21,"isDeleted":false,"id":"main-svcs-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":490,"y":260,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":240,"height":120,"seed":21,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Main-domain services\n• onboarding-api (Flask)\n• Keycloak + DB\n• Fuseki / Scorpio + TMForum API\n• sync-fuseki / sync-dome\n• tenant-validator (auth_request)\n• mailpit (SMTP debug)\n• Loki + Alloy (audit)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Main services","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":22,"isDeleted":false,"id":"admin-svcs","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":480,"y":410,"strokeColor":"#1e1e1e","backgroundColor":"#fee2e2","width":260,"height":80,"seed":22,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":23,"isDeleted":false,"id":"admin-svcs-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":490,"y":420,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":240,"height":60,"seed":23,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Admin connector\n• cp-admin (Eclipse EDC, Sovity-CE)\n• admin DS-Manager backend + DB","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Admin","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":24,"isDeleted":false,"id":"tenant-svcs","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":480,"y":510,"strokeColor":"#1e1e1e","backgroundColor":"#dbeafe","width":260,"height":160,"seed":24,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":25,"isDeleted":false,"id":"tenant-svcs-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":490,"y":520,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":240,"height":140,"seed":25,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Per-tenant stack (× N)\n• cp-{tenant} (Eclipse EDC)\n• db-{tenant} (EDC Postgres)\n• dsr-frontend-{tenant}\n• dsr-backend-{tenant}\n• dsr-postgres-{tenant}\n• dsr-redis-{tenant}\n• dsr-did-init-{tenant}","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Tenant","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":26,"isDeleted":false,"id":"storage","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":790,"y":250,"strokeColor":"#1e1e1e","backgroundColor":"#f3e8ff","width":300,"height":80,"seed":26,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":27,"isDeleted":false,"id":"storage-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":800,"y":260,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":280,"height":60,"seed":27,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Storage\n• MinIO (S3-compatible)\n• storage-portal (Flask, X-Participant gated)\n bucket per tenant","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Storage","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":28,"isDeleted":false,"id":"vols","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":790,"y":350,"strokeColor":"#1e1e1e","backgroundColor":"#fef9c3","width":300,"height":120,"seed":28,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":29,"isDeleted":false,"id":"vols-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":800,"y":360,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":280,"height":100,"seed":29,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Persistent state (Docker volumes)\n• traefik-acme (LE certs)\n• dsr-keycloak-db, dsr-redis-*\n• dsr-postgres-*, db-* (per-tenant + admin)\n• minio-data\n• registry-data (onboarding)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Volumes","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":30,"isDeleted":false,"id":"deploy","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":790,"y":490,"strokeColor":"#1e1e1e","backgroundColor":"#e0e7ff","width":300,"height":180,"seed":30,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":31,"isDeleted":false,"id":"deploy-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":800,"y":500,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":280,"height":160,"seed":31,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Deploy pipeline\n make deploy-prod-remote PROJECT=citcom\n ↓ rsync\n ↓ ssh: make activate (renders active/)\n ↓ docker compose pull (force-fresh)\n ↓ docker compose up -d (all stacks)\n ↓ force-recreate nginx-* + Keycloak\n ↓ keycloak_sync.py (self-heal)\n ↓ register-admin-connector.py","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Deploy","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":40,"isDeleted":false,"id":"a-cf","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":180,"y":160,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":0,"height":90,"seed":40,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[0,90]]}, + {"type":"text","version":1,"versionNonce":41,"isDeleted":false,"id":"a-cf-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":190,"y":190,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":120,"height":18,"seed":41,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"443/tcp","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"443/tcp","lineHeight":1.25} + ], + "appState": { + "gridSize": null, + "viewBackgroundColor": "#ffffff" + }, + "files": {} +} diff --git a/docs/documentation/mv_data_space/eclipse/diagrams/transfer-flow.excalidraw b/docs/documentation/mv_data_space/eclipse/diagrams/transfer-flow.excalidraw new file mode 100644 index 00000000..89bd123f --- /dev/null +++ b/docs/documentation/mv_data_space/eclipse/diagrams/transfer-flow.excalidraw @@ -0,0 +1,62 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + {"type":"text","version":1,"versionNonce":1,"isDeleted":false,"id":"title","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":20,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":900,"height":40,"seed":1,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":24,"fontFamily":1,"text":"CitCom — Transfer flow (libelium consumes valencia's Albufera)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"CitCom — Transfer flow (libelium consumes valencia's Albufera)","lineHeight":1.25}, + + {"type":"line","version":1,"versionNonce":2,"isDeleted":false,"id":"lane1","fillStyle":"solid","strokeWidth":2,"strokeStyle":"dashed","roughness":1,"opacity":100,"angle":0,"x":140,"y":120,"strokeColor":"#9ca3af","backgroundColor":"transparent","width":0,"height":700,"seed":2,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startBinding":null,"endBinding":null,"lastCommittedPoint":null,"startArrowhead":null,"endArrowhead":null,"points":[[0,0],[0,700]]}, + {"type":"line","version":1,"versionNonce":3,"isDeleted":false,"id":"lane2","fillStyle":"solid","strokeWidth":2,"strokeStyle":"dashed","roughness":1,"opacity":100,"angle":0,"x":340,"y":120,"strokeColor":"#9ca3af","backgroundColor":"transparent","width":0,"height":700,"seed":3,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":null,"points":[[0,0],[0,700]]}, + {"type":"line","version":1,"versionNonce":4,"isDeleted":false,"id":"lane3","fillStyle":"solid","strokeWidth":2,"strokeStyle":"dashed","roughness":1,"opacity":100,"angle":0,"x":620,"y":120,"strokeColor":"#9ca3af","backgroundColor":"transparent","width":0,"height":700,"seed":4,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":null,"points":[[0,0],[0,700]]}, + {"type":"line","version":1,"versionNonce":5,"isDeleted":false,"id":"lane4","fillStyle":"solid","strokeWidth":2,"strokeStyle":"dashed","roughness":1,"opacity":100,"angle":0,"x":900,"y":120,"strokeColor":"#9ca3af","backgroundColor":"transparent","width":0,"height":700,"seed":5,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":null,"points":[[0,0],[0,700]]}, + + {"type":"rectangle","version":1,"versionNonce":10,"isDeleted":false,"id":"actor1","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":80,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#dbeafe","width":120,"height":50,"seed":10,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":11,"isDeleted":false,"id":"actor1-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":85,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":110,"height":30,"seed":11,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":13,"fontFamily":1,"text":"libelium\nDS-Manager (consumer)","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"libelium\nDS-Manager (consumer)","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":12,"isDeleted":false,"id":"actor2","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":280,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#fef3c7","width":120,"height":50,"seed":12,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":13,"isDeleted":false,"id":"actor2-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":285,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":110,"height":30,"seed":13,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":13,"fontFamily":1,"text":"cp-libelium\n(EDC)","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"cp-libelium\n(EDC)","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":14,"isDeleted":false,"id":"actor3","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":560,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#fef3c7","width":120,"height":50,"seed":14,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":15,"isDeleted":false,"id":"actor3-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":565,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":110,"height":30,"seed":15,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":13,"fontFamily":1,"text":"cp-valencia\n(EDC)","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"cp-valencia\n(EDC)","lineHeight":1.25}, + + {"type":"rectangle","version":1,"versionNonce":16,"isDeleted":false,"id":"actor4","fillStyle":"hachure","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":840,"y":80,"strokeColor":"#1e1e1e","backgroundColor":"#f3e8ff","width":120,"height":50,"seed":16,"groupIds":[],"frameId":null,"roundness":{"type":3},"boundElements":[],"updated":1730000000000,"link":null,"locked":false}, + {"type":"text","version":1,"versionNonce":17,"isDeleted":false,"id":"actor4-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":845,"y":92,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":110,"height":30,"seed":17,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":13,"fontFamily":1,"text":"MinIO\n(Albufera CSV)","textAlign":"center","verticalAlign":"middle","containerId":null,"originalText":"MinIO\n(Albufera CSV)","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":20,"isDeleted":false,"id":"s1","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":140,"y":160,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":200,"height":0,"seed":20,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[200,0]]}, + {"type":"text","version":1,"versionNonce":21,"isDeleted":false,"id":"s1-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":140,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":190,"height":18,"seed":21,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"1) POST /catalog/request","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"1) POST /catalog/request","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":22,"isDeleted":false,"id":"s2","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":340,"y":190,"strokeColor":"#7c3aed","backgroundColor":"transparent","width":280,"height":0,"seed":22,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[280,0]]}, + {"type":"text","version":1,"versionNonce":23,"isDeleted":false,"id":"s2-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":350,"y":170,"strokeColor":"#7c3aed","backgroundColor":"transparent","width":260,"height":18,"seed":23,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"2) DSP catalog request (peer-to-peer EDC↔EDC)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"2) DSP catalog request (peer-to-peer EDC↔EDC)","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":24,"isDeleted":false,"id":"s3","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":620,"y":230,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":-480,"height":0,"seed":24,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[-480,0]]}, + {"type":"text","version":1,"versionNonce":25,"isDeleted":false,"id":"s3-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":210,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":460,"height":18,"seed":25,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"3) dcat:Catalog ← Albufera dataset (with offer policy id)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"3) dcat:Catalog ← Albufera dataset (with offer policy id)","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":26,"isDeleted":false,"id":"s4","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":140,"y":280,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":200,"height":0,"seed":26,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[200,0]]}, + {"type":"text","version":1,"versionNonce":27,"isDeleted":false,"id":"s4-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":260,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":190,"height":18,"seed":27,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"4) POST /contractnegotiations","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"4) POST /contractnegotiations","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":28,"isDeleted":false,"id":"s5","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":340,"y":310,"strokeColor":"#7c3aed","backgroundColor":"transparent","width":280,"height":0,"seed":28,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[280,0]]}, + {"type":"text","version":1,"versionNonce":29,"isDeleted":false,"id":"s5-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":350,"y":290,"strokeColor":"#7c3aed","backgroundColor":"transparent","width":260,"height":18,"seed":29,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"5) DSP contract request → AGREED → VERIFIED → FINALIZED","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"5) DSP contract request → AGREED → VERIFIED → FINALIZED","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":30,"isDeleted":false,"id":"s6","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":140,"y":380,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":200,"height":0,"seed":30,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[200,0]]}, + {"type":"text","version":1,"versionNonce":31,"isDeleted":false,"id":"s6-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":360,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":190,"height":18,"seed":31,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"6) POST /transferprocesses (HttpData-PULL)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"6) POST /transferprocesses (HttpData-PULL)","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":32,"isDeleted":false,"id":"s7","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":620,"y":410,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":-480,"height":0,"seed":32,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[-480,0]]}, + {"type":"text","version":1,"versionNonce":33,"isDeleted":false,"id":"s7-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":390,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":460,"height":18,"seed":33,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"7) EDR ← {endpoint http://cp-valencia:9006/api/public, bearer, expiry}","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"7) EDR ← {endpoint, bearer, expiry}","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":34,"isDeleted":false,"id":"s8","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":140,"y":470,"strokeColor":"#16a34a","backgroundColor":"transparent","width":480,"height":0,"seed":34,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[480,0]]}, + {"type":"text","version":1,"versionNonce":35,"isDeleted":false,"id":"s8-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":450,"strokeColor":"#16a34a","backgroundColor":"transparent","width":460,"height":18,"seed":35,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"8) GET endpoint with Authorization: Bearer …","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"8) GET endpoint with Authorization: Bearer","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":36,"isDeleted":false,"id":"s9","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":620,"y":510,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":280,"height":0,"seed":36,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":null,"endArrowhead":"arrow","points":[[0,0],[280,0]]}, + {"type":"text","version":1,"versionNonce":37,"isDeleted":false,"id":"s9-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":630,"y":490,"strokeColor":"#1e1e1e","backgroundColor":"transparent","width":260,"height":18,"seed":37,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"9) read object via dataAddress (S3/HTTP)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"9) read object via dataAddress","lineHeight":1.25}, + + {"type":"arrow","version":1,"versionNonce":38,"isDeleted":false,"id":"s10","fillStyle":"solid","strokeWidth":2,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":140,"y":580,"strokeColor":"#16a34a","backgroundColor":"transparent","width":760,"height":0,"seed":38,"groupIds":[],"frameId":null,"roundness":{"type":2},"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"startArrowhead":"arrow","endArrowhead":null,"points":[[0,0],[760,0]]}, + {"type":"text","version":1,"versionNonce":39,"isDeleted":false,"id":"s10-l","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":150,"y":560,"strokeColor":"#16a34a","backgroundColor":"transparent","width":700,"height":18,"seed":39,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"10) CSV bytes streamed back to libelium DS-Manager (Albufera water level — 2 022 698 bytes)","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"10) CSV bytes streamed back","lineHeight":1.25}, + + {"type":"text","version":1,"versionNonce":40,"isDeleted":false,"id":"legend","fillStyle":"solid","strokeWidth":1,"strokeStyle":"solid","roughness":1,"opacity":100,"angle":0,"x":40,"y":640,"strokeColor":"#525252","backgroundColor":"transparent","width":900,"height":80,"seed":40,"groupIds":[],"frameId":null,"roundness":null,"boundElements":[],"updated":1730000000000,"link":null,"locked":false,"fontSize":12,"fontFamily":1,"text":"Legend:\n black arrows = HTTP inside the cluster (DS-Manager ↔ EDC management API, MinIO read).\n purple arrows = DSP — Eclipse Dataspace Protocol — peer-to-peer EDC ↔ EDC.\n green arrows = data-plane traffic (the actual asset bytes).","textAlign":"left","verticalAlign":"top","containerId":null,"originalText":"Legend","lineHeight":1.25} + ], + "appState": { + "gridSize": null, + "viewBackgroundColor": "#ffffff" + }, + "files": {} +} diff --git a/docs/documentation/mv_data_space/eclipse/index.md b/docs/documentation/mv_data_space/eclipse/index.md new file mode 100644 index 00000000..536ee17c --- /dev/null +++ b/docs/documentation/mv_data_space/eclipse/index.md @@ -0,0 +1,63 @@ +# CitCom.ai — Data Space Technical Documentation + +This folder is the canonical technical documentation for the **CitCom.ai +data space**, deployed at `https://citcom.dataspaceready.eu`. + +It describes the **Eclipse Dataspace Components (EDC)** stack we run, the +deployment topology, and the data flows between participants. + +## How to view the diagrams + +The docs use two diagram formats. **Pick whichever fits where you're +reading from**: + +* **Mermaid blocks** embedded in `architecture.md` and `deployment.md` + (` ```mermaid `…` ``` ` fenced code). GitLab and GitHub render these + inline in the browser — **just open the `.md` file in the web UI** + and the diagrams appear in the right place. No extra tooling. + +* **Excalidraw sources** under [`diagrams/`](diagrams/) — same content + as the inline Mermaid, but as an editable canvas. To open them: + + | You're using… | What to do | + |---|---| + | **VS Code** | Install the [`pomdtr.excalidraw-editor`](https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor) extension and double-click the `.excalidraw` file. | + | **Cursor / JetBrains / Vim** | Same extension on VS Code is the easiest path; alternatively download the file and use the web option below. | + | **A browser** | Go to , then *Menu → Open* and pick the `.excalidraw` file from your local clone. Or paste the file content via *Menu → Load from URL* against the GitLab raw link. | + | **GitLab web UI** | The file shows up as JSON — that's expected. GitLab has no native preview for Excalidraw; use one of the options above to view the canvas. | + + If you only need to look at them (no editing), the Mermaid versions + in the `.md` files are usually enough. + +## Contents + +| Doc | Audience | What it covers | +|---|---|---| +| [`architecture.md`](architecture.md) | Tech reviewers, integrators | Component model, Eclipse EDC role, federated catalog, identity, transfer protocol (DSP). | +| [`deployment.md`](deployment.md) | DevOps, ops handover | Server topology, deploy pipeline, provisioning, self-healing, troubleshooting. | +| [`diagrams/`](diagrams/) | All | Excalidraw sources for the inline diagrams. Open with `pomdtr.excalidraw-editor` (VS Code) or `excalidraw.com`. | + +## Quick context + +**CitCom.ai** is one of four production data spaces built on the same +monorepo (`Sovity-monorepo`); the other three are Geo4Water, +GeoSpaceData, and BeatTheHeat. The platform is intentionally +project-agnostic: each data space is a thin per-project overlay +(`projects//`) on top of a shared `platform/` core. + +The **Eclipse stack** — that is, the open-source Eclipse Dataspace +Components (EDC) — is the bedrock of the connector layer. CitCom uses +the Sovity Community Edition 16.4.2 distribution of Eclipse EDC +(`ghcr.io/sovity/edc-ce:16.4.2`), which adds management UIs and +opinionated defaults but is upstream-compatible. The architecture +document goes into detail on which EDC modules we rely on (DSP catalog, +contract negotiation, transfer process, EDR proxy). + +## How to read this + +* If you came here to **understand what CitCom is and how the pieces + fit**, start with `architecture.md`. +* If you came here to **deploy, redeploy, or troubleshoot**, jump to + `deployment.md`. +* The diagrams are the same ones embedded in the markdown — open them + in Excalidraw to edit. diff --git a/mkdocs.yml b/mkdocs.yml index 586d9657..b767a733 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -63,6 +63,10 @@ nav: - documentation/mv_data_space/fiware/trust_anchor.md - documentation/mv_data_space/fiware/consumer.md - documentation/mv_data_space/fiware/provider.md + - Eclipse: + - documentation/mv_data_space/eclipse/index.md + - Architecture: documentation/mv_data_space/eclipse/architecture.md + - Deployment: documentation/mv_data_space/eclipse/deployment.md - documentation/mv_data_space/index.md - Local Digital Twins: - documentation/local_digital_twins/index.md diff --git a/requirements.txt b/requirements.txt index 7388d576..f500764b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,14 +1,16 @@ # Documentation static site generator & deployment tool -mkdocs==1.6.1 +Markdown<3.6 +mkdocs==1.5.3 +mkdocs-material==9.5 +Pygments==2.19.2 # Extensions -pymdown-extensions==10.14.3 +pymdown-extensions==10.7 mkdocs-git-revision-date-localized-plugin==1.4.5 neoteroi-mkdocs==1.1.0 mkdocs-macros-plugin==1.3.7 mkdocs-swagger-ui-tag==0.7.1 # Theme -mkdocs-material==9.6.9 mkdocs-git-authors-plugin==0.9.4 mkdocs-glightbox==0.4.0 \ No newline at end of file