A FastAPI-based REST and FIWARE NGSI-LD service that provides a single, authorisation-enforced data access layer over a ThingsBoard-managed IoT deployment. The CDS API exposes asset, device, telemetry, building, market-price and per-device flexibility data, with hierarchical, attribute-based access control delegated to a Keycloak identity provider.
IoT platforms such as ThingsBoard provide rich device-management and data-ingestion capabilities, but their open-source editions typically lack the fine-grained, hierarchy-aware access control needed for multi-tenant or multi-stakeholder deployments — e.g. campuses, energy districts or smart-city installations where building managers should see only their own rooms, energy analysts should see only the buildings they are responsible for, and external service consumers should only see explicitly authorised slices of the data.
Bolting access control onto each downstream consumer is fragile, inconsistent and hard to audit. The CDS API addresses this gap by sitting between the IoT platform and its consumers, enforcing a single hierarchical access-control model rooted in Keycloak claims, and exposing the data through both a developer-friendly native REST surface and a FIWARE NGSI-LD compliant facade for cross-platform interoperability with smart-city deployments.
The pattern can be applied to any deployment that uses ThingsBoard (or a similar IoT platform with an asset/device/telemetry data model) together with Keycloak for identity, where multiple distinct consumers need read access to overlapping subsets of the same data.
- Authenticated, hierarchy-aware data access. Every protected endpoint enforces a Keycloak-issued, hierarchical permissions model with * and ** wildcards. Users with access to one floor or one room see only that part of the building; users with access to a whole building see the building plus all its sub-assets.
- Telemetry retrieval. Latest values, latest-before-a-timestamp, sliding-window history (1–24 hours) and per-device history (up to 168 hours), with optional sub-asset filtering and an optional FIWARE NGSI-LD output format.
- Asset, device and building metadata. Hierarchical asset trees, device attributes and relations, building catalogues with optional city filtering, and a permission-filtered drilldown that returns to a calling user only the part of a building's hierarchy they are entitled to see.
- FIWARE NGSI-LD facade. Standards-compliant /entities query (with the q filter language and keyValues/normalized formats), entity retrieval by URN, temporal API for device telemetry history, and a custom nested-hierarchy extension. Same access model as the native endpoints.
- Market and flexibility integrations. Day-ahead electricity prices retrieved from the ENTSO-E Transparency Platform and persisted locally, plus per-device flexibility forecasts produced by an upstream modelling pipeline, both exposed through stable schema-defined endpoints.
- Operator administration surface. User CRUD against Keycloak, in-memory hierarchy cache with on-demand refresh, what-if permission preview, server-side telemetry aggregations (windowed avg/min/max/sum/count/first/last), per-user dashboards and preferences, audit and error logs.
- Tamper-evident audit trail. All state-changing administrative actions are recorded in an append-only audit_logs table with a SHA-256 hash chain enforced by database triggers; an on-demand verifier function detects tampering even by an actor with the application's database role.
- Operational hygiene. Stateless JWT validation against the published JWKS, per-request permission caching, hierarchy cache for sub-millisecond authorisation under realistic permission sets, structured error logging with paginated administrative access, idempotent application-database schema creation on startup, and Kubernetes-ready deployment with secret-driven configuration.
The application is organised as a single FastAPI process exposing a versioned base path (/cds-api/v2) and a set of routers, each responsible for one concern:
- telemetry/ — latest, latest-before, history and history-before queries at building, sub-asset and per-device granularity.
- assets/, devices/, buildings/ — hierarchy and metadata retrieval, with classifications (building/floor/room) inferred at response time.
- fiware/ — NGSI-LD compliant /entities, /entities/{urn} and /temporal/entities/{urn} endpoints, plus a custom nested-hierarchy extension.
- energy_prices/ — ENTSO-E day-ahead price retrieval, persisted in the application database via a synchronisation routine.
- flexibility/ — per-device flexibility forecasts produced by an upstream modelling pipeline, persisted in the application database.
- services/ — service-account endpoints gated by realm roles for trusted backend integrations.
- auth/ — request-level dependencies that decode the bearer token, look up the user's permissions, and authorise the request against the requested resource.
- admin/ — user-management, hierarchy-cache, audit-log and operator-facing endpoints (used by a companion administrative UI).
Authorisation is implemented as a small set of FastAPI dependencies (require_authenticated, require_building_access, require_device_access, require_realm_role) declared once at the router level. The dependencies validate the JWT against the published JWKS, extract the hierarchical permissions claim, and either grant the request, grant it with a per-resource access filter (so the same handler can serve full-access and partial-access users without bespoke code paths), or reject with HTTP 403.
Two PostgreSQL databases are used: the ThingsBoard database (read-only) for the underlying asset, device and telemetry data, and an application-owned configuration database for user preferences, dashboards, audit and error logs, ENTSO-E prices and per-device flexibility forecasts. Schema for the second database is created idempotently on application startup.
A companion Kafka consumer can be enabled in the same image (different entry point) to ingest forwarded telemetry from upstream services.
The CDS API exists to give every consumers one consistent, secure and standards-aligned way of reading the BMS & sensor data the underlying platform holds, without each consumer having to learn the internal data model, manage its own access-control logic, or talk directly to the database. Its role is twofold: (1) abstract the underlying data sources behind a stable, versioned API surface, and (2) enforce a uniform, hierarchical authorisation model so that what each user can see is decided centrally by Keycloak rather than re-implemented by each application.
- Runtime: Python 3.11+, FastAPI, Uvicorn (ASGI), psycopg2.
- Identity provider: Keycloak (OAuth 2.0 / OpenID Connect), JWKS-based JWT validation, RS256 signatures.
- Token-claim authorisation: hierarchical URI patterns with * (one level) and ** (any depth) wildcards, projected as a custom JWT claim by Keycloak and validated stateless on each request.
- Data sources: PostgreSQL (ThingsBoard's primary database, plus a separate application-config database), Apache Kafka (optional consumer for forwarded telemetry / fused outputs), MinIO (S3-compatible object storage, optional, for large simulation artefacts), ENTSO-E Transparency Platform (day-ahead market data integration).
- Standards: FIWARE NGSI-LD (entities, temporal API, query language q), Smart Data Models for Building, Device, BuildingSpace.
- Operational: Docker, Kubernetes (Deployment + Service + Ingress), GitLab CI for image build and rollout, structured error log persisted in the application database, append-only audit log with SHA-256 hash chain and database-level append-only triggers.
- Testing: pytest with FastAPI's TestClient, mocked Keycloak client for unit tests, optional integration tests against a real Keycloak instance.