Distributed health-data platform

CnesData

A contract-led platform for moving municipal source extracts from edge environments into a tenant-isolated central data layer.

  • Python
  • FastAPI
  • Go
  • PostgreSQL
  • Parquet
  • Docker
  • React
CnesData public repository ↗Architecture documentation ↗

01 / Context

Context & contribution

Problem

Municipal source systems produce operational data at the edge, while reconciliation and downstream analysis need stable contracts, source provenance, and an explicit boundary between tenants.

Context

CnesData is developed as a public software platform. The repository demonstrates the application boundaries and local development path; this case study does not claim a municipal rollout, production scale, or processing results.

Contribution

  • Defined canonical domain contracts and generated schemas shared by clients and services.
  • Built the Go edge-agent path for source extraction, compressed Parquet files, and manifest registration.
  • Implemented central API workflows for orchestration, device activation, access, and dashboard reads.
  • Kept tenant context explicit across authentication, persistence, and row-level isolation.
  • Developed the web dashboard around typed API contracts and operational state.

02 / System boundaries

System View

Conceptual architecture map for public discussion. Connections describe system responsibilities rather than a live deployment; processing and deployment boundaries remain illustrative.

System map

Central API is the branching hub between contracts, clients, and downstream boundaries.

Component details

Shared models and exported schemas establish one vocabulary for landing work, clients, and downstream services.

Sends to
  • Defines extraction payloads
  • Defines API contracts

Edge extraction writes compressed Parquet artifacts and registers multi-file manifests with the central platform.

Receives from
  • Defines extraction payloads
Sends to
  • Registers extraction manifests

A FastAPI service coordinates activation, extraction requests, manifest registration, and landing metadata.

Receives from
  • Defines API contracts
  • Registers extraction manifests
  • Uses typed API contracts
  • Shows the deployment shape
Sends to
  • Carries tenant context
  • Creates a processing-work boundary

Tenant context is carried through authorization and PostgreSQL row-level security boundaries.

Receives from
  • Carries tenant context

The typed dashboard presents activation, agent state, access, and overview workflows.

Sends to
  • Uses typed API contracts

Worker foundations exist, but the full landing-to-Gold processing path is not presented as complete.

Receives from
  • Creates a processing-work boundary

Kubernetes is a target deployment shape rather than a published production claim.

Sends to
  • Shows the deployment shape

03 / Interactive scenario

Simulation

Synthetic demonstration · Illustrative

Raw-object contract walkthrough

An illustrative raw-object contract using fictional identifiers and content. It does not represent the full current CnesData API or measured operational results.

No backend calls, mTLS authentication, DBC conversion, Parquet generation or real health data. Steps do not measure latency. These results are separate from the conceptual architecture map above.

Interactive controls require JavaScript. All three complete transcripts are available below.

0 of 1 attempts · ready

No attempts yet.

Stored objects

  • No objects stored.

Attempts

  1. No attempts yet.

Static transcripts

Every step is rendered with the page and stays available without JavaScript.

First write to a new key

Synthetic demonstration · Illustrative. Initial state: no stored objects.

  1. Stored synthetic-content-A at synthetic-key-K. One object exists.

    Result: stored. Stored objects: 1.

    • synthetic-key-K → synthetic-content-A

Identical replay without duplication

Synthetic demonstration · Illustrative. Initial state: no stored objects.

  1. Stored synthetic-content-A at synthetic-key-K. One object exists.

    Result: stored. Stored objects: 1.

    • synthetic-key-K → synthetic-content-A

  2. Accepted identical replay of synthetic-content-A at synthetic-key-K. The stored object is unchanged; no duplicate was created.

    Result: replayed. Stored objects: 1.

    • synthetic-key-K → synthetic-content-A

Content conflict preserves the first write

Synthetic demonstration · Illustrative. Initial state: no stored objects.

  1. Stored synthetic-content-A at synthetic-key-K. One object exists.

    Result: stored. Stored objects: 1.

    • synthetic-key-K → synthetic-content-A

  2. Rejected conflicting synthetic-content-B at synthetic-key-K. The originally stored content remains intact.

    Result: conflict. Stored objects: 1.

    • synthetic-key-K → synthetic-content-A

04 / Technical choices

Engineering

Decisions

  • Use generated contracts to reduce drift between services instead of allowing each application to define payloads independently.
  • Register a manifest for a complete extraction so file membership and storage references remain explicit.
  • Preserve tenant context at the storage boundary rather than relying only on application-level filtering.
  • Keep audit-rule execution outside this repository and expose stable data contracts for downstream consumers.

Reliability

  • Contract and OpenAPI artifacts can be regenerated from source models and checked for drift.
  • Landing work has explicit states and ownership boundaries so incomplete processing is visible.
  • Local development composes the API, object storage, identity provider, database, agent, and dashboard without implying production deployment.

05 / Outcomes

Results

Outcomes

  • The public implementation provides a coherent path from edge extraction to tenant-scoped landing records.
  • Shared contracts make incomplete and planned portions visible instead of hiding them behind broad platform claims.

06 / Public record

Evidence & limitations

Limitations

  • Full Parquet-to-Gold processing remains planned and is not described as complete.
  • Kubernetes remains a target deployment model; this page makes no production availability or scale claim.
  • No municipal records, health data, or quantitative operational results are published here.

Public evidence

Implemented

CnesData public repository

Source code, contracts, application boundaries, tests, and current-status documentation.

Open public evidence ↗