From e28ad5fbf679996d4102e00b380d800e1b22309b Mon Sep 17 00:00:00 2001 From: Diego Ripley Date: Wed, 18 Feb 2026 09:28:34 -0500 Subject: [PATCH] Improve homepage high-level overview. It was not scaling properly before, but now it is SVG since it is more stable than before --- content/_index.md | 265 ++---------------------- high-level-overview-homepage.mermaid | 247 ++++++++++++++++++++++ static/high-level-overview-homepage.svg | 1 + 3 files changed, 267 insertions(+), 246 deletions(-) create mode 100644 high-level-overview-homepage.mermaid create mode 100644 static/high-level-overview-homepage.svg diff --git a/content/_index.md b/content/_index.md index 7eebdf9..f5f917c 100644 --- a/content/_index.md +++ b/content/_index.md @@ -63,255 +63,28 @@ See our processing pipeline in action. View samples and documentation for our cu **Note:** The data sources in the diagram below are **prioritized from left to right**, reflecting our current focus on processing high-value [statistical](https://www.dataforcanada.org/docs/processes/statistical_products/), [foundational](https://www.dataforcanada.org/docs/processes/foundation/#download-and-preview), and [orthoimagery](https://www.dataforcanada.org/docs/processes/orthoimagery/#download-and-preview) datasets first. -```mermaid -flowchart TD - classDef linkNode stroke:#0000EE,color:#0000EE,stroke-width:2px; - - %% --------------------------------------------------------- - %% 1. DATA SOURCES - %% --------------------------------------------------------- - subgraph ds [Data Sources] - Statistical@{ shape: lean-l} - Foundation@{ shape: lean-l} - Orthoimagery@{ shape: lean-l} - EnvironmentClimate@{ shape: lean-l, label: "Environment, Climate, & Health"} - FieldImagery@{ shape: lean-l, label: "Field Imagery"} - Elevation@{ shape: lean-l} - WebCorpus@{ shape: lean-l, label: "Web Corpus"} - end +{{< rawhtml >}} + +{{< /rawhtml >}} - %% Intermediate format (Standalone) - FlatGeoBuf@{ shape: lean-l} - - %% Box: Vector Tiles (Pastel Orange) - subgraph vt [Vector Tiles] - VectorTiles@{ shape: lean-l, label: "Mapbox Vector Tiles"} - NextGenVectorTiles@{ shape: lean-l, label: "Next-Gen Vector Tiles"} - end - - %% Box: Visuals (Pastel Blue - No Name) - subgraph visuals [" "] - WebP@{ shape: lean-l} - JPG@{ shape: lean-l} - PNG@{ shape: lean-l} - end - - %% Box: Portable Databases (Pastel Green) - subgraph pkg [Portable Databases] - PMTiles@{ shape: lean-l} - SQLite@{ shape: lean-l} - end - - %% Box: Enterprise (Pastel Purple) - subgraph ent [Enterprise] - FileGeodatabase@{shape: lean-l, label: "File Geodatabase"} - end - end - - %% --------------------------------------------------------- - %% 5. DISTRIBUTION INFRASTRUCTURE - %% --------------------------------------------------------- - subgraph di [Distribution Infrastructure] - ObjectStorage@{ shape: bow-rect, label: "Object Storage"} - Metadata@{ shape: rect, label: "FAIR Data Catalogue"} - HTTP@{ shape: rect, label: "Systems-Ready Data"} - DecentralizedDistribution@{ shape: rect, label: "Decentralized Distribution"} - end - - %% --------------------------------------------------------- - %% 6. EXPERIMENTAL INFRASTRUCTURE - %% --------------------------------------------------------- - subgraph ei [Experimental Infrastructure] - GeoSpatialServices@{ shape: rect, label: "Geospatial Services"} - end - - %% --------------------------------------------------------- - %% 7. CONSUMPTION - %% --------------------------------------------------------- - subgraph "Consumption" - DataSci@{ shape: rect, label: "Data People & Developers"} - Systems@{ shape: rect, label: "Systems"} - end - - %% ========================================================= - %% RELATIONSHIPS - %% ========================================================= - - %% Data Sources <--> Data for Canada Packages Collection (Box) - Statistical a1@<--> pp - a1@{animate: true, animation: slow} - Foundation a2@<--> pp - a2@{animate: true, animation: slow} - Orthoimagery a3@<--> pp - a3@{animate: true, animation: slow} - EnvironmentClimate a5@<--> pp - a5@{animate: true, animation: fast} - FieldImagery a4@<--> pp - a4@{animate:true, animation: fast} - Elevation a6@<--> pp - a6@{animate: true, animation: slow} - WebCorpus a7@<--> pp - a7@{animate: true, animation: fast} - - pp a10@--> df - a10@{animate: true, animation: fast} - - %% Long-Term Storage --> FlatGeoBuf - sot a10000@<--> FlatGeoBuf - a10000@{animate: true, animation: fast} - - %% FlatGeoBuf --> Vector Tiles (Box) - FlatGeoBuf a11@--> vt - a11@{animate: true, animation: fast} - - %% Long-Term Storage <--> Visuals (Box) - sot a12@<--> visuals - a12@{animate: true, animation: slow} - - %% Vector Tiles --> Portable Databases (Box) - vt a90@<--> pkg - a90@{animate: true, animation: fast} - - %% Visuals --> Portable Databases (Box) - visuals a93@<--> pkg - a93@{animate: true, animation: slow} - - %% Long-Term Storage --> Enterprise (Box) - sot a100@<--> ent - a100@{animate: true, animation: slow} - - %% Visuals --> Enterprise (Box) - visuals a102@--> ent - a102@{animate: true, animation: slow} - - %% Dissemination Formats --> Distribution Infrastructure - df a13@<--> di - a13@{animate: true, animation: slow} - - %% Distribution Infrastructure Flow - ObjectStorage a15@<--> Metadata - a15@{animate: true, animation: slow} - Metadata a16@<--> HTTP - a16@{animate: true, animation: slow} - - HTTP a17@<--> ei - a17@{animate: true, animation: slow} - HTTP a18@<--> DecentralizedDistribution - a18@{animate: true, animation: slow} - HTTP a19@<--> DataSci - a19@{animate: true, animation: slow} - - DecentralizedDistribution a20@--> Systems - a20@{animate: true, animation: fast} - DecentralizedDistribution a21@--> DataSci - a21@{animate: true, animation: fast} - - Systems a22@ <--> DataSci - a22@{animate: true, animation: fast} - ei a23@ <--> DataSci - a23@{animate: true, animation: slow} - - %% ========================================================= - %% STYLING - %% ========================================================= - -classDef linkNode stroke:#333333,color:#333333,stroke-width:1.5px; - -%%style pp fill:#D32F2F,stroke:#8E0000,color:#FFFFFF -style DataforCanadaPackagesCollection fill:#B71C1C,stroke:#7F0000,color:#FFFFFF -style FAIRDataDis fill:#B71C1C,stroke:#7F0000,color:#FFFFFF -style DecentralizedDistribution fill:#B71C1C,stroke:#7F0000,color:#FFFFFF -style HTTP fill:#B71C1C,stroke:#7F0000,color:#FFFFFF -style Systems fill:#B71C1C,stroke:#7F0000,color:#FFFFFF -style Metadata fill:#B71C1C,stroke:#7F0000,color:#FFFFFF - -%%style df fill:#D32F2F,stroke:#8E0000,color:#FFFFFF -style sot fill:#EF9A9A,stroke:#C62828,color:#000000 - -style Parquet fill:#FFCDD2,stroke:#E57373,color:#000000 -style Zarr fill:#FFCDD2,stroke:#E57373,color:#000000 -style GeoTIFF fill:#FFCDD2,stroke:#E57373,color:#000000 -style JPEGXL fill:#FFCDD2,stroke:#E57373,color:#000000 -style WARC fill:#FFCDD2,stroke:#E57373,color:#000000 -style AV1 fill:#FFCDD2,stroke:#E57373,color:#000000 - -style ds fill:#FB8C00,stroke:#E65100,color:#000000 - -style pkg fill:#FFB74D,stroke:#EF6C00,color:#000000 -style SQLite fill:#EF6C00,stroke:#E65100,color:#000000 -style PMTiles fill:#FFCC80,stroke:#FB8C00,color:#000000 - -style vt fill:#FBC02D,stroke:#F9A825,color:#000000 -style FlatGeoBuf fill:#FBC02D,stroke:#F9A825,color:#000000 - -style visuals fill:#FBC02D,stroke:#F9A825,color:#000000 - -style ent fill:#66BB6A,stroke:#2E7D32,color:#000000 -style DataSci fill:#D32F2F,stroke:#8E0000,color:#FFFFFF -style GeoSpatialServices fill:#FFCC80,stroke:#FB8C00,color:#000000 - - -class Foundation,Statistical,Orthoimagery,FieldImagery,EnvironmentClimate,Elevation,WebCorpus linkNode -class Parquet,FlatGeoBuf,SQLite,FileGeodatabase,VectorTiles,NextGenVectorTiles,GeoTIFF,Zarr,WebP,PMTiles,JPEGXL,AV1,WARC linkNode - - %% ========================================================= - %% CLICK ACTIONS - %% ========================================================= - click DataforCanadaPackagesCollection "https://github.com/dataforcanada/dataforcanadapkgs-labs/" _blank - - click Foundation "https://github.com/dataforcanada/process-foundation-labs/" _blank - click Statistical "https://github.com/dataforcanada/process-statistical-labs/" _blank - click Orthoimagery "https://github.com/dataforcanada/process-orthoimagery-labs/" _blank - click FieldImagery "https://github.com/dataforcanada/process-field-imagery-labs/" _blank - click EnvironmentClimate "https://github.com/dataforcanada/process-environmental-climate-health-labs/" _blank - click Elevation "https://github.com/dataforcanada/process-elevation-labs/" _blank - click WebCorpus "https://github.com/dataforcanada/process-web-corpus-labs/" _blank - - click Parquet "https://github.com/apache/parquet-format/" _blank - click FlatGeoBuf "https://flatgeobuf.org/" _blank - click SQLite "https://www.geopackage.org/" _blank - click FileGeodatabase "https://gdal.org/en/stable/drivers/vector/openfilegdb.html" _blank - click VectorTiles "https://github.com/mapbox/vector-tile-spec/" _blank - click NextGenVectorTiles "https://github.com/maplibre/maplibre-tile-spec/" _blank - click GeoTIFF "https://cogeo.org/" _blank - click Zarr "https://github.com/zarr-developers/geozarr-spec/" _blank - click WebP "https://developers.google.com/speed/webp/" _blank - click PMTiles "https://github.com/protomaps/PMTiles/blob/main/spec/v3/spec.md" _blank - click JPEGXL "https://jpeg.org/jpegxl/" _blank - click AV1 "https://aomedia.org/specifications/av1/" _blank - click WARC "https://github.com/iipc/warc-specifications/" _blank - click FAIRDataDis "https://stac-utils.github.io/stac-geoparquet/latest/spec/stac-geoparquet-spec/" _blank - - click HTTP "https://www.dataforcanada.org/docs/" _blank - click DecentralizedDistribution "https://www.dataforcanada.org/docs/dissemination/" _blank - click Metadata "https://stac-utils.github.io/stac-geoparquet/latest/spec/stac-geoparquet-spec/" _blank - click GeoSpatialServices "https://github.com/dataforcanada/geo-services-labs/" _blank -``` +{{< svg src="static/high-level-overview-homepage.svg" alt="My Mermaid Diagram" >}} ## Get Involved diff --git a/high-level-overview-homepage.mermaid b/high-level-overview-homepage.mermaid new file mode 100644 index 0000000..03f9b45 --- /dev/null +++ b/high-level-overview-homepage.mermaid @@ -0,0 +1,247 @@ +flowchart TD + classDef linkNode stroke:#0000EE,color:#0000EE,stroke-width:2px; + + %% --------------------------------------------------------- + %% 1. DATA SOURCES + %% --------------------------------------------------------- + subgraph ds [Data Sources] + Statistical@{ shape: lean-l} + Foundation@{ shape: lean-l} + Orthoimagery@{ shape: lean-l} + EnvironmentClimate@{ shape: lean-l, label: "Environment, Climate, & Health"} + FieldImagery@{ shape: lean-l, label: "Field Imagery"} + Elevation@{ shape: lean-l} + WebCorpus@{ shape: lean-l, label: "Web Corpus"} + end + + %% --------------------------------------------------------- + %% 3. PROCESSING PIPELINE + %% --------------------------------------------------------- + subgraph pp [Processing Pipeline] + %% Not the orchestrator, but a key towards achieving project mission. + DataforCanadaPackagesCollection@{ shape: rect, label: "Data for Canada Packages Collection"} + %% Internal Link + end + + %% --------------------------------------------------------- + %% 4. DISSEMINATION FORMATS + %% --------------------------------------------------------- + subgraph df [Dissemination Formats] + + %% Box: Long-Term Storage (Pastel Gold) + subgraph sot [Long-Term Storage] + Parquet@{ shape: lean-l} + Zarr@{ shape: lean-l} + GeoTIFF@{ shape: lean-l} + AV1@{ shape: lean-l, label: "Next-Gen Video"} + JPEGXL@{ shape: lean-l, label: "Next-Gen Imagery"} + WARC@{ shape: lean-l, label: "Unstructured Web Data"} + FAIRDataDis@{ shape: lean-l, label: "FAIR Data Catalogue"} + end + + %% Intermediate format (Standalone) + FlatGeoBuf@{ shape: lean-l} + + %% Box: Vector Tiles (Pastel Orange) + subgraph vt [Vector Tiles] + VectorTiles@{ shape: lean-l, label: "Mapbox Vector Tiles"} + NextGenVectorTiles@{ shape: lean-l, label: "Next-Gen Vector Tiles"} + end + + %% Box: Visuals (Pastel Blue - No Name) + subgraph visuals [" "] + WebP@{ shape: lean-l} + JPG@{ shape: lean-l} + PNG@{ shape: lean-l} + end + + %% Box: Portable Databases (Pastel Green) + subgraph pkg [Portable Databases] + PMTiles@{ shape: lean-l} + SQLite@{ shape: lean-l} + end + + %% Box: Enterprise (Pastel Purple) + subgraph ent [Enterprise] + FileGeodatabase@{shape: lean-l, label: "File Geodatabase"} + end + end + + %% --------------------------------------------------------- + %% 5. DISTRIBUTION INFRASTRUCTURE + %% --------------------------------------------------------- + subgraph di [Distribution Infrastructure] + ObjectStorage@{ shape: bow-rect, label: "Object Storage"} + Metadata@{ shape: rect, label: "FAIR Data Catalogue"} + HTTP@{ shape: rect, label: "Systems-Ready Data"} + DecentralizedDistribution@{ shape: rect, label: "Decentralized Distribution"} + end + + %% --------------------------------------------------------- + %% 6. EXPERIMENTAL INFRASTRUCTURE + %% --------------------------------------------------------- + subgraph ei [Experimental Infrastructure] + GeoSpatialServices@{ shape: rect, label: "Geospatial Services"} + end + + %% --------------------------------------------------------- + %% 7. CONSUMPTION + %% --------------------------------------------------------- + subgraph "Consumption" + DataSci@{ shape: rect, label: "Data People & Developers"} + Systems@{ shape: rect, label: "Systems"} + end + + %% ========================================================= + %% RELATIONSHIPS + %% ========================================================= + + %% Data Sources <--> Data for Canada Packages Collection (Box) + Statistical a1@<--> pp + a1@{animate: true, animation: slow} + Foundation a2@<--> pp + a2@{animate: true, animation: slow} + Orthoimagery a3@<--> pp + a3@{animate: true, animation: slow} + EnvironmentClimate a5@<--> pp + a5@{animate: true, animation: fast} + FieldImagery a4@<--> pp + a4@{animate:true, animation: fast} + Elevation a6@<--> pp + a6@{animate: true, animation: slow} + WebCorpus a7@<--> pp + a7@{animate: true, animation: fast} + + pp a10@--> df + a10@{animate: true, animation: fast} + + %% Long-Term Storage --> FlatGeoBuf + sot a10000@<--> FlatGeoBuf + a10000@{animate: true, animation: fast} + + %% FlatGeoBuf --> Vector Tiles (Box) + FlatGeoBuf a11@--> vt + a11@{animate: true, animation: fast} + + %% Long-Term Storage <--> Visuals (Box) + sot a12@<--> visuals + a12@{animate: true, animation: slow} + + %% Vector Tiles --> Portable Databases (Box) + vt a90@<--> pkg + a90@{animate: true, animation: fast} + + %% Visuals --> Portable Databases (Box) + visuals a93@<--> pkg + a93@{animate: true, animation: slow} + + %% Long-Term Storage --> Enterprise (Box) + sot a100@<--> ent + a100@{animate: true, animation: slow} + + %% Visuals --> Enterprise (Box) + visuals a102@--> ent + a102@{animate: true, animation: slow} + + %% Dissemination Formats --> Distribution Infrastructure + df a13@<--> di + a13@{animate: true, animation: slow} + + %% Distribution Infrastructure Flow + ObjectStorage a15@<--> Metadata + a15@{animate: true, animation: slow} + Metadata a16@<--> HTTP + a16@{animate: true, animation: slow} + + HTTP a17@<--> ei + a17@{animate: true, animation: slow} + HTTP a18@<--> DecentralizedDistribution + a18@{animate: true, animation: slow} + HTTP a19@<--> DataSci + a19@{animate: true, animation: slow} + + DecentralizedDistribution a20@--> Systems + a20@{animate: true, animation: fast} + DecentralizedDistribution a21@--> DataSci + a21@{animate: true, animation: fast} + + Systems a22@ <--> DataSci + a22@{animate: true, animation: fast} + ei a23@ <--> DataSci + a23@{animate: true, animation: slow} + + %% ========================================================= + %% STYLING + %% ========================================================= + +classDef linkNode stroke:#333333,color:#333333,stroke-width:1.5px; + +%%style pp fill:#D32F2F,stroke:#8E0000,color:#FFFFFF +style DataforCanadaPackagesCollection fill:#B71C1C,stroke:#7F0000,color:#FFFFFF +style FAIRDataDis fill:#B71C1C,stroke:#7F0000,color:#FFFFFF +style DecentralizedDistribution fill:#B71C1C,stroke:#7F0000,color:#FFFFFF +style HTTP fill:#B71C1C,stroke:#7F0000,color:#FFFFFF +style Systems fill:#B71C1C,stroke:#7F0000,color:#FFFFFF +style Metadata fill:#B71C1C,stroke:#7F0000,color:#FFFFFF + +%%style df fill:#D32F2F,stroke:#8E0000,color:#FFFFFF +style sot fill:#EF9A9A,stroke:#C62828,color:#000000 + +style Parquet fill:#FFCDD2,stroke:#E57373,color:#000000 +style Zarr fill:#FFCDD2,stroke:#E57373,color:#000000 +style GeoTIFF fill:#FFCDD2,stroke:#E57373,color:#000000 +style JPEGXL fill:#FFCDD2,stroke:#E57373,color:#000000 +style WARC fill:#FFCDD2,stroke:#E57373,color:#000000 +style AV1 fill:#FFCDD2,stroke:#E57373,color:#000000 + +style ds fill:#FB8C00,stroke:#E65100,color:#000000 + +style pkg fill:#FFB74D,stroke:#EF6C00,color:#000000 +style SQLite fill:#EF6C00,stroke:#E65100,color:#000000 +style PMTiles fill:#FFCC80,stroke:#FB8C00,color:#000000 + +style vt fill:#FBC02D,stroke:#F9A825,color:#000000 +style FlatGeoBuf fill:#FBC02D,stroke:#F9A825,color:#000000 + +style visuals fill:#FBC02D,stroke:#F9A825,color:#000000 + +style ent fill:#66BB6A,stroke:#2E7D32,color:#000000 +style DataSci fill:#D32F2F,stroke:#8E0000,color:#FFFFFF +style GeoSpatialServices fill:#FFCC80,stroke:#FB8C00,color:#000000 + + +class Foundation,Statistical,Orthoimagery,FieldImagery,EnvironmentClimate,Elevation,WebCorpus linkNode +class Parquet,FlatGeoBuf,SQLite,FileGeodatabase,VectorTiles,NextGenVectorTiles,GeoTIFF,Zarr,WebP,PMTiles,JPEGXL,AV1,WARC linkNode + + %% ========================================================= + %% CLICK ACTIONS + %% ========================================================= + click DataforCanadaPackagesCollection "https://github.com/dataforcanada/dataforcanadapkgs-labs/" _blank + + click Foundation "https://github.com/dataforcanada/process-foundation-labs/" _blank + click Statistical "https://github.com/dataforcanada/process-statistical-labs/" _blank + click Orthoimagery "https://github.com/dataforcanada/process-orthoimagery-labs/" _blank + click FieldImagery "https://github.com/dataforcanada/process-field-imagery-labs/" _blank + click EnvironmentClimate "https://github.com/dataforcanada/process-environmental-climate-health-labs/" _blank + click Elevation "https://github.com/dataforcanada/process-elevation-labs/" _blank + click WebCorpus "https://github.com/dataforcanada/process-web-corpus-labs/" _blank + + click Parquet "https://github.com/apache/parquet-format/" _blank + click FlatGeoBuf "https://flatgeobuf.org/" _blank + click SQLite "https://www.geopackage.org/" _blank + click FileGeodatabase "https://gdal.org/en/stable/drivers/vector/openfilegdb.html" _blank + click VectorTiles "https://github.com/mapbox/vector-tile-spec/" _blank + click NextGenVectorTiles "https://github.com/maplibre/maplibre-tile-spec/" _blank + click GeoTIFF "https://cogeo.org/" _blank + click Zarr "https://github.com/zarr-developers/geozarr-spec/" _blank + click WebP "https://developers.google.com/speed/webp/" _blank + click PMTiles "https://github.com/protomaps/PMTiles/blob/main/spec/v3/spec.md" _blank + click JPEGXL "https://jpeg.org/jpegxl/" _blank + click AV1 "https://aomedia.org/specifications/av1/" _blank + click WARC "https://github.com/iipc/warc-specifications/" _blank + click FAIRDataDis "https://stac-utils.github.io/stac-geoparquet/latest/spec/stac-geoparquet-spec/" _blank + + click HTTP "https://www.dataforcanada.org/docs/" _blank + click DecentralizedDistribution "https://www.dataforcanada.org/docs/dissemination/" _blank + click Metadata "https://stac-utils.github.io/stac-geoparquet/latest/spec/stac-geoparquet-spec/" _blank + click GeoSpatialServices "https://github.com/dataforcanada/geo-services-labs/" _blank \ No newline at end of file diff --git a/static/high-level-overview-homepage.svg b/static/high-level-overview-homepage.svg new file mode 100644 index 0000000..4834a29 --- /dev/null +++ b/static/high-level-overview-homepage.svg @@ -0,0 +1 @@ +

Consumption

Distribution Infrastructure

Data Sources

Experimental Infrastructure

Geospatial Services

Dissemination Formats

Enterprise

File Geodatabase

Long-Term Storage

Parquet

Zarr

GeoTIFF

Next-Gen Video

Next-Gen Imagery

Unstructured Web Data

FAIR Data Catalogue

FlatGeoBuf

Vector Tiles

Mapbox Vector Tiles

Next-Gen Vector Tiles

WebP

JPG

PNG

Portable Databases

PMTiles

SQLite

Processing Pipeline

Data for Canada Packages Collection

Statistical

Foundation

Orthoimagery

Environment, Climate, & Health

Field Imagery

Elevation

Web Corpus

Object Storage

FAIR Data Catalogue

Systems-Ready Data

Decentralized Distribution

Data People & Developers

Systems

 \ No newline at end of file