dataforcanada/d4c-service-main-site
1
0
Fork 0
mirror of https://github.com/dataforcanada/d4c-service-main-site.git synced 2026-06-13 14:00:51 +02:00
Code Issues Packages Projects Releases Wiki Activity

Simplify homepage high-level overview diagram. Change terminology

Browse Source
This commit is contained in:
Diego Ripley
2026-02-11 18:08:38 -05:00
parent 6d03f28f32
commit 0fbfce35aa
1 changed files with 126 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/_index.md
+126 -74
View File
@@ -62,9 +62,14 @@ See our processing pipeline in action. View samples and documentation for our cu
**Note:** The data sources in the diagram below are **prioritized from left to right**, reflecting our current focus on processing high-value [statistical](https://www.dataforcanada.org/docs/processes/statistical_products/), [foundational](https://www.dataforcanada.org/docs/processes/foundation/#download-and-preview), and [orthoimagery](https://www.dataforcanada.org/docs/processes/orthoimagery/#download-and-preview) datasets first. **Note:** The data sources in the diagram below are **prioritized from left to right**, reflecting our current focus on processing high-value [statistical](https://www.dataforcanada.org/docs/processes/statistical_products/), [foundational](https://www.dataforcanada.org/docs/processes/foundation/#download-and-preview), and [orthoimagery](https://www.dataforcanada.org/docs/processes/orthoimagery/#download-and-preview) datasets first.
```mermaid ```mermaid
flowchart TD flowchart TD
classDef linkNode stroke:#0000EE,color:#0000EE,stroke-width:2px; classDef linkNode stroke:#0000EE,color:#0000EE,stroke-width:2px;
%% ---------------------------------------------------------
%% 1. DATA SOURCES
%% ---------------------------------------------------------
subgraph ds [Data Sources] subgraph ds [Data Sources]
Statistical@{ shape: lean-l} Statistical@{ shape: lean-l}
Foundation@{ shape: lean-l} Foundation@{ shape: lean-l}
@@ -75,29 +80,63 @@ flowchart TD
WebCorpus@{ shape: lean-l, label: "Web Corpus"} WebCorpus@{ shape: lean-l, label: "Web Corpus"}
end end
%% ---------------------------------------------------------
%% 2. PROCESSING PIPELINE
%% ---------------------------------------------------------
subgraph pp [Processing Pipeline] subgraph pp [Processing Pipeline]
Raw@{ shape: rect, label: "Raw Data Ingestion"} Raw@{ shape: rect, label: "Raw Data Ingestion"}
Transform@{ shape: rect, label: "Transform and Optimize"} Transform@{ shape: rect, label: "Transform and Optimize"}
%% Internal Link
Raw --> Transform
end end
%% ---------------------------------------------------------
%% 3. DISSEMINATION FORMATS
%% ---------------------------------------------------------
subgraph df [Dissemination Formats] subgraph df [Dissemination Formats]
Parquet@{ shape: lean-l}
%% Box: Long-Term Storage (Pastel Gold)
subgraph sot [Long-Term Storage]
Parquet@{ shape: lean-l}
Zarr@{ shape: lean-l}
GeoTIFF@{ shape: lean-l}
AV1@{ shape: lean-l, label: "Next-Gen Video"}
JPEGXL@{ shape: lean-l, label: "Next-Gen Imagery"}
WARC@{ shape: lean-l, label: "Unstructured Web Data"}
end
%% Intermediate format (Standalone)
FlatGeoBuf@{ shape: lean-l} FlatGeoBuf@{ shape: lean-l}
VectorTiles@{ shape: lean-l, label: "Vector Tiles"}
NextGenVectorTiles@{ shape: lean-l, label: "Next-Gen Vector Tiles"} %% Box: Vector Tiles (Pastel Orange)
PMTiles@{ shape: lean-l} subgraph vt [Vector Tiles]
SQLite@{ shape: lean-l} VectorTiles@{ shape: lean-l, label: "Mapbox Vector Tiles"}
FileGeodatabase@{shape: lean-l, label: "File Geodatabase"} NextGenVectorTiles@{ shape: lean-l, label: "Next-Gen Vector Tiles"}
GeoTIFF@{ shape: lean-l} end
Zarr@{ shape: lean-l}
WebP@{ shape: lean-l} %% Box: Visuals (Pastel Blue - No Name)
JPG@{ shape: lean-l} subgraph visuals [" "]
PNG@{ shape: lean-l} WebP@{ shape: lean-l}
JPEGXL@{ shape: lean-l, label: "JPEG XL"} JPG@{ shape: lean-l}
AV1@{ shape: lean-l, label: "AV1"} PNG@{ shape: lean-l}
WARC@{ shape: lean-l} end
%% Box: Portable Databases (Pastel Green)
subgraph pkg [Portable Databases]
PMTiles@{ shape: lean-l}
SQLite@{ shape: lean-l}
end
%% Box: Enterprise (Pastel Purple)
subgraph ent [Enterprise]
FileGeodatabase@{shape: lean-l, label: "File Geodatabase"}
end
end end
%% ---------------------------------------------------------
%% 4. DISTRIBUTION INFRASTRUCTURE
%% ---------------------------------------------------------
subgraph di [Distribution Infrastructure] subgraph di [Distribution Infrastructure]
ObjectStorage@{ shape: bow-rect, label: "Object Storage"} ObjectStorage@{ shape: bow-rect, label: "Object Storage"}
Metadata@{ shape: rect, label: "FAIR Data Catalog"} Metadata@{ shape: rect, label: "FAIR Data Catalog"}
@@ -105,96 +144,119 @@ flowchart TD
DecentralizedDistribution@{ shape: rect, label: "Decentralized Distribution"} DecentralizedDistribution@{ shape: rect, label: "Decentralized Distribution"}
end end
%% ---------------------------------------------------------
%% 5. EXPERIMENTAL INFRASTRUCTURE
%% ---------------------------------------------------------
subgraph ei [Experimental Infrastructure] subgraph ei [Experimental Infrastructure]
GeoSpatialServices@{ shape: rect, label: "Geospatial Services"} GeoSpatialServices@{ shape: rect, label: "Geospatial Services"}
%%Martin@{ shape: rect}
%%GeoServer@{ shape: rect}
%%ZOOProject@{ shape: rect, label: "ZOO-Project"}
%%BBOXServer@{ shape: rect, label: "BBOX Server"}
%%Panoramax@{ shape: rect}
%%Pelias@{ shape: rect}
end end
%% ---------------------------------------------------------
%% 6. CONSUMPTION
%% ---------------------------------------------------------
subgraph "Consumption" subgraph "Consumption"
DataSci@{ shape: rect, label: "Data People & Developers"} DataSci@{ shape: rect, label: "Data People & Developers"}
Systems@{ shape: rect, label: "Systems"} Systems@{ shape: rect, label: "Systems"}
end end
%% Relationships %% =========================================================
Statistical a1@--> Raw %% RELATIONSHIPS
%% =========================================================
%% Data Sources --> Processing Pipeline (Box)
Statistical a1@--> pp
a1@{animate: true, animation: slow} a1@{animate: true, animation: slow}
Foundation a2@--> Raw Foundation a2@--> pp
a2@{animate: true, animation: slow} a2@{animate: true, animation: slow}
Orthoimagery a3@--> Raw Orthoimagery a3@--> pp
a3@{animate: true, animation: slow} a3@{animate: true, animation: slow}
EnvironmentClimate a5@--> Raw EnvironmentClimate a5@--> pp
a5@{animate: true, animation: fast} a5@{animate: true, animation: fast}
FieldImagery a4@--> Raw FieldImagery a4@--> pp
a4@{animate:true, animation: fast} a4@{animate:true, animation: fast}
Elevation a6@--> Raw Elevation a6@--> pp
a6@{animate: true, animation: slow} a6@{animate: true, animation: slow}
WebCorpus a7@--> Raw WebCorpus a7@--> pp
a7@{animate: true, animation: fast} a7@{animate: true, animation: fast}
Raw a8@--> Transform
a8@{animate: true, animation: slow} %% Processing Pipeline --> Long-Term Storage (Box)
Transform a9@--> df Transform a9@--> sot
a9@{animate: true, animation: slow} a9@{animate: true, animation: slow}
Parquet a10@--> FlatGeoBuf
%% Long-Term Storage --> FlatGeoBuf
sot a10@--> FlatGeoBuf
a10@{animate: true, animation: fast} a10@{animate: true, animation: fast}
Parquet a100@--> FileGeodatabase
Parquet a110@--> SQLite %% FlatGeoBuf --> Vector Tiles (Box)
a110@{animate: true, animation: slow} FlatGeoBuf a11@--> vt
a100@{animate: true, animation: slow}
FlatGeoBuf a11@--> VectorTiles
a11@{animate: true, animation: fast} a11@{animate: true, animation: fast}
FlatGeoBuf a91@--> NextGenVectorTiles
a91@{animate: true, animation: fast} %% Long-Term Storage --> Visuals (Box)
VectorTiles a90@ --> PMTiles sot a12@--> visuals
a90@{animate: true, animation: fast}
VectorTiles a96@ --> SQLite
a96@{animate: true, animation: slow}
NextGenVectorTiles a92@ --> PMTiles
a92@{animate: true, animation: slow}
NextGenVectorTiles a95@ --> SQLite
a95@{animate: true, animation: slow}
Zarr a12@ --> WebP
a12@{animate: true, animation: slow} a12@{animate: true, animation: slow}
df a13@ --> di
a13@{animate: true, animation: slow} %% Vector Tiles --> Portable Databases (Box)
GeoTIFF a14@--> WebP vt a90@--> pkg
a14@{animate: true, animation: slow} a90@{animate: true, animation: fast}
WebP a93@--> PMTiles
%% Visuals --> Portable Databases (Box)
visuals a93@--> pkg
a93@{animate: true, animation: slow} a93@{animate: true, animation: slow}
WebP a94@--> SQLite
a94@{animate: true, animation: slow} %% Long-Term Storage --> Enterprise (Box)
WebP a103@--> JPG sot a100@--> ent
a103@{animate: true, animation: slow} a100@{animate: true, animation: slow}
WebP a104@--> PNG
a104@{animate: true, animation: slow} %% Visuals --> Enterprise (Box)
JPG a101@--> FileGeodatabase visuals a102@--> ent
a101@{animate: true, animation: slow}
PNG a102@--> FileGeodatabase
a102@{animate: true, animation: slow} a102@{animate: true, animation: slow}
%% Dissemination Formats --> Distribution Infrastructure
df a13@--> di
a13@{animate: true, animation: slow}
%% Distribution Infrastructure Flow
ObjectStorage a15@--> Metadata ObjectStorage a15@--> Metadata
a15@{animate: true, animation: slow} a15@{animate: true, animation: slow}
Metadata a16@--> HTTP Metadata a16@--> HTTP
a16@{animate: true, animation: slow} a16@{animate: true, animation: slow}
HTTP a17@--> ei HTTP a17@--> ei
a17@{animate: true, animation: slow} a17@{animate: true, animation: slow}
HTTP a18@--> DecentralizedDistribution HTTP a18@--> DecentralizedDistribution
a18@{animate: true, animation: slow} a18@{animate: true, animation: slow}
HTTP a19@--> DataSci HTTP a19@--> DataSci
a19@{animate: true, animation: slow} a19@{animate: true, animation: slow}
DecentralizedDistribution a20@--> Systems DecentralizedDistribution a20@--> Systems
a20@{animate: true, animation: fast} a20@{animate: true, animation: fast}
DecentralizedDistribution a21@--> DataSci DecentralizedDistribution a21@--> DataSci
a21@{animate: true, animation: fast} a21@{animate: true, animation: fast}
Systems a22@ --> DataSci Systems a22@ --> DataSci
a22@{animate: true, animation: fast} a22@{animate: true, animation: fast}
ei a23@ --> DataSci ei a23@ --> DataSci
a23@{animate: true, animation: slow} a23@{animate: true, animation: slow}
%% URLs %% =========================================================
%% STYLING
%% =========================================================
%% Color Palette (Option 1: Functional)
style sot fill:#FFF9C4
style ent fill:#E1BEE7
style vt fill:#FFCCBC
style pkg fill:#C8E6C9
style visuals fill:#B3E5FC
%% Link Node Styling
class Foundation,Statistical,Orthoimagery,FieldImagery,EnvironmentClimate,Elevation,WebCorpus linkNode
class Parquet,FlatGeoBuf,SQLite,FileGeodatabase,VectorTiles,NextGenVectorTiles,GeoTIFF,Zarr,WebP,PMTiles,JPEGXL,AV1,WARC linkNode
class DecentralizedDistribution,HTTP,Metadata,GeoSpatialServices linkNode
%% =========================================================
%% CLICK ACTIONS
%% =========================================================
click Foundation "https://github.com/dataforcanada/process-foundation-labs/" _blank click Foundation "https://github.com/dataforcanada/process-foundation-labs/" _blank
click Statistical "https://github.com/dataforcanada/process-statistical-labs/" _blank click Statistical "https://github.com/dataforcanada/process-statistical-labs/" _blank
click Orthoimagery "https://github.com/dataforcanada/process-orthoimagery-labs/" _blank click Orthoimagery "https://github.com/dataforcanada/process-orthoimagery-labs/" _blank
@@ -220,19 +282,9 @@ flowchart TD
click DecentralizedDistribution "https://www.dataforcanada.org/docs/dissemination/" _blank click DecentralizedDistribution "https://www.dataforcanada.org/docs/dissemination/" _blank
click Metadata "https://stac-utils.github.io/stac-geoparquet/latest/spec/stac-geoparquet-spec/" _blank click Metadata "https://stac-utils.github.io/stac-geoparquet/latest/spec/stac-geoparquet-spec/" _blank
click GeoSpatialServices "https://github.com/dataforcanada/geo-services-labs/" _blank click GeoSpatialServices "https://github.com/dataforcanada/geo-services-labs/" _blank
click Martin "https://martin.maplibre.org/" _blank
click GeoServer "https://geoserver.org/" _blank
click ZOOProject "https://zoo-project.org/" _blank
click BBOXServer "https://www.bbox.earth/" _blank
click Panoramax "https://gitlab.com/panoramax" _blank
click Pelias "https://pelias.io" _blank
%% APPLY STYLES TO LINKED NODES
class Foundation,Statistical,Orthoimagery,FieldImagery,EnvironmentClimate,Elevation,WebCorpus linkNode
class Parquet,FlatGeoBuf,SQLite,FileGeodatabase,VectorTiles,NextGenVectorTiles,GeoTIFF,Zarr,WebP,PMTiles,JPEGXL,AV1,WARC linkNode
class DecentralizedDistribution,HTTP,Metadata,GeoSpatialServices linkNode
``` ```
## Get Involved ## Get Involved
We are actively looking for new members and partners to help shape this project. We are actively looking for new members and partners to help shape this project.