dataforcanada/d4c-datapkg-statistical
1
0
Fork 0
mirror of https://github.com/dataforcanada/d4c-datapkg-statistical.git synced 2026-06-13 14:10:55 +02:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 6d11c45665 Restore README header for d4c-datapkg-statistical master Diego Ripley 2026-02-25 14:06:09 -05:00
  • 3530ce3496 Update README.md Diego Ripley 2026-02-20 13:17:17 -05:00
  • 3c09459255 Update README.md Diego Ripley 2026-02-09 07:49:35 -05:00
  • f398714330 Update copyright holder in LICENSE file Diego Ripley 2026-02-01 20:29:52 -05:00
  • 5b17c4e414 Update project name from 'dev' to 'labs' in README Diego Ripley 2026-02-01 20:29:16 -05:00
  • be1396e652 Update repository name in README.md Diego Ripley 2026-01-29 07:20:45 -05:00
  • a13cf62310 Remove vector tiles HTML experiment. All the work is available at https://www.diegoripley.ca/files/census_of_population_2021_vector_tiles_august_12_2025/. I have the code in another repo, but I need to clean it up before publishing Diego Ripley 2025-10-01 11:34:44 -04:00
  • d23018d8a6 Add NAR 2025-07 and update download links Diego Ripley 2025-09-22 17:06:19 +00:00
  • 37c3a286d8 Update tippecanoe Diego Ripley 2025-09-18 19:17:24 +00:00
  • dff7ea6fad Fix mistake in creating pop_ctr Census of Population file. Select distinct pop_ctr_dguid before joining to Census of Pop attribute data Diego Ripley 2025-09-18 18:52:45 +00:00
  • 1a7ea7e40a Update GDAL, install some other Python packages, and change port for postgres Diego Ripley 2025-09-16 18:18:55 +00:00
  • 901d051567 Made modifications and processed CMA for 2021 Census of Population Diego Ripley 2025-09-13 15:13:29 +00:00
  • f5a2831cf6 Add code that was used for https://www.diegoripley.ca/files/census_of_population_vector_tiles_subset_august_12_2025/ Diego Ripley 2025-08-10 10:02:41 -04:00
  • bc9e7b5f8c Made improvements Diego Ripley 2025-08-09 17:02:35 +00:00
  • bd246e297a 2021 Census of Population for capital cities in Canada, and Toronto, at the Dissemination Area (DA) level Diego Ripley 2025-08-08 13:24:31 -04:00
  • 7d33ab4587 Playing around with uploading data to Zenodo Diego Ripley 2025-07-10 17:04:28 +00:00
  • d38fb18de1 Add Census of Population 2021 example that describes the structure of the data Diego Ripley 2025-07-10 16:56:42 +00:00
  • 6b42a80529 Expand example to country Diego Ripley 2025-07-10 16:11:11 +00:00
  • 7462ab15bf Rename presentation example Diego Ripley 2025-07-10 12:09:06 -04:00
  • 2c22c4fc10 Add example for presentation Diego Ripley 2025-07-10 11:19:26 -04:00
  • aafb10b261 Add image showing download speed of torrent Diego Ripley 2025-06-28 21:36:24 +00:00
  • bf653b7160 Add s2maps torrent example Diego Ripley 2025-06-28 17:17:25 -04:00
  • ed319c2e92 Create torrent of a dataset. Has HTTP seed URLs that work (when created through py3createtorrent). I am able to max out my internet connection when downloading the torrent. Diego Ripley 2025-06-28 21:08:05 +00:00
  • 4d8fb1e7e9 Download datasets using rclone. HTTP end-point is https://data-01.dataforcanada.org Diego Ripley 2025-06-27 09:47:31 -04:00
  • 6ad2e2c4d6 Scraping the table names from from https://www150.statcan.gc.ca/n1/en/type/data Diego Ripley 2025-06-27 09:11:42 -04:00
  • b88a2272b4 Clean Dockerfile Diego Ripley 2025-06-27 08:49:32 -04:00
  • 79a37a3afc Some fixes for ubuntu user Diego Ripley 2025-06-27 08:47:11 -04:00
  • 2f346bfa4c Fix #4. Needed to mount ~/.ssh and make sure that it has the same UID/GID as the host user Diego Ripley 2025-06-26 15:14:33 +00:00
  • a55e1d325d Made changes Diego Ripley 2025-06-26 13:35:37 +00:00
  • 4ed5fb4bbb Add DuckDB example for duplicate column name Diego Ripley 2025-06-25 15:38:12 +00:00
  • b71a7b326e DuckDB issue with duplicate column names (ex. 'Value' and 'VALUE' are treated the same) Diego Ripley 2025-06-25 15:30:36 +00:00
  • e929850d4a Finish comment on issue with Value and VALUE columns being treated the same by DuckDB Diego Ripley 2025-06-21 18:03:29 +00:00
  • 8875722d10 Made changes to processing of data tables Diego Ripley 2025-06-21 18:01:16 +00:00
  • 7c8211cb5f Found some issues with the output parquet files Diego Ripley 2025-06-21 05:26:50 +00:00
  • 887291d2f7 Read all DGUIDs from subset parquet output (100,000 records each) Diego Ripley 2025-06-21 00:54:26 -04:00
  • 72ca6c87e1 Made changes Diego Ripley 2025-06-20 17:32:01 -04:00
  • 5a95616b3c Calculate CSV file size by viewing inside of zip file Diego Ripley 2025-06-20 16:01:20 -04:00
  • e836363cd1 Had to optimize the code. Leaving it outside of function for now in case I need to continue working on it Diego Ripley 2025-06-20 16:00:51 -04:00
  • f6d88c5fd0 Continue work on processing data tables Diego Ripley 2025-06-19 15:58:30 -04:00
  • ab8f40c708 Keeping track of processed files in case processing crashes and I have to restart again Diego Ripley 2025-06-19 11:40:37 -04:00
  • faa63451ab Experiment with Jupyter notebook on downloading and processing statcan cubes Diego Ripley 2025-06-18 21:26:51 +00:00
  • c0899080f4 Remove scratch files after processing. Was running out of space Diego Ripley 2025-06-18 09:26:18 -04:00
  • f85ce79ff2 Add polars to Dockerfile Diego Ripley 2025-06-17 21:19:42 +00:00
  • ea603f2914 Convert statcan CSV into parquet Diego Ripley 2025-06-17 20:46:24 +00:00
  • 4d16ef8232 Process 2024-06 national address register Diego Ripley 2025-06-07 15:39:28 +00:00
  • b3c5f8767f Move export of 2021 geographic boundaries Diego Ripley 2025-06-06 22:51:49 +00:00
  • 5e26ec282e Process 2024-12 national address register. Still need to make some improvements Diego Ripley 2025-06-06 22:49:47 +00:00
  • 2350d6d8d7 Update README files Diego Ripley 2025-06-06 11:20:41 +00:00
  • ce999a3a4d Remove unused code Diego Ripley 2025-06-04 17:02:21 +00:00
  • 3565b7c5a4 Update DuckDB lonboard example Diego Ripley 2025-06-04 17:01:04 +00:00
  • fad884efc8 Create 2021 cartographic boundary files Diego Ripley 2025-06-04 16:36:45 +00:00
  • 13a86dc3dc Add 2006, 2011, 2016 cartographic boundary files Diego Ripley 2025-06-03 14:34:53 +00:00
  • 0203e04b45 Add 2021 cartographic boundary files Diego Ripley 2025-06-03 13:33:03 +00:00
  • 23e3133118 Add 2021 cartographic boundary files Diego Ripley 2025-06-02 22:03:49 +00:00
  • 5a188a469c Update README Diego Ripley 2025-06-02 14:13:58 +00:00
  • 7e0b001075 Enable downloading of 2021 hydro Diego Ripley 2025-06-02 13:56:50 +00:00
  • 4eae3622d2 Update DuckDB lonboard example Diego Ripley 2025-05-30 22:30:35 +00:00
  • 34d2c50046 Update DuckDB longboard example Diego Ripley 2025-05-30 17:29:58 +00:00
  • bd7e8ee9f4 Should have used count_total_4 in calculation instead of count_total_1 Diego Ripley 2025-05-30 16:15:27 +00:00
  • daf5a2d154 Fix #6 Diego Ripley 2025-05-30 16:11:45 +00:00
  • 6aedfe7c46 Update DuckDB example so it calculates the area of a polygon properly Diego Ripley 2025-05-30 15:26:22 +00:00
  • a8b05a96f9 Add jenkspy package to Dockerfile Diego Ripley 2025-05-27 15:23:59 +00:00
  • b7f8506708 Add DuckDB and lonboard example. Still need lots of work to get it right Diego Ripley 2025-05-27 15:22:10 +00:00
  • d4a25af7b1 Format DuckDB SQL code Diego Ripley 2025-05-25 20:05:15 +00:00
  • b4b7e20219 Add buckaroo and remove rclone from the Dockerfile Diego Ripley 2025-05-25 19:39:19 +00:00
  • b260d259c8 Add DuckDB example for selecting a Census of Population variable at the Dissemination Area level, calculating a field, and exporting the dataset into multiple formats Diego Ripley 2025-05-25 19:35:30 +00:00
  • 8dab61a685 Update URL for FED 2021, 2013 representation order Diego Ripley 2025-05-24 19:37:12 +00:00
  • f93e4d0cec Initial commit Diego Ripley 2025-05-24 13:37:31 -04:00