Support joins from sqlite tables in tile-join (#308)

* Sketching out sqlite options for tile-join

* Enable sqlite3 serialized multithreading

* Fix some unnecessary round trips from std::string to char * and back

* Gathering join keys for sql query

* Make a query

* Actually open the gpkg. Fix the query quoting.

* Actually do the query and get results back

* Join the attributes onto the feature

* Set matched if the sql join matches

* Add a flag to get the feature ID from the query

* Observe attribute exclusion when joining from sql queries

* Add a flag to exclude all attributes from the tile side of the join

* Make tile-join bounding boxes reflect feature bounds, not tile bounds

* More tests

* An empty tileset has empty bounds at null island

* Gather the results from each thread *after* the thread finishes

* Missed a test

* Fix accidental inclusion of the top left of the tile in the bbox

* Case smashing and prefix trimming in the select

* Add test of sql join

* Make the join column option a join expression option

* Fix antimeridian adjustment. Z0 can't wait until the end of the tile

* Checkpoint on accepting multiple joined rows per tiled feature

* Adding the joined attribute should be per-feature, not per-attribute

* Forgot to update the test. Order of joined attributes has changed.

* Get the attributes back in the right order

* Add test of sql join with limit

* Allow multiple tile features to have the same join key

* Update tests for a country name with two distinct geometries

* Update version and changelog

* Forgot to mention the bounding box improvements
This commit is contained in:
Erica Fischer
2025-01-13 12:25:00 -08:00
committed by GitHub
parent 7165ae6999
commit 10f7f0a3c2
29 changed files with 564 additions and 224 deletions
+14
View File
@@ -578,6 +578,20 @@ join-test: tippecanoe tippecanoe-decode tile-join
./tippecanoe-decode -x generator tests/ne_110m_ocean/join/joined.mbtiles > tests/ne_110m_ocean/join/joined.mbtiles.json.check
cmp tests/ne_110m_ocean/join/joined.mbtiles.json.check tests/ne_110m_ocean/join/joined.mbtiles.json
rm -f tests/ne_110m_ocean/join/ocean.mbtiles tests/ne_110m_ocean/join/countries.mbtiles tests/ne_110m_ocean/join/joined.mbtiles tests/ne_110m_ocean/join/joined.mbtiles.json.check
#
# Test sql join
#
./tile-join -i -f -o tests/join-sql/countries.pmtiles --join-sqlite tests/join-sql/countries.gpkg --join-table countries --join-tile-attribute ne10-admin0:name_en --join-table-expression 'lower(country)' tests/join-sql/bboxes.pmtiles
./tippecanoe-decode -x generator tests/join-sql/countries.pmtiles > tests/join-sql/countries.pmtiles.json.check
cmp tests/join-sql/countries.pmtiles.json.check tests/join-sql/countries.pmtiles.json
rm -f tests/join-sql/countries.pmtiles tests/join-sql/countries.pmtiles.json.check
#
# Test sql join with limit
#
./tile-join --join-count-limit 3 -i -f -o tests/join-sql/countries-limit3.pmtiles --join-sqlite tests/join-sql/countries.gpkg --join-table countries --join-tile-attribute ne10-admin0:name_en --join-table-expression 'lower(country)' tests/join-sql/bboxes.pmtiles
./tippecanoe-decode -x generator tests/join-sql/countries-limit3.pmtiles > tests/join-sql/countries-limit3.pmtiles.json.check
cmp tests/join-sql/countries-limit3.pmtiles.json.check tests/join-sql/countries-limit3.pmtiles.json
rm -f tests/join-sql/countries-limit3.pmtiles tests/join-sql/countries-limit3.pmtiles.json.check
accumulate-test:
# there are 144 features with POP1950 in the original dataset