6 Commits
Author SHA1 Message Date
Erica Fischer 10f7f0a3c2 Support joins from sqlite tables in tile-join (#308)
* Sketching out sqlite options for tile-join

* Enable sqlite3 serialized multithreading

* Fix some unnecessary round trips from std::string to char * and back

* Gathering join keys for sql query

* Make a query

* Actually open the gpkg. Fix the query quoting.

* Actually do the query and get results back

* Join the attributes onto the feature

* Set matched if the sql join matches

* Add a flag to get the feature ID from the query

* Observe attribute exclusion when joining from sql queries

* Add a flag to exclude all attributes from the tile side of the join

* Make tile-join bounding boxes reflect feature bounds, not tile bounds

* More tests

* An empty tileset has empty bounds at null island

* Gather the results from each thread *after* the thread finishes

* Missed a test

* Fix accidental inclusion of the top left of the tile in the bbox

* Case smashing and prefix trimming in the select

* Add test of sql join

* Make the join column option a join expression option

* Fix antimeridian adjustment. Z0 can't wait until the end of the tile

* Checkpoint on accepting multiple joined rows per tiled feature

* Adding the joined attribute should be per-feature, not per-attribute

* Forgot to update the test. Order of joined attributes has changed.

* Get the attributes back in the right order

* Add test of sql join with limit

* Allow multiple tile features to have the same join key

* Update tests for a country name with two distinct geometries

* Update version and changelog

* Forgot to mention the bounding box improvements
2025-01-13 12:25:00 -08:00
Erica Fischer 6a8f1b83d8 Fix some undefined behavior (#209)
* Fix some undefined behavior

* Avoid overflow in line simplification calculations

* Oops, missed a test

* Didn't mean to add that to the Makefile

* Revert "Revert "[ci] test in debug mode (#202)""

This reverts commit c95c328e47.

* Fix reference to out-of-scope pointer

* Fix invalid shift and out-of-bounds vector element reference

* Update changelog and version
2024-03-01 10:11:41 -08:00
Erica Fischer cc5c1c79df Speed up overzooming in tile-join (#147)
* Clip away entire features by bbox. Avoid unnecessary recompression.

* Move parent tile decoding in tile-join out of overzoom proper

* An ever-growing cache of parent tiles

* Limit the size of the cache

* Remove the current reader *before* checking if we can run the queue

* Clean up

* Add missing #include

* Add comment

* When the tile-join cache fills up, evict the least recently used

* Fix microsecond math

* Factoring out tile-join's cache for testing

* Add unit tests for tile-join cache

* Update changelog and version
2023-10-04 12:14:22 -07:00
Erica Fischer 390771f855 Polygon shards (#105)
* Put all of this back in geometry.cpp for conflict resolution

* Stabilize line simplification to behave the same regardless of winding

* Round instead of truncating when clipping lines

* Restore non-Wagyu polygon clipping from prior to 2fdec7d2

* Make it round, not truncate, which reverts the last commit's test diffs

* Clip in floating point, not integers, which makes no difference

* Track nodes added at tile edges during clipping

* Scale geometry up before wagyu to prevent changes from precision loss

* Actually do the shared edge detection

* Fix cases where nodes were not being added at the tile boundary

* One more place I should have rounded

* Narrow down where the discrepancy comes in

* Revert "Narrow down where the discrepancy comes in"

This reverts commit 221c4c5fc0ac9a6567e091c6a94b3d87dc8ade83.

* Another attempt to narrow it down

* The discrepancy seems to be introduced in reordering. No obvious bug

* Was still truncating instead of rounding in projection

* Also makes no difference...

* Just forget that line reversal exists for a minute

* Just reversal no coalescing

* Try clipping in integers instead of floating point

* Don't simplify after coalescing if they said no simplification

* Check whether behavior is consistent with intentional simplification

* Replace more floating point with integer

* Are these three features enough to demonstrate the problem?

* Add a few more nearby borders

* All the features that touch tile 6/16/23

* Stay in integers in line simplification

* More attempts to solve failures to simplify consistently

* Fix most of the overflow errors

* Fix known cases of integer overflow

* All the tests change again

* Pull clipping and scaling code back out into clip.cpp

* Resolve the test conflicts

* Stabilize choice of which three points to keep with different windings

* Almost right, I think!

* Fix collapse of islands to shards

* Self-intersections in the same feature don't count

* Revert "Self-intersections in the same feature don't count"

This reverts commit e04b19916e.

* Don't scale down geometry if we are going to look for shared nodes

* Fix the missing multiply that was keeping simplification from happening

* Fix one more opportunity for overflow

* Somehow I deleted this test?

* Lost this test too

* Clean up debugging printfs

* Restore code sequence from main to make it reviewable

* Remove unneeded rounding

* Update documentation

* This test is no longer useful

* Back to floating point Douglas-Peucker to fix undersimplification

* Try an older ubuntu

* Revert "Try an older ubuntu"

This reverts commit 13fefacfd7.

* Log OS info

* Remove tests that are no longer needed

* Oops, did need that one after all

* Fix the arm vs x86 discrepancy?

* Try another quantization

* Cleanup from review

* Add a test for the actual purpose of this PR
2023-09-14 16:51:38 -07:00
Erica Fischer 2ec6180003 Fix "strategies" accounting for 0-length linestrings and degenerate polygons (#137)
* Dropping a 0-length feature doesn't count as dropping-as-needed

* Add an option not to reduce tiny polygons to dust at maxzoom

* Add test for prevention of tiny polygon reduction at maxzoom

* Fix accounting for tiny polygons not to include degenerate geometries

* Revert "Add test for prevention of tiny polygon reduction at maxzoom"

This reverts commit f931bbd73e.

* Revert "Add an option not to reduce tiny polygons to dust at maxzoom"

This reverts commit 03f0882bb6.

* Fix tests

* Another test that no longer has any really tiny polygons

* Oops, that broke LineString simplification

* This time for sure!

* Update changelog and version
2023-08-29 10:54:39 -07:00
Erica Fischer 6778aeac52 Add an option to extend zooms if still dropping, but with a limit (#131)
* Add an option to extend zooms if still dropping, but with a limit

* At least when to overzoom, even if not actually doing it yet

* Refactor to give tile-join access to overzoom()

* Didn't work, but *might* have worked

* OK, it did something now

* Ah, there's the bug!

* Hook up pmtiles and dirtiles as overzooming sources

* Add command line option to enable or disable overzooming

* Add (currently broken) test of overzooming in tile-join

* Slightly more abstraction for the tile-join readers

* Factor out duplicated code

* Move construction into a constructor

* More changing accessors to methods

* Reduce magic

* Start tracking a list of the tiles at maxzoom

* I think it worked?

* Add missing #include

* Fix sequence of overzoomed tiles (Y sorts backwards for TMS)

* Don't spend memory on overzooming when we aren't going to use it

* Diff rather than cmp, in the hope of figuring out this broken test

* Keep full coordinate precision if we might extend zooms

* Try a slightly different byte limit

* Make drop-densest more consistent across tile boundaries

* Also affects this test

* Does it behave any differently if it can extend forever?

* I think the discrepancy is a thread-safety problem here

* Revert "Does it behave any differently if it can extend forever?"

This reverts commit 0dff0a0acc.

* Lost this change to the test

* This time for sure!

* Revert "Also affects this test"

This reverts commit cd1f7c2e78.

* Revert "Make drop-densest more consistent across tile boundaries"

This reverts commit 563f7d2bc2.

* Revert "Try a slightly different byte limit"

This reverts commit 2e271213d6.

* Add some more explanatory comments

* Amend the join-test to detect my current bug

* Allow overzooming to complete the zoom if it ever starts

* Forgot to correct the test

* Update changelog and version

* Cleanups from code review

* Remove version number from fixture to fix test
2023-08-25 12:42:38 -07:00