Commit Graph
11 Commits
Author SHA1 Message Date
Erica Fischer a30722caa5 More switches from maps to unordered_maps 2024-01-24 12:41:01 -08:00
Erica Fischer b91908c967 Avoid an int->string->int conversion 2024-01-24 11:54:01 -08:00
Erica Fischer bdc0f207ae Don't convert types of attributes without accumulators 2024-01-24 11:14:50 -08:00
Erica Fischer cbf222754b Add --accumulate-attribute to tippecanoe-overzoom (#189)
* Starting to factor out attribute accumulation into its own file

* Continuing to factor out attribute accumulation

* Reduce duplicate code

* Plumbing the accumulate-attribute option around

* Call the attribute accumulator

* Test that accumulation works

* Add missing #includes

* Don't sort within individual multiplier clusters

Doing so throws off the spatial distribution of the low zooms

* Docs and changelog

* Add comments
2024-01-23 15:41:36 -08:00
Erica Fischer 679a0d62f2 Make feature ordering cooperate with --retain-points-multiplier (#188)
* Make feature ordering cooperate with --retain-points-multiplier

* Forgot to check in the actual code changes???

* Sort within each multiplier cluster as well as between clusters

* Correct description of behavior in changelog

* Drag original feature sequence along in megatiles for post-filter sort

* Plumb the preserve-input-order flag through overzoom

* Sort in overzoom if requested

* Use within-tile input sequence numbers, not global sequence numbers

* Documentation

* Reverse direction of search to prevent accidental skipping

* Add some comments about converting between attribute representations
2024-01-21 12:09:05 -08:00
Erica Fischer 5d92a17193 Add point retention multiplier (#179)
* Add an option to retain N times as many points as usual at each zoom

* Tests for point multipler with specified and guessed maxzooms

* Work in progress on inverse spatial ordering

* Fix inverse spatial feature order

* --reorder was depending on a feature index that wasn't being preserved

* Separate ordering by feature_minzoom from ordering inverse-spatially

* Add a test for the inverse spatial ordering

* Store the basezoom/droprate/multiplier decisions in tileset metadata

* Progress on adding filters to tippecanoe-overzoom

* Type promotion for comparison

* Look up the attribute value for ordering

* Add test of thinning and ordering features

* Plumb tippecanoe_decisions metadata through pmtiles

* Be careful not to put infinities in JSON

* Fix accidental dropping in what is meant to preserve sparse points

* Start distinguishing true, false, and null in expressions

* Most of the type conversions

* Add boolean conversions

* Literals and conjunctions

* Add filtering to tippecanoe-overzoom

* Add a test of filtering in overzoom

* Fix boolean conjunctions

* Handle the combination of cluster size and filtering

* Rework dot dropping to reconcile density threshold and multiplier

* Revert "Rework dot dropping to reconcile density threshold and multiplier"

This reverts commit f253a66382.

* Retain points by multiplier within each tile, not in global probability

* Test that intends to verify that the multiplier is reversible

* Get the test to detect the discrepancy

* Mark the start of multiplier clusters with a magic attribute

* Add string-contains

* Add in and ni operators

* Revert "Look up the attribute value for ordering"

This reverts commit 56bc73e49a.

* Revert "Type promotion for comparison"

This reverts commit 6f3256f5af.

* Make number formatting in tippecanoe_decisions consistent

* Revert "Add a test for the inverse spatial ordering"

This reverts commit c8047de9ab.

* Revert "Separate ordering by feature_minzoom from ordering inverse-spatially"

This reverts commit 35b19a223c.

* Revert "Fix inverse spatial feature order"

This reverts commit 5978ecdb44.

* Revert "Work in progress on inverse spatial ordering"

This reverts commit fdf230f632.

* Somehow missed the tests associated with that last revert

* Round-robin assign attributes to partials from across the multiplier

* Count the multiplier separately in each layer

* Fix distribution of accumulated attribute across multiplier features

* Update changelog, version, and docs

* Add "is null" and "isnt null" expressions

* Update interpretation of FSL expressions to pass the tests

* Test to assert that polygons are unaffected by the multiplier

* Clean up and comment

* Remove accidental unused case
2024-01-18 15:56:09 -08:00
Erica Fischer d359461e61 Reduce tile-join overzooming memory usage (#162)
* 16 bits is enough for tile numbers

* Revert "16 bits is enough for tile numbers"

This reverts commit 71a0c4e1cf.

* Check what child tiles each overzoomed tile will have

and don't queue further overzooming of empty tiles

* Clean up naming; use std::move to avoid copying large arrays
2023-11-08 12:51:28 -08:00
Erica Fischer cc5c1c79df Speed up overzooming in tile-join (#147)
* Clip away entire features by bbox. Avoid unnecessary recompression.

* Move parent tile decoding in tile-join out of overzoom proper

* An ever-growing cache of parent tiles

* Limit the size of the cache

* Remove the current reader *before* checking if we can run the queue

* Clean up

* Add missing #include

* Add comment

* When the tile-join cache fills up, evict the least recently used

* Fix microsecond math

* Factoring out tile-join's cache for testing

* Add unit tests for tile-join cache

* Update changelog and version
2023-10-04 12:14:22 -07:00
Erica Fischer 390771f855 Polygon shards (#105)
* Put all of this back in geometry.cpp for conflict resolution

* Stabilize line simplification to behave the same regardless of winding

* Round instead of truncating when clipping lines

* Restore non-Wagyu polygon clipping from prior to 2fdec7d2

* Make it round, not truncate, which reverts the last commit's test diffs

* Clip in floating point, not integers, which makes no difference

* Track nodes added at tile edges during clipping

* Scale geometry up before wagyu to prevent changes from precision loss

* Actually do the shared edge detection

* Fix cases where nodes were not being added at the tile boundary

* One more place I should have rounded

* Narrow down where the discrepancy comes in

* Revert "Narrow down where the discrepancy comes in"

This reverts commit 221c4c5fc0ac9a6567e091c6a94b3d87dc8ade83.

* Another attempt to narrow it down

* The discrepancy seems to be introduced in reordering. No obvious bug

* Was still truncating instead of rounding in projection

* Also makes no difference...

* Just forget that line reversal exists for a minute

* Just reversal no coalescing

* Try clipping in integers instead of floating point

* Don't simplify after coalescing if they said no simplification

* Check whether behavior is consistent with intentional simplification

* Replace more floating point with integer

* Are these three features enough to demonstrate the problem?

* Add a few more nearby borders

* All the features that touch tile 6/16/23

* Stay in integers in line simplification

* More attempts to solve failures to simplify consistently

* Fix most of the overflow errors

* Fix known cases of integer overflow

* All the tests change again

* Pull clipping and scaling code back out into clip.cpp

* Resolve the test conflicts

* Stabilize choice of which three points to keep with different windings

* Almost right, I think!

* Fix collapse of islands to shards

* Self-intersections in the same feature don't count

* Revert "Self-intersections in the same feature don't count"

This reverts commit e04b19916e.

* Don't scale down geometry if we are going to look for shared nodes

* Fix the missing multiply that was keeping simplification from happening

* Fix one more opportunity for overflow

* Somehow I deleted this test?

* Lost this test too

* Clean up debugging printfs

* Restore code sequence from main to make it reviewable

* Remove unneeded rounding

* Update documentation

* This test is no longer useful

* Back to floating point Douglas-Peucker to fix undersimplification

* Try an older ubuntu

* Revert "Try an older ubuntu"

This reverts commit 13fefacfd7.

* Log OS info

* Remove tests that are no longer needed

* Oops, did need that one after all

* Fix the arm vs x86 discrepancy?

* Try another quantization

* Cleanup from review

* Add a test for the actual purpose of this PR
2023-09-14 16:51:38 -07:00
Erica Fischer 6778aeac52 Add an option to extend zooms if still dropping, but with a limit (#131)
* Add an option to extend zooms if still dropping, but with a limit

* At least when to overzoom, even if not actually doing it yet

* Refactor to give tile-join access to overzoom()

* Didn't work, but *might* have worked

* OK, it did something now

* Ah, there's the bug!

* Hook up pmtiles and dirtiles as overzooming sources

* Add command line option to enable or disable overzooming

* Add (currently broken) test of overzooming in tile-join

* Slightly more abstraction for the tile-join readers

* Factor out duplicated code

* Move construction into a constructor

* More changing accessors to methods

* Reduce magic

* Start tracking a list of the tiles at maxzoom

* I think it worked?

* Add missing #include

* Fix sequence of overzoomed tiles (Y sorts backwards for TMS)

* Don't spend memory on overzooming when we aren't going to use it

* Diff rather than cmp, in the hope of figuring out this broken test

* Keep full coordinate precision if we might extend zooms

* Try a slightly different byte limit

* Make drop-densest more consistent across tile boundaries

* Also affects this test

* Does it behave any differently if it can extend forever?

* I think the discrepancy is a thread-safety problem here

* Revert "Does it behave any differently if it can extend forever?"

This reverts commit 0dff0a0acc.

* Lost this change to the test

* This time for sure!

* Revert "Also affects this test"

This reverts commit cd1f7c2e78.

* Revert "Make drop-densest more consistent across tile boundaries"

This reverts commit 563f7d2bc2.

* Revert "Try a slightly different byte limit"

This reverts commit 2e271213d6.

* Add some more explanatory comments

* Amend the join-test to detect my current bug

* Allow overzooming to complete the zoom if it ever starts

* Forgot to correct the test

* Update changelog and version

* Cleanups from code review

* Remove version number from fixture to fix test
2023-08-25 12:42:38 -07:00
Erica Fischer 430d8edd17 Tool for overzooming individual tiles (#121)
* Starting work on overzooming

* Factor clipping out of geometry.cpp to simplify linkage

* Pull out more geometry functions into now-badly-named clip.cpp

* Not surprisingly, there is a bug

* Found the bug

* Make indent

* Pass attributes through

* Make formatting more consistent

* Fix typos in comments

* Fix the typos better

* Add tippecanoe-overzoom to the install list

* Give overzoom a predictable exit status

* Forgot to translate to and from polygon ring closepaths

* Fix geometry collapse at z21

* Add attribute stripping; don't generate layers if they have no features

* Add docs for tippecanoe-overzoom (as it will be, not as it is)

* Change overzoom to accept input and output files as arguments

* Working on overzooming tests

* Oops

* Hook up and test the detail and buffer options, and the empty-tile case

* Also test attribute stripping

* Fix error message

* Update changelog and version
2023-08-09 11:48:16 -07:00