Commit Graph
1991 Commits
Author SHA1 Message Date
Erica Fischer 9fb7b6882d There aren't too many features if we intentionally truncated them 2025-05-06 11:02:48 -07:00
Erica Fischer 143f5c87f9 Report the miscalculation of feature dropping, but don't stop test-143f5c8 2025-05-06 10:53:02 -07:00
Erica Fischer db3ca59c21 Don't bother coalescing geometry that is too small to be seen test-db3ca59 2025-05-06 09:41:03 -07:00
Erica Fischer c594fcf4c9 Move polygon dust back to after as-needed dropping and coalescing 2025-05-06 09:28:26 -07:00
Erica Fischer c1ca367893 Fix potentially unintialized variable test-c1ca367 2025-05-05 15:50:16 -07:00
Erica Fischer c6debdcbd7 Fix overflow 2025-05-05 15:09:55 -07:00
Erica Fischer 887db65698 Divide and conquer when cleaning complex polygons after coalescing 2025-05-05 13:12:12 -07:00
Erica Fischer b7b5c8484f Update tests before I make further changes to behavior 2025-05-05 12:36:11 -07:00
Erica Fischer 8425e0ebca Search more aggressively for feature-dropping thresholds test-8425e0e 2025-05-02 16:56:15 -07:00
Erica Fischer 6cc8703cd7 Fix test 2025-04-28 16:51:27 -07:00
Erica Fischer 60239d5b1c Remove debug printf 2025-04-28 16:32:33 -07:00
Erica Fischer c69b61abef Ripple effects into other tests 2025-04-28 16:28:07 -07:00
Erica Fischer ee391ec481 Straighten out the order of events so labels can be dropped 2025-04-28 16:24:38 -07:00
Erica Fischer 2dea09dd01 A test fixture that still gets stuck 2025-04-28 14:33:53 -07:00
Erica Fischer 03647fb4a5 Convert label anchors earlier in tiling to keep from getting stuck later test-03647fb 2025-04-28 12:56:15 -07:00
Erica Fischer a52d15a0b1 Working on label anchor loops 2025-04-28 11:21:44 -07:00
Erica Fischer 9037920faf Make sure we don't index off the end of the array here test-9037920 2025-04-25 15:59:10 -07:00
Erica Fischer 82e8d96ea6 Add unit test for line_is_too_small test-82e8d96 2025-04-25 13:40:49 -07:00
Erica Fischer a84864e795 Fix too_small geometry test for multigeometries 2025-04-25 13:31:55 -07:00
Erica Fischer 1ba93dc302 Prevent potential infinite loops in as-needed dropping and coalescing 2025-04-25 12:06:31 -07:00
Erica Fischer 94929b048c Add --deduplicate-by-id option to tippecanoe-overzoom (#331)
* Add option to deduplicate by feature ID in overzoom

* Add test, fix default

* Update version and changelog
2.77.0
2025-04-03 12:48:29 -07:00
Erica Fischer bfb62ee2db Add missing case for accumulating the mean of attributes that are inconsistently present (#329)
* Add missing case for accumulating the mean of attributes that are inconsistently present

* Add more specific test
2025-03-20 14:53:04 -07:00
Robert Martin a0532e73ac docs: fix typo in readme re: gamma (#325) 2025-03-19 13:57:23 -07:00
Denis Govorkov 9d1637f7c1 Option to not averaging clusters of points (#326) 2025-03-19 13:56:36 -07:00
cbobinec de4e1e478f docs : fix small error in JSON keys and values example (#327)
fix error in JSON keys and values example
2025-03-19 13:54:05 -07:00
Erica Fischer 423fea1544 Reduce memory consumption during tiling and tile encoding (#319)
* Improve the memory spike during tile construction

* Remove `need_tilestats`. Add a bunch of debug logging

* Remove debug logging

* Update version and changelog
2.75.1
2025-01-31 12:57:41 -08:00
Erica Fischer 583fc3744a Reduce attribute accumulation memory consumption (#318)
* Add a flag to use an H3 index for the feature index

* Give mvt_value and serial_val a double-with-count concept

* Switch mean over to internal accumulation state

* Get rid of the attribute accumulation map

* Change vectors of features to vectors of pointers to features

* Fix --coalesce

* Revert "Add a flag to use an H3 index for the feature index"

This reverts commit b9b48f42c9.

* Update version and changelog
2.75.0
2025-01-30 21:58:35 -08:00
Ryan Mast 390c362452 Add native AArch64 Linux CI test jobs (#314) 2025-01-17 14:09:02 -08:00
Erica Fischer 10f7f0a3c2 Support joins from sqlite tables in tile-join (#308)
* Sketching out sqlite options for tile-join

* Enable sqlite3 serialized multithreading

* Fix some unnecessary round trips from std::string to char * and back

* Gathering join keys for sql query

* Make a query

* Actually open the gpkg. Fix the query quoting.

* Actually do the query and get results back

* Join the attributes onto the feature

* Set matched if the sql join matches

* Add a flag to get the feature ID from the query

* Observe attribute exclusion when joining from sql queries

* Add a flag to exclude all attributes from the tile side of the join

* Make tile-join bounding boxes reflect feature bounds, not tile bounds

* More tests

* An empty tileset has empty bounds at null island

* Gather the results from each thread *after* the thread finishes

* Missed a test

* Fix accidental inclusion of the top left of the tile in the bbox

* Case smashing and prefix trimming in the select

* Add test of sql join

* Make the join column option a join expression option

* Fix antimeridian adjustment. Z0 can't wait until the end of the tile

* Checkpoint on accepting multiple joined rows per tiled feature

* Adding the joined attribute should be per-feature, not per-attribute

* Forgot to update the test. Order of joined attributes has changed.

* Get the attributes back in the right order

* Add test of sql join with limit

* Allow multiple tile features to have the same join key

* Update tests for a country name with two distinct geometries

* Update version and changelog

* Forgot to mention the bounding box improvements
2.74.0
2025-01-13 12:25:00 -08:00
Erica Fischer 7165ae6999 Fix clipping bug when the clip region doesn't intersect the tile (#312) 2.73.0 2024-12-11 10:24:12 -08:00
Ryan Mast d70e3326da Consolidate code for getting platform details into one place (#311)
Consolidate code for getting platform details into a single place
2024-12-09 15:21:50 -08:00
Ryan Mast a8bd7ac723 Add macOS CI jobs (#310) 2024-12-06 09:01:55 -08:00
Erica Fischer bdfb06cd6a Make tippecanoe-overzoom accept filters from a file (#307)
* Make tippecanoe-overzoom accept filters from a file

* Accept clip polygons from a file too

* Add test of clipping by polygon from file

* Add a test of reading an overzoom filter from a file

* Update version and changelog
2.72.0
2024-12-05 12:05:50 -08:00
Sergey Fedorov 1d81935cd5 mbtiles.cpp: isinf to std::isinf (#303) 2024-11-30 22:33:56 -08:00
Erica Fischer dcc616d3d4 Adding optional clipping to tippecanoe-overzoom (#298)
* Plumb a clip bounding box around through overzoom

* Actually do some clipping

* Add a test

* Fix post-binning clipping

* Factoring out geometry parsing from feature parsing

* Accept a clip polygon argument to tippecanoe-overzoom

* Progress in the direction of polygon clipping

* Fix the wagyu flags. We need intersection, not union

* Remove debug spew

* Clip points to polygon bounds too

* Copy the geometric binning code to serve as intersection-finding code

* Add clipper2 for linestring clipping

* Compiles, but does not actually seem to clip. Hmm.

* Oh, it helps if I actually call the function

* Add clipping tests

* Add missing fixture, and don't crash if it is missing

* Remember to do polygon clipping after binning too

* Fix scaling before post-binning clipping. Add test.

* Remove unused parts of clipper

* Rename for consistency

* Revert accidentally added line

* Clip the clip regions to the tile bounds to reduce their complexity

* Add a test of clipping the clip region down to the tile boundary

* Update version and changelog
2.71.0
2024-11-27 13:43:40 -08:00
Erica Fischer 11e3196c9a Raise tippecanoe-decode tile size limit to 250 MB (#299)
* Raise tippecanoe-decode tile size limit to 250 MB

* Update version and changelog
2.70.1
2024-11-21 09:17:22 -08:00
Erica Fischer 905b58cdf4 Reducing attribute tagging within tippecanoe-overzoom (#296)
* Strip out unwanted attributes earlier in the process

* Skip aggregations whose attributes have been excluded

* Forgot one

* Add some more tests

* Update version and changelog
2.70.0
2024-11-15 13:14:30 -08:00
Erica Fischer 6d46578ad8 Avoid crash if the first bin gets clipped away (#294)
* Avoid crash if the first bin gets clipped away

* Add test

* Update version and changelog
2.69.0
2024-11-13 10:48:33 -08:00
Erica Fischer 794ae2b1eb Tippecanoe-decode and tippecanoe-overzoom changes to improve binning performance (#292)
* Add a tippecanoe-decode option to restrict which attributes to decode

* Plumb buffer and feature limit around

* Check the feature limit

* Clarifying cases where output detail can be unspecified

* Clip bins to the tile buffer instead of just passing them through

* Add missing include

* Missed some tests

* Add --no-tile-compression option to tippecanoe-overzoom

* Update version and changelog
2.68.0
2024-11-08 15:46:04 -08:00
Erica Fischer 23667bb8eb Reduce memory consumption from attribute accumulation (#290)
* Progress on plumbing a string pool for full_keys through

* More plumbing for key_pool

* Don't keep features with identical locations as multiplier features

* Revert "Don't keep features with identical locations as multiplier features"

This reverts commit 413f0c8024.

* Adjust calculated maxzoom to account for duplicate feature locations

* Update changelog and version

* Add a test affected by the maxzoom change with duplicate locations

* Round the drop rate a little for cross-platform test consistency
2.67.0
2024-11-05 14:39:41 -08:00
Erica Fischer b3b89e1e07 Performance optimizations for binning (#283)
* Add an all-mvt_value attribute accumulation path

* Only bin by ID, not geometrically

* A little cleanup; changelog and version; test

* Remove accidental double-conversion

* Replace duplicated code with template

* Update changelog
2.66.0
2024-11-01 15:27:13 -07:00
Patrick M YoungandPatrick Young 120b8f4e41 Fixes bool atrributes coming from FGB files (#289)
Fixes https://github.com/felt/tippecanoe/issues/180.

Co-authored-by: Patrick Young <pyoung@snapchat.com>
2024-11-01 12:27:05 -07:00
Erica Fischer 28efc40e6e Choose the megatile features from those that will be in the next N zooms (#280)
* Choose the megatile features from those that will be in the next N zooms

* Take fractional zooms into account in multiplier feature choices

* Fix more tests

* Add a flag to retain multiplier features by minimum distance

* Limit feature expansion from multiplier density to 2x

* The multiplier cap was a bad idea

* Revert "The multiplier cap was a bad idea"

This reverts commit 6f8273a4c8.

* Revert "Limit feature expansion from multiplier density to 2x"

This reverts commit a26e41309d.

* Revert "Add a flag to retain multiplier features by minimum distance"

This reverts commit 01f14a4255.

* Remove the multiplier sequence, which should no longer matter

* Revert "Revert "Add a flag to retain multiplier features by minimum distance""

This reverts commit 776da4a1b8.

* Revert "Revert "Limit feature expansion from multiplier density to 2x""

This reverts commit 44a683d808.

* Revert "Revert "The multiplier cap was a bad idea""

This reverts commit 80f7cb1c0e.

* Track two kinds of previous index for next_feature

* Fix multiplier density threshold, I think

* Oh, I didn't git add the code changes

* Update version and changelog

* Try to install sqlite3 to fix the automated build

* Deleted too much

* Only let --preserve-point-density-threshold shift density around

* Remove the density debt concept, since it doesn't help

* Make the drop states a vector instead of an array

* Revert "Make the drop states a vector instead of an array"

This reverts commit 66c7abb6fa.

* Revert "Remove the density debt concept, since it doesn't help"

This reverts commit 707bb0c562.

* Revert "Only let --preserve-point-density-threshold shift density around"

This reverts commit ecf01f2231.
2.65.0
2024-10-17 15:59:44 -07:00
Erica Fischer 78661f1cb2 Binning by ID (#276)
* Binning by ID

* Add test of binning by ID
2.64.0
2024-10-03 10:56:58 -07:00
Erica Fischer 8409bf9ce4 Top-level null filter now evaluates to true (#277) 2.63.0 2024-10-02 15:19:30 -07:00
Erica Fischer 23c6559b03 Remove buggy optimization to avoid reclipping in overzoom (#275)
* Remove buggy optimization to avoid reclipping in overzoom

* Add clarifying comment
2024-10-01 10:22:41 -07:00
Erica Fischer 66e5d66300 Another round of binning fixes (#274)
* Bin more aggressively if a point doesn't meet the pnpoly test

* Don't clip the points if we are binning

* Fix output of features added to the bin after its closure

* Update tests

* Update version and changelog
2.62.5
2024-09-30 08:57:13 -07:00
Erica Fischer 4822d02f24 Fix count accumulation in overzoom (#272)
* Fix count accumulation in overzoom

* Add test

* Increment version and changelog
2.62.4
2024-09-24 23:55:43 -07:00
Erica Fischer c5f2f0da34 More work on plumbing attribute accumulation through (#263)
* Plumb bounding boxes through potential intersections

* Quick bbox reject for bins that can't possibly intersect

* Inching toward attribute accumulation in megatile handling

* Some sort of test for how all these things interact with each other.

Automatic numeric attribute accumulation does *not* apply to attributes
that have an explicit attribute accumulator set, because the order of
operations is too messy and weird

* More sketching

* More sketching

* Actually do some accumulation

* Put all that behind an --accumulate-numeric flag

* Use the same attribute accumulation logic in binning as in megatiles

* Fix backwards conditional

* Add means, but somehow I have some counts of 0

* Handle aggregated attributes with no base attribute in the feature

* Checkpoint before I break everything

* Found a flaw, now to debug

* Fix a typo that broke accumulation

* Add binning tests

* Make sure IDs make it through on the bins

* Fix count/mean accumulation

* Make the numeric accumulation prefix configurable

* Make sure the accumulate test still works with a different prefix

* Forgot to update this test

* More testing to make sure cluster sizes make it all the way through

* Fix neglected --accumulate-attribute when binning

* Mark unexercised attribute accumulation cases as "can't happen"

* Factor out numeric preservation

* Attrs with the accumulation prefix are just preserved, not accumulated

* Test behavior of prefixed attributes

* Plumbing for exclude and exclude-prefix

* Implement and test attribute prefix stripping in overzoom

* Update version and changelog

* For debugging, make an attribute list of source feature IDs

* Revert "For debugging, make an attribute list of source feature IDs"

This reverts commit 65fc99c9d1.
2.62.3
2024-09-20 15:30:29 -07:00
Erica Fischer 534242887d Pass bin IDs through to the output (#266)
* Pass bin IDs through to the output

* Update version and changelog
2.62.2
2024-09-16 17:08:56 -07:00