Commit Graph
38 Commits
Author SHA1 Message Date
Denis Govorkov 9d1637f7c1 Option to not averaging clusters of points (#326) 2025-03-19 13:56:36 -07:00
Erica Fischer c5f2f0da34 More work on plumbing attribute accumulation through (#263)
* Plumb bounding boxes through potential intersections

* Quick bbox reject for bins that can't possibly intersect

* Inching toward attribute accumulation in megatile handling

* Some sort of test for how all these things interact with each other.

Automatic numeric attribute accumulation does *not* apply to attributes
that have an explicit attribute accumulator set, because the order of
operations is too messy and weird

* More sketching

* More sketching

* Actually do some accumulation

* Put all that behind an --accumulate-numeric flag

* Use the same attribute accumulation logic in binning as in megatiles

* Fix backwards conditional

* Add means, but somehow I have some counts of 0

* Handle aggregated attributes with no base attribute in the feature

* Checkpoint before I break everything

* Found a flaw, now to debug

* Fix a typo that broke accumulation

* Add binning tests

* Make sure IDs make it through on the bins

* Fix count/mean accumulation

* Make the numeric accumulation prefix configurable

* Make sure the accumulate test still works with a different prefix

* Forgot to update this test

* More testing to make sure cluster sizes make it all the way through

* Fix neglected --accumulate-attribute when binning

* Mark unexercised attribute accumulation cases as "can't happen"

* Factor out numeric preservation

* Attrs with the accumulation prefix are just preserved, not accumulated

* Test behavior of prefixed attributes

* Plumbing for exclude and exclude-prefix

* Implement and test attribute prefix stripping in overzoom

* Update version and changelog

* For debugging, make an attribute list of source feature IDs

* Revert "For debugging, make an attribute list of source feature IDs"

This reverts commit 65fc99c9d1.
2024-09-20 15:30:29 -07:00
Erica Fischer 5b18eea673 Work in progress on binning features in overzoom (#258)
* Factoring out tilestats management from GeoJSON file reading

* Move code around so overzoom can link against parse_layers

* Read the file of bins

* Plumb the bins through to overzoom()

* Some zip code bins to test with

* (Currently non-functional) test of binning

* Starting to spell out the bin matching loop

* Can't flatten points, so don't flatten bins either

* More fleshing out bin traversal

* Bounding box of tile-relative mvt geometry

* Smallest enclosing tile from bbox

* Most of the bin scan

* Add point in polygon check. It crashes.

* Find the matching bins

* GDAL-style bounding boxes have eaten my brain

* Make some features to bin into

* Increment a count as features are found to be within the bins

* Fix longitude wraparound in overzoom bins

* Fix the tests

* Push off attribute copying until after bin assignment

* Carry sum of numeric attributes into the bins

* Also add mean, min, and max

* Add --calculate-feature-index since I keep needing it for testing

* Add an option to accumulate sum/mean/max/min/count of all numeric attrs

* Don't bake in tippecanoe:mean, since we redo it from sum and count

* Forgot to update this test fixture after removing tiled mean

* Update version and changelog
2024-09-05 12:06:51 -07:00
Erica Fischer bc3ef87c3f Add --generate-variable-depth-tile-pyramid option (#251)
* Track output position at the file level instead of within each tile

* Track file position where the child tile data begins

* Add option and document its intended behavior

* Changing the detail loop to account for stopping early

* I forgot I already added an option for this

* Stop early if we can make a complete tile

* Add a test of zoom truncation with limited feature count

* Forgot to commit the actual code change

* Make room for a vertex count in the header of each serialized tile

* Estimate tile complexity; don't try truncating when unlikely to work

* Be more conservative, because ever retrying a tile is a big speed hit

* If stopping early, don't simplify or clean; leave that to overzoom

* Add tiny polygon reduction / dust to overzoom

* Don't try to stop early in the children if we dropped anything by rate

* Fflush here too before pwriting

* Don't stop early if we ended up dropping any features.

Rework the can-the-next-zoom-stop-early logic to avoid going
one zoom further than needed.

* Fix warning

* Fix warnings

* Oops, checking for the wrong expected return value

* Cleanup from adding line simplification in overzoom

* Current (wrong) behavior when combining coalescing and truncating

* Keep a list of parent tiles to skip rather than truncating

* Now the coalesced tiles in z12 get children in z13

* Don't double-count feature dropping when the zoom level is retried

* Correct README description

* Remove todo about special case below basezoom, which is accounted for

* Be a little more aggressive in drop-densest determination

* Scale tile feature limit for megatiles in the same way as byte limit

* Fully deprecate -detect-shared-borders into an alias

* Track the distances found in the douglas-peucker recursion

* Serialize and deserialize the distance with the vertices

* Revert "Serialize and deserialize the distance with the vertices"

This reverts commit 753f1b7909.

* Revert "Track the distances found in the douglas-peucker recursion"

This reverts commit e5361f8c22.

* Revert "Fully deprecate -detect-shared-borders into an alias"

This reverts commit 0698aeb766.

* Better tracking of whether we failed to make a full-detail tile

* Put a bloom filter in front of the binary search for shared nodes

* Forgot to take out this printf

* Improve dispatch of tiling tasks

* Still dispatch the biggest tasks first

* Track zoom truncation in the strategies list in the tileset metadata

* Prescan for small deltas before doing proper simplification

* Revert "Prescan for small deltas before doing proper simplification"

This reverts commit d1d8238b83.

* Update version and changelog

* Rename to --generate-variable-depth-tile-pyramid
2024-08-06 16:05:52 -07:00
Erica Fischer e6d05bc317 Fix tile-join crash when trying to merge empty tilesets with --overzoom (#138)
* Fix tile-join crash when trying to merge empty tilesets with --overzoom

* Add an option not to reduce tiny polygons to dust at maxzoom

* Add test for prevention of tiny polygon reduction at maxzoom

* Change version number
2023-08-31 13:02:03 -07:00
Erica Fischer 8eec2be462 Improve quality of coalescing-as-needed; add options to limit feature count (#25)
* Add an option to limit geometry vertex count

* Update docs and changelog

* Track desired feature count and geometry size in strategies

* Remove dead code for a long-forgotten inaccessible option

* Fix the option name in changelog

* Refine coalesce-smallest to only coalesce onto other small features

* Clean polygons before coalescing-as-needed

* Don't accumulate tiny polygon holes as negative dust

* Add the option not to limit the feature count at maxzoom

* Add options to limit feature count more abruptly in each tile

* Revert "Add the option not to limit the feature count at maxzoom"

This reverts commit ace173ab7a.

* Revert "Fix the option name in changelog"

This reverts commit 38dbce8405.

* Revert "Add an option to limit geometry vertex count"

This reverts commit d379fdf06b.

* Remove test for reverted option

* Remove more leftovers from geometry size limiting

* Update changelog
2022-11-17 13:09:10 -08:00
Erica Fischer b763625862 Add an option to generate label points in place of polygons (#20)
* Add an option to generate label points in place of polygons

* Change all these places where I said "extent" but really meant "area"

* Revert "Change all these places where I said "extent" but really meant "area""

This reverts commit 403828d2f7.

* Add --order-smallest-first and --order-largest-first options

* Use Turf's center-of-mass algorithm for polygon label points

* If the label point isn't within the polygon, find one that is

* Don't choose a label point that is too close to a border

* Try a little harder to find an optimal label point

* Checkerboard which tiles labels are generated in, to reduce adjacency

* Use a label point for the general representative point for polygons

(Skipping the iteration to find one that is as far as possible from
the borders)

This makes the labels look better in many cases (like France at z1)
but unfortunately ripples into changing the sequence of polygons in
many tests, so the diff is big.

* Revert "Use a label point for the general representative point for polygons"

This reverts commit 2261adf05e.

* Checkpoint work on spiral labels

* Clip label spirals to the feature bounds

* Fix label test

* Be careful not to place spiral labels too close to borders either

* For spiral anchors, only check tile scale, not feature size

* Update test

* Only try to find a central label point for the largest ring

* In tiny polygon dust, keep the attributes of the largest feature
2022-10-13 14:21:36 -07:00
Erica Fischer 4ef430b913 Add an option for the Visvalingam simplification algorithm (#16)
* First attempt at porting Paul Mach's Visvalingam implementation

* Make indent

* Mostly working

* Approximate equivalence from Douglas-Peucker to Visvalingam levels

* Don't simplify away tile boundary crossing points

* Add a command-line option and test for Visvalingam simplification

* Update changelog

* Cleanup in response to review feedback

* Include <stdio.h> to fix compiler warnings
2022-09-30 12:00:07 -07:00
Erica Fischer a447dfc089 Extra coordinate precision; feature ordering; compression improvements
* Add an option to retain extra coordinate precision at maxzoom

* Make sure not to shift away the extra detail from coordinates

* Add an option to convert double-precision attributes to single

* Sort attribute values in tiles to make them compress a little better

* Slightly improve polygon simplification

By choosing a point that would be retained after simplification
to be the start/end point that always gets retained

* I regret making all of these tests involve polygons

* Add an option to specify the size of tiny polygons

* Fix accidental requiring of argument for --single-precision

* Guard against duplicate points when generating "sizes" for them

* Restore the intended behavior that tiny polygons don't get simplified

* Make the extra detail settable rather than always maximizing it

* Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)"

This reverts commit fec5e8354c.

* Add an option to prevent choosing a base zoom higher than the maxzoom

* Keep the drop rate high enough when the basezoom gets constrained

* Revert "Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)""

This reverts commit db6bc27d9e.

* Add --order-by and --order-descending options

* Accept multiple --order-by and --order-descending-by sort keys
2022-09-06 13:08:11 -07:00
Eric Fischer 42ee195bc3 Add an option to keep intersection nodes from being simplified away 2019-05-13 17:39:07 -07:00
Eric Fischer 6376c283ba Add Hilbert Curve feature sequencing as an option 2018-12-12 10:51:00 -08:00
Eric Fischer 53de8a2014 Add an option to convert stringified number feature IDs to numbers 2018-11-01 17:07:33 -07:00
Eric Fischer 6f295f29fc Add an option to automatically assign ids to features 2018-07-26 15:08:47 -07:00
Eric Fischer 87a1bb7851 Add an option to treat empty CSV columns as nulls, not empty strings 2018-07-19 14:33:33 -07:00
Eric Fischer 48486b5d7b Add --use-source-polygon-winding and --reverse-source-polygon-winding 2018-06-01 12:59:28 -07:00
Eric Fischer 2e5b597ca1 Add --coalesce-fraction-as-needed and --coalesce-densest-as-needed 2018-02-06 14:39:05 -08:00
Eric Fischer b03dabe098 Add an option for clustering features 2017-12-14 14:30:08 -08:00
Eric Fischer be922702ce Experiment with coalescing features to reduce tile size 2017-10-06 18:01:08 -07:00
Eric Fischer 372194cee9 Add an option not to produce tilestats 2017-08-15 10:35:41 -07:00
Eric Fischer 65c095cc2b Clean up #includes and add fields for counting attributes 2017-07-14 16:56:23 -07:00
Eric Fischer cf3a0800b8 Add an option to increase maxzoom if features are still being dropped 2017-05-31 10:47:34 -07:00
Shan-Chun Kuo a320248857 Do make indent, add flag --output-to-directory or -e to write pbf files to a directory, also add flag --no-tile-compression or -pC to get raw protobuf 2017-04-07 13:33:01 +01:00
Shan-Chun Kuo aee12ac1fe Add flag --raw-tiles or -pC to get raw protobuf 2017-04-05 00:34:54 +01:00
Eric Fischer ea77e3db6f Add --detect-longitude-wraparound option 2017-03-15 13:37:50 -07:00
Eric Fischer 0db8d9ed8b Option to snap low zooms to a stairstep grid 2016-11-28 15:45:38 -08:00
Eric Fischer e9aa8c1b7d Add an option to prevent tiny polygon reduction 2016-11-21 11:26:37 -08:00
Eric Fischer dc9e68b128 Make feature-dropping option names a little more consistent 2016-11-17 12:40:11 -08:00
Eric Fischer 013e6512b4 Add an option to drop the smallest features to make tiles small enough 2016-11-09 17:09:05 -08:00
Eric Fischer 38ce49d2d4 Another option for plain fractional dropping, but across the whole zoom 2016-11-04 12:26:13 -07:00
Eric Fischer 10fc9254d1 Add an option to discover the minimum workable gap between features 2016-10-25 16:28:32 -07:00
Eric Fischer 2798bf7b6f Add an option to merge adjacent polygons together to reduce tile size 2016-10-24 17:22:07 -07:00
Eric Fischer 83e73e8840 Add an option to dynamically increase gamma until tiles are small enough 2016-10-24 12:29:36 -07:00
Eric Fischer 896c9d8398 Rename option to --detect-shared-borders and add a test 2016-10-06 16:16:51 -07:00
Eric Fischer f7daa05515 Add a critical point where the set of rings using a polygon edge changes 2016-10-05 17:16:18 -07:00
Eric Fischer 692112ec3a Add an option to calculate feature density as a feature attribute 2016-05-27 16:25:40 -07:00
Eric Fischer 1d636e5c0d Consistently treat "prevent" and "additional" options as globals 2016-05-25 11:38:52 -07:00
Eric Fischer 43ffd6fe11 Fix the warnings about the unused array of option names 2016-05-03 16:34:19 -07:00
Eric Fischer ee97e6c307 Whittle down tile.hpp to the things that actually related to tile.cpp 2016-04-27 15:09:06 -07:00