Commit Graph
1974 Commits
Author SHA1 Message Date
Erica Fischer 65fc99c9d1 For debugging, make an attribute list of source feature IDs test-65fc99c9 2024-09-20 13:23:02 -07:00
Erica Fischer 469b49d83f Update version and changelog 2024-09-19 14:02:36 -07:00
Erica Fischer 3456fa75ed Implement and test attribute prefix stripping in overzoom test-3456fa75 2024-09-19 13:02:56 -07:00
Erica Fischer 5ec93a12b8 Plumbing for exclude and exclude-prefix 2024-09-19 12:39:30 -07:00
Erica Fischer 5e95095284 Test behavior of prefixed attributes 2024-09-19 09:49:38 -07:00
Erica Fischer 100828f019 Attrs with the accumulation prefix are just preserved, not accumulated 2024-09-19 09:42:31 -07:00
Erica Fischer 6f12453100 Factor out numeric preservation 2024-09-19 09:30:03 -07:00
Erica Fischer 0299706c47 Mark unexercised attribute accumulation cases as "can't happen" 2024-09-19 09:00:24 -07:00
Erica Fischer e60ca59502 Fix neglected --accumulate-attribute when binning test-e60ca595 2024-09-18 14:35:39 -07:00
Erica Fischer d3dfe7f45d More testing to make sure cluster sizes make it all the way through 2024-09-18 12:36:38 -07:00
Erica Fischer 1f9be23aba Forgot to update this test test-1f9be23a 2024-09-18 10:49:20 -07:00
Erica Fischer 4ff265912d Make sure the accumulate test still works with a different prefix 2024-09-18 10:40:51 -07:00
Erica Fischer 59026112f6 Make the numeric accumulation prefix configurable 2024-09-18 10:36:09 -07:00
Erica Fischer 21bf44720d Fix count/mean accumulation test-21bf4472 2024-09-17 22:15:17 -07:00
Erica Fischer 172f7382b5 Merge remote-tracking branch 'origin/main' into more-attribute-accumulation test-172f7382 2024-09-17 17:26:39 -07:00
Erica Fischer 19c144761b Make sure IDs make it through on the bins 2024-09-17 17:23:54 -07:00
Erica Fischer aa3ef1fbbf Add binning tests 2024-09-17 17:21:59 -07:00
Erica Fischer 060e2ee1c1 Fix a typo that broke accumulation 2024-09-17 15:16:31 -07:00
Erica Fischer 64b4a820c2 Found a flaw, now to debug 2024-09-17 14:16:58 -07:00
Erica Fischer 9a8c8df007 Checkpoint before I break everything 2024-09-17 14:04:02 -07:00
Erica Fischer 534242887d Pass bin IDs through to the output (#266)
* Pass bin IDs through to the output

* Update version and changelog
2.62.2
2024-09-16 17:08:56 -07:00
Erica Fischer 27a4f4ecef Handle aggregated attributes with no base attribute in the feature 2024-09-16 17:03:05 -07:00
Erica Fischer a2adb60638 Add means, but somehow I have some counts of 0 test-a2adb6063 2024-09-13 12:39:53 -07:00
Erica Fischer 7a0878da9b Fix backwards conditional 2024-09-13 11:32:51 -07:00
Erica Fischer 56655e6ebf Use the same attribute accumulation logic in binning as in megatiles 2024-09-13 11:11:56 -07:00
Erica Fischer e2cc71b42f Put all that behind an --accumulate-numeric flag 2024-09-12 16:10:24 -07:00
Erica Fischer 9739f342e6 Actually do some accumulation 2024-09-12 15:33:14 -07:00
Erica Fischer 50bf0dc3be More sketching 2024-09-12 14:42:21 -07:00
Erica Fischer 88f8b95f27 More sketching 2024-09-12 14:02:39 -07:00
Erica Fischer b234cb9dac Some sort of test for how all these things interact with each other.
Automatic numeric attribute accumulation does *not* apply to attributes
that have an explicit attribute accumulator set, because the order of
operations is too messy and weird
2024-09-12 11:36:16 -07:00
Erica Fischer becfac9da9 Inching toward attribute accumulation in megatile handling 2024-09-12 11:08:21 -07:00
Erica Fischer 78b4fc19ec Quick bbox reject for bins that can't possibly intersect 2024-09-12 10:08:28 -07:00
Erica Fischer 3e3e7a5e4d Plumb bounding boxes through potential intersections 2024-09-12 09:57:39 -07:00
Erica Fischer 84f6e887a7 Another try at fixing longitude wraparound for bins (#261)
* Another try at fixing longitude wraparound for bins

* Gonna get it right this time

* Forgot to update the comment

* The filter case was not supposed to reinterpret geometry

* Copy antimeridian-crossing geometries to the other side too

* Getting closer to getting antimeridian-crossing polygons right

* Update changelog and version
2.62.1
2024-09-10 14:25:57 -07:00
Erica Fischer 51fcf142df Fix bad interaction between dynamic dropping and limiting by truncation (#260)
* Fix bad interaction between dynamic dropping and limiting by truncation

* Don't enforce the tile size limit here if they said not to
2.62.0
2024-09-05 13:39:53 -07:00
Erica Fischer 5b18eea673 Work in progress on binning features in overzoom (#258)
* Factoring out tilestats management from GeoJSON file reading

* Move code around so overzoom can link against parse_layers

* Read the file of bins

* Plumb the bins through to overzoom()

* Some zip code bins to test with

* (Currently non-functional) test of binning

* Starting to spell out the bin matching loop

* Can't flatten points, so don't flatten bins either

* More fleshing out bin traversal

* Bounding box of tile-relative mvt geometry

* Smallest enclosing tile from bbox

* Most of the bin scan

* Add point in polygon check. It crashes.

* Find the matching bins

* GDAL-style bounding boxes have eaten my brain

* Make some features to bin into

* Increment a count as features are found to be within the bins

* Fix longitude wraparound in overzoom bins

* Fix the tests

* Push off attribute copying until after bin assignment

* Carry sum of numeric attributes into the bins

* Also add mean, min, and max

* Add --calculate-feature-index since I keep needing it for testing

* Add an option to accumulate sum/mean/max/min/count of all numeric attrs

* Don't bake in tippecanoe:mean, since we redo it from sum and count

* Forgot to update this test fixture after removing tiled mean

* Update version and changelog
2024-09-05 12:06:51 -07:00
Monty Taylor f3b575ead4 Minor dockerfile optimizations (#240)
* Replace build-essential in Docker builder image

build-essential is overkill, it pulls in all of the tools one needs to
make to build debian packages, which we are not doing. Even though this
is a builder image, it still takes time to download all the extra crud.

Replace build-essential with make gcc and g++, which are all that are
needed.

* Replace dev packages in final Docker image

The dev packages are only needed in the builder image. They add header
files and docs and things that are not needed in the final image.

Removing them and replacing them with just the runtime libraries reduces
the final image size by about 25% (160M->124M) in a local build.

Additionally, just for simplicity, zlib1g is already part of base
ubuntu minimal, so we don't even have to list it.

* Remove build-essential mention from the README

Just like the in Dockerfile, people don't need all of those packages
to build directly on their system.
2024-08-28 10:35:21 -07:00
Rizki Alhamid f44230f5f5 Fix a bug when reading MultiLineString from .fgb file (#241)
fix logic
2024-08-28 10:34:33 -07:00
Erica Fischer 40bb4ff732 Be more careful to retry when the feature count is exceeded (#257)
* Clip before dealing with multiplier or filters in overzoom

* Be more careful to retry when the feature count is exceeded

* Adjust the estimated total feature count for the multiplier too

* Fix the feature count estimates, I think

* Pass build info into the version string

* Report the actual max zoom of any tiles as the metadata maxzoom

* Revert unneeded renaming to make the diff more readable

* Clean up the adjustments to tile sizes and feature counts

* Update version and changelog

* Dropping a feature into a multiplier cluster still effectively drops it

* Update changelog

* Rethink the changelog description

* Don't try to truncate zooms if we are still tiling at z18
2.60.0
2024-08-20 10:54:16 -07:00
Erica Fischer d891ca7289 Fix latitude bboxes for features that extend beyond the mercator plane (#254)
* Fix latitude bboxes for features that extend beyond the mercator plane

* Update some more test expectations

* Update version and changelog
2.59.0
2024-08-08 15:30:52 -07:00
Erica Fischer bc3ef87c3f Add --generate-variable-depth-tile-pyramid option (#251)
* Track output position at the file level instead of within each tile

* Track file position where the child tile data begins

* Add option and document its intended behavior

* Changing the detail loop to account for stopping early

* I forgot I already added an option for this

* Stop early if we can make a complete tile

* Add a test of zoom truncation with limited feature count

* Forgot to commit the actual code change

* Make room for a vertex count in the header of each serialized tile

* Estimate tile complexity; don't try truncating when unlikely to work

* Be more conservative, because ever retrying a tile is a big speed hit

* If stopping early, don't simplify or clean; leave that to overzoom

* Add tiny polygon reduction / dust to overzoom

* Don't try to stop early in the children if we dropped anything by rate

* Fflush here too before pwriting

* Don't stop early if we ended up dropping any features.

Rework the can-the-next-zoom-stop-early logic to avoid going
one zoom further than needed.

* Fix warning

* Fix warnings

* Oops, checking for the wrong expected return value

* Cleanup from adding line simplification in overzoom

* Current (wrong) behavior when combining coalescing and truncating

* Keep a list of parent tiles to skip rather than truncating

* Now the coalesced tiles in z12 get children in z13

* Don't double-count feature dropping when the zoom level is retried

* Correct README description

* Remove todo about special case below basezoom, which is accounted for

* Be a little more aggressive in drop-densest determination

* Scale tile feature limit for megatiles in the same way as byte limit

* Fully deprecate -detect-shared-borders into an alias

* Track the distances found in the douglas-peucker recursion

* Serialize and deserialize the distance with the vertices

* Revert "Serialize and deserialize the distance with the vertices"

This reverts commit 753f1b7909.

* Revert "Track the distances found in the douglas-peucker recursion"

This reverts commit e5361f8c22.

* Revert "Fully deprecate -detect-shared-borders into an alias"

This reverts commit 0698aeb766.

* Better tracking of whether we failed to make a full-detail tile

* Put a bloom filter in front of the binary search for shared nodes

* Forgot to take out this printf

* Improve dispatch of tiling tasks

* Still dispatch the biggest tasks first

* Track zoom truncation in the strategies list in the tileset metadata

* Prescan for small deltas before doing proper simplification

* Revert "Prescan for small deltas before doing proper simplification"

This reverts commit d1d8238b83.

* Update version and changelog

* Rename to --generate-variable-depth-tile-pyramid
2.58.0
2024-08-06 16:05:52 -07:00
Erica Fischer 50deb9ce63 Add multi-tile input to tippecanoe-overzoom (#249)
* Reviving multi-source-tile overzoom: the clip.cpp side

* Reviving multi-source-tile overzoom: the overzoom.cpp side

* Update readme

* Update version and changelog
2.57.0
2024-07-23 13:21:10 -07:00
Erica Fischer 47e774adc4 Improve the appearance of coalesce-densest-as-needed tiles (#247)
* Start to distinguish fixed cluster density setting from as-needed density

* Make consistent {drop,coalesce}-densest decisions between zooms

* Actually track the previous index instead of just intending to

* Clean up collinearities in coalesced features

* To determine densest, look at actual physical distance, not just index

* Don't actually need the previous index in serial_feature now

* Center of mass of one feature to most distant point of the next

* Add apologetic comment

* Wait, how did the tests pass before?

* Revert "Wait, how did the tests pass before?"

This reverts commit f73c8ee543.

* Add --maximum-string-attribute-length option

* Update version and changelog

* A little more testing to make sure
2.56.0
2024-07-16 13:23:23 -07:00
Drew Good 939579e5a2 Update README (#244) 2024-07-03 11:10:35 -07:00
Erica Fischer e11583df1c Fix hash collision in the string pool (#239)
* Current broken behavior

* More blatant test

* Fix hash collision in string pool

* Update version and changelog
2.55.0
2024-06-07 15:01:14 -07:00
Erica Fischer bb4f220678 Reduce tiling memory (#227)
* Trying to reduce memory in tiling

* Let the simplification workers go out of scope earlier

* Bail out quickly once the maximum feature count is reached

* Update changelog and version
2.53.0
2024-04-03 07:12:26 -07:00
Dane Springmeyer 266ef3c286 Add tippecanoe-overzoom to .gitignore (#224)
add tippecanoe-overzoom to .gitignore
2024-03-25 11:00:18 -07:00
Dane Springmeyer 1519b50791 Upgrade catch from v1.5.7 to v2.13.10 (#223)
upgrade catch from v1.5.7 to v2.13.10
2024-03-25 10:39:11 -07:00
Erica Fischer bd48ba8ea1 Fix accidental loss (at all zooms) of features with an explicit minzoom (#221)
* Fix accidental loss (at all zooms) of features with an explicit minzoom

* Try to stabilize centers and bounding boxes between architectures
2.52.0
2024-03-22 10:20:10 -07:00
Erica Fischer e796400377 Fix null behavior in in/ni expressions (#214)
* Fix null behavior in in/ni expressions

* Add test for null attributes in "ni" filters

* Remove unidecode from "in" expression evaluation again
2.51.0
2024-03-08 13:01:12 -08:00