* Sketching out sqlite options for tile-join
* Enable sqlite3 serialized multithreading
* Fix some unnecessary round trips from std::string to char * and back
* Gathering join keys for sql query
* Make a query
* Actually open the gpkg. Fix the query quoting.
* Actually do the query and get results back
* Join the attributes onto the feature
* Set matched if the sql join matches
* Add a flag to get the feature ID from the query
* Observe attribute exclusion when joining from sql queries
* Add a flag to exclude all attributes from the tile side of the join
* Make tile-join bounding boxes reflect feature bounds, not tile bounds
* More tests
* An empty tileset has empty bounds at null island
* Gather the results from each thread *after* the thread finishes
* Missed a test
* Fix accidental inclusion of the top left of the tile in the bbox
* Case smashing and prefix trimming in the select
* Add test of sql join
* Make the join column option a join expression option
* Fix antimeridian adjustment. Z0 can't wait until the end of the tile
* Checkpoint on accepting multiple joined rows per tiled feature
* Adding the joined attribute should be per-feature, not per-attribute
* Forgot to update the test. Order of joined attributes has changed.
* Get the attributes back in the right order
* Add test of sql join with limit
* Allow multiple tile features to have the same join key
* Update tests for a country name with two distinct geometries
* Update version and changelog
* Forgot to mention the bounding box improvements
* Make tippecanoe-overzoom accept filters from a file
* Accept clip polygons from a file too
* Add test of clipping by polygon from file
* Add a test of reading an overzoom filter from a file
* Update version and changelog
* Plumb a clip bounding box around through overzoom
* Actually do some clipping
* Add a test
* Fix post-binning clipping
* Factoring out geometry parsing from feature parsing
* Accept a clip polygon argument to tippecanoe-overzoom
* Progress in the direction of polygon clipping
* Fix the wagyu flags. We need intersection, not union
* Remove debug spew
* Clip points to polygon bounds too
* Copy the geometric binning code to serve as intersection-finding code
* Add clipper2 for linestring clipping
* Compiles, but does not actually seem to clip. Hmm.
* Oh, it helps if I actually call the function
* Add clipping tests
* Add missing fixture, and don't crash if it is missing
* Remember to do polygon clipping after binning too
* Fix scaling before post-binning clipping. Add test.
* Remove unused parts of clipper
* Rename for consistency
* Revert accidentally added line
* Clip the clip regions to the tile bounds to reduce their complexity
* Add a test of clipping the clip region down to the tile boundary
* Update version and changelog
* Strip out unwanted attributes earlier in the process
* Skip aggregations whose attributes have been excluded
* Forgot one
* Add some more tests
* Update version and changelog
* Add a tippecanoe-decode option to restrict which attributes to decode
* Plumb buffer and feature limit around
* Check the feature limit
* Clarifying cases where output detail can be unspecified
* Clip bins to the tile buffer instead of just passing them through
* Add missing include
* Missed some tests
* Add --no-tile-compression option to tippecanoe-overzoom
* Update version and changelog
* Progress on plumbing a string pool for full_keys through
* More plumbing for key_pool
* Don't keep features with identical locations as multiplier features
* Revert "Don't keep features with identical locations as multiplier features"
This reverts commit 413f0c8024.
* Adjust calculated maxzoom to account for duplicate feature locations
* Update changelog and version
* Add a test affected by the maxzoom change with duplicate locations
* Round the drop rate a little for cross-platform test consistency
* Add an all-mvt_value attribute accumulation path
* Only bin by ID, not geometrically
* A little cleanup; changelog and version; test
* Remove accidental double-conversion
* Replace duplicated code with template
* Update changelog
* Choose the megatile features from those that will be in the next N zooms
* Take fractional zooms into account in multiplier feature choices
* Fix more tests
* Add a flag to retain multiplier features by minimum distance
* Limit feature expansion from multiplier density to 2x
* The multiplier cap was a bad idea
* Revert "The multiplier cap was a bad idea"
This reverts commit 6f8273a4c8.
* Revert "Limit feature expansion from multiplier density to 2x"
This reverts commit a26e41309d.
* Revert "Add a flag to retain multiplier features by minimum distance"
This reverts commit 01f14a4255.
* Remove the multiplier sequence, which should no longer matter
* Revert "Revert "Add a flag to retain multiplier features by minimum distance""
This reverts commit 776da4a1b8.
* Revert "Revert "Limit feature expansion from multiplier density to 2x""
This reverts commit 44a683d808.
* Revert "Revert "The multiplier cap was a bad idea""
This reverts commit 80f7cb1c0e.
* Track two kinds of previous index for next_feature
* Fix multiplier density threshold, I think
* Oh, I didn't git add the code changes
* Update version and changelog
* Try to install sqlite3 to fix the automated build
* Deleted too much
* Only let --preserve-point-density-threshold shift density around
* Remove the density debt concept, since it doesn't help
* Make the drop states a vector instead of an array
* Revert "Make the drop states a vector instead of an array"
This reverts commit 66c7abb6fa.
* Revert "Remove the density debt concept, since it doesn't help"
This reverts commit 707bb0c562.
* Revert "Only let --preserve-point-density-threshold shift density around"
This reverts commit ecf01f2231.
* Bin more aggressively if a point doesn't meet the pnpoly test
* Don't clip the points if we are binning
* Fix output of features added to the bin after its closure
* Update tests
* Update version and changelog
* Plumb bounding boxes through potential intersections
* Quick bbox reject for bins that can't possibly intersect
* Inching toward attribute accumulation in megatile handling
* Some sort of test for how all these things interact with each other.
Automatic numeric attribute accumulation does *not* apply to attributes
that have an explicit attribute accumulator set, because the order of
operations is too messy and weird
* More sketching
* More sketching
* Actually do some accumulation
* Put all that behind an --accumulate-numeric flag
* Use the same attribute accumulation logic in binning as in megatiles
* Fix backwards conditional
* Add means, but somehow I have some counts of 0
* Handle aggregated attributes with no base attribute in the feature
* Checkpoint before I break everything
* Found a flaw, now to debug
* Fix a typo that broke accumulation
* Add binning tests
* Make sure IDs make it through on the bins
* Fix count/mean accumulation
* Make the numeric accumulation prefix configurable
* Make sure the accumulate test still works with a different prefix
* Forgot to update this test
* More testing to make sure cluster sizes make it all the way through
* Fix neglected --accumulate-attribute when binning
* Mark unexercised attribute accumulation cases as "can't happen"
* Factor out numeric preservation
* Attrs with the accumulation prefix are just preserved, not accumulated
* Test behavior of prefixed attributes
* Plumbing for exclude and exclude-prefix
* Implement and test attribute prefix stripping in overzoom
* Update version and changelog
* For debugging, make an attribute list of source feature IDs
* Revert "For debugging, make an attribute list of source feature IDs"
This reverts commit 65fc99c9d1.
* Another try at fixing longitude wraparound for bins
* Gonna get it right this time
* Forgot to update the comment
* The filter case was not supposed to reinterpret geometry
* Copy antimeridian-crossing geometries to the other side too
* Getting closer to getting antimeridian-crossing polygons right
* Update changelog and version
* Factoring out tilestats management from GeoJSON file reading
* Move code around so overzoom can link against parse_layers
* Read the file of bins
* Plumb the bins through to overzoom()
* Some zip code bins to test with
* (Currently non-functional) test of binning
* Starting to spell out the bin matching loop
* Can't flatten points, so don't flatten bins either
* More fleshing out bin traversal
* Bounding box of tile-relative mvt geometry
* Smallest enclosing tile from bbox
* Most of the bin scan
* Add point in polygon check. It crashes.
* Find the matching bins
* GDAL-style bounding boxes have eaten my brain
* Make some features to bin into
* Increment a count as features are found to be within the bins
* Fix longitude wraparound in overzoom bins
* Fix the tests
* Push off attribute copying until after bin assignment
* Carry sum of numeric attributes into the bins
* Also add mean, min, and max
* Add --calculate-feature-index since I keep needing it for testing
* Add an option to accumulate sum/mean/max/min/count of all numeric attrs
* Don't bake in tippecanoe:mean, since we redo it from sum and count
* Forgot to update this test fixture after removing tiled mean
* Update version and changelog
* Replace build-essential in Docker builder image
build-essential is overkill, it pulls in all of the tools one needs to
make to build debian packages, which we are not doing. Even though this
is a builder image, it still takes time to download all the extra crud.
Replace build-essential with make gcc and g++, which are all that are
needed.
* Replace dev packages in final Docker image
The dev packages are only needed in the builder image. They add header
files and docs and things that are not needed in the final image.
Removing them and replacing them with just the runtime libraries reduces
the final image size by about 25% (160M->124M) in a local build.
Additionally, just for simplicity, zlib1g is already part of base
ubuntu minimal, so we don't even have to list it.
* Remove build-essential mention from the README
Just like the in Dockerfile, people don't need all of those packages
to build directly on their system.
* Clip before dealing with multiplier or filters in overzoom
* Be more careful to retry when the feature count is exceeded
* Adjust the estimated total feature count for the multiplier too
* Fix the feature count estimates, I think
* Pass build info into the version string
* Report the actual max zoom of any tiles as the metadata maxzoom
* Revert unneeded renaming to make the diff more readable
* Clean up the adjustments to tile sizes and feature counts
* Update version and changelog
* Dropping a feature into a multiplier cluster still effectively drops it
* Update changelog
* Rethink the changelog description
* Don't try to truncate zooms if we are still tiling at z18
* Track output position at the file level instead of within each tile
* Track file position where the child tile data begins
* Add option and document its intended behavior
* Changing the detail loop to account for stopping early
* I forgot I already added an option for this
* Stop early if we can make a complete tile
* Add a test of zoom truncation with limited feature count
* Forgot to commit the actual code change
* Make room for a vertex count in the header of each serialized tile
* Estimate tile complexity; don't try truncating when unlikely to work
* Be more conservative, because ever retrying a tile is a big speed hit
* If stopping early, don't simplify or clean; leave that to overzoom
* Add tiny polygon reduction / dust to overzoom
* Don't try to stop early in the children if we dropped anything by rate
* Fflush here too before pwriting
* Don't stop early if we ended up dropping any features.
Rework the can-the-next-zoom-stop-early logic to avoid going
one zoom further than needed.
* Fix warning
* Fix warnings
* Oops, checking for the wrong expected return value
* Cleanup from adding line simplification in overzoom
* Current (wrong) behavior when combining coalescing and truncating
* Keep a list of parent tiles to skip rather than truncating
* Now the coalesced tiles in z12 get children in z13
* Don't double-count feature dropping when the zoom level is retried
* Correct README description
* Remove todo about special case below basezoom, which is accounted for
* Be a little more aggressive in drop-densest determination
* Scale tile feature limit for megatiles in the same way as byte limit
* Fully deprecate -detect-shared-borders into an alias
* Track the distances found in the douglas-peucker recursion
* Serialize and deserialize the distance with the vertices
* Revert "Serialize and deserialize the distance with the vertices"
This reverts commit 753f1b7909.
* Revert "Track the distances found in the douglas-peucker recursion"
This reverts commit e5361f8c22.
* Revert "Fully deprecate -detect-shared-borders into an alias"
This reverts commit 0698aeb766.
* Better tracking of whether we failed to make a full-detail tile
* Put a bloom filter in front of the binary search for shared nodes
* Forgot to take out this printf
* Improve dispatch of tiling tasks
* Still dispatch the biggest tasks first
* Track zoom truncation in the strategies list in the tileset metadata
* Prescan for small deltas before doing proper simplification
* Revert "Prescan for small deltas before doing proper simplification"
This reverts commit d1d8238b83.
* Update version and changelog
* Rename to --generate-variable-depth-tile-pyramid
* Reviving multi-source-tile overzoom: the clip.cpp side
* Reviving multi-source-tile overzoom: the overzoom.cpp side
* Update readme
* Update version and changelog
* Start to distinguish fixed cluster density setting from as-needed density
* Make consistent {drop,coalesce}-densest decisions between zooms
* Actually track the previous index instead of just intending to
* Clean up collinearities in coalesced features
* To determine densest, look at actual physical distance, not just index
* Don't actually need the previous index in serial_feature now
* Center of mass of one feature to most distant point of the next
* Add apologetic comment
* Wait, how did the tests pass before?
* Revert "Wait, how did the tests pass before?"
This reverts commit f73c8ee543.
* Add --maximum-string-attribute-length option
* Update version and changelog
* A little more testing to make sure
* Trying to reduce memory in tiling
* Let the simplification workers go out of scope earlier
* Bail out quickly once the maximum feature count is reached
* Update changelog and version
* Fix some undefined behavior
* Avoid overflow in line simplification calculations
* Oops, missed a test
* Didn't mean to add that to the Makefile
* Revert "Revert "[ci] test in debug mode (#202)""
This reverts commit c95c328e47.
* Fix reference to out-of-scope pointer
* Fix invalid shift and out-of-bounds vector element reference
* Update changelog and version
* Stabilize feature order in overzoom
* I want my sorts to be stable, please
* Revert "[ci] test in debug mode (#202)"
This reverts commit 853ada87b5.
* No need to reinitialize here
Erica Fischer