* Remove the concept of "separate metadata"
This was an extra level of attribute indirection (features point
to metadata records which point to key and value strings) which was
intended to reduce the size of temporary storage for features with
large numbers of attributes that were also spread across large numbers
of tiles at maxzoom.
For other kinds of features, the extra indirection slowed things down
instead, and, especially when maxzoom guessing was being used, many more
features were having their metadata externalized than could actually
benefit from it.
* Shave a few bytes off temporary files by using more unsigned integers
* Flush stderr after logging progress
* Revert "Shave a few bytes off temporary files by using more unsigned integers"
This reverts commit eef29084ec.
* Limit the size of the string pools and trees to fit in memory
* Add missing #include
* Move the string pool and search tree from mmap to allocated memory
* Sort in allocated rather than mapped memory too
* Also use pread instead of mapping to read in the data to sort
* When the pool gets too big, switch to just the file, not memory
* Switch string pool from memory to disk when memory is 10% full
* Add to-memory versions of the serialization functions
* Crashy work in progress toward compression
* Fix the pointer bug that was causing the crash
* Serialize features into memory rather than straight to disk
* Compress individual features in the temporary files
* Don't need to store the length of the geometry
* Remove per-feature compression; move minzoom back into the object
* Start adding a stream compressor object
* Track file position within fwrite_check()
* Add compressed stream writer functions
* Pull the writing of the serialized feature out to the callers
* Starting toward compression again from a different point
* Hook up more compression functions
* Remove unused code from the other day
* Make enough deflate calls to flush out all the buffered data
* Start on decompression
* Tile number is uncompressed, tile content is compressed
* Work on alternating compressed and uncompressed in decompression
* Closer, but still doesn't work
* Sort of works
* Works until we get to concatenated tiles
* More attempts that don't work
* One bug down
* It made a tileset!
* Handle nonzero initial zooms
* Fix seeking within compressed feature streams
* Tests pass!
* Remove debug spew
* Oops: remember to delete the temporary files so they don't hang around
* Test that fails with the current compression code
* Properly account for bytes read while closing the compressed stream
* Limit the number of warnings about bad label points
* A little more armor when closing decompression
* This time for sure
* A different, less fragile, test that failed previously with compression
* Move feature stream compression to its own file
* Remove now-unused code to deserialize from a file
* Forgot to add the new files
* Remove a little debugging logging
* Add a couple of comments on what it means to be within decompression
* Fix indentation
* Update changelog. Remove stray debugging comment.
* Avoid placing polygon labels in holes
* Traverse vertices in hilbert order to find potential label points
* Update test to reflect new label placements
* Sorting by Y coordinate is better and easier than Hilbert order
* Give a bonus for being near the center of mass
* Make label points a little more border-shy
* Tall places, not just wide places; border lines, not just border points
* Clean up
* Try diagonals through the features too
* Limit the search for label points to prevent slowing down too much
* At this point the center of mass bonus is doing more harm than good
* Forgot to update test
* Round coordinates instead of truncating them
* Update all the tests for coordinate rounding changes
* Curses, integer division still truncates
* Fix tests
* Also round instead of shifting when looking for no-op linetos
* Should I worry that the same change for moveto doesn't change any tests?
* Also round instead of shifting when scaling down to maxzoom resolution
* Replace another explicit shift, for origin point
* Round instead of shift when writing clipped geometries to the next zoom
* Fix low-zoom gridding and smaller-than-a-pixel checks
* Don't guess an excessively large maxzoom when there is only one feature
* Add a test for guessing the maxzoom of a single point
* Explicitly sort by index if no other order distinguishes features
* Another affected test
* add pmtiles.hpp from github.com/protomaps/PMTiles [#10]
* tippecanoe main writes pmtiles output. [#10]
* detect output format using suffix
* after mbtiles is done writing, replace with pmtiles based on map/image tables.
* add method to write_json for writing json sub-object.
* tippecanoe-decode reads pmtiles input. [#10]
* tile-join reads and writes pmtiles. [#10]
* pmtiles test suite for decode and tile-join [#10]
* add base GitHub CI action for compiling and test suite.
* update pmtiles.hpp with z>15 fix
* Fix some ordering problems with pmtiles decode
* Pmtiles should also pass the raw tiles tests
* Eradicate spaces from tileset metadata JSON fields
* Eradicate spaces from more test fixtures
* Update more tests
* Pmtiles tests pass now too
* Remove unnecessary sort (and make indent)
* Update changelog
* The allow-existing test for pmtiles needs -o, not -e
* Declare --allow-existing to be unsupported for pmtiles.
It was always a bad idea even for mbtiles.
Co-authored-by: Brandon Liu <bdon@bdon.org>
* Add an option to limit geometry vertex count
* Update docs and changelog
* Track desired feature count and geometry size in strategies
* Remove dead code for a long-forgotten inaccessible option
* Fix the option name in changelog
* Refine coalesce-smallest to only coalesce onto other small features
* Clean polygons before coalescing-as-needed
* Don't accumulate tiny polygon holes as negative dust
* Add the option not to limit the feature count at maxzoom
* Add options to limit feature count more abruptly in each tile
* Revert "Add the option not to limit the feature count at maxzoom"
This reverts commit ace173ab7a.
* Revert "Fix the option name in changelog"
This reverts commit 38dbce8405.
* Revert "Add an option to limit geometry vertex count"
This reverts commit d379fdf06b.
* Remove test for reverted option
* Remove more leftovers from geometry size limiting
* Update changelog
* Add an option to drop the densest features by rate, not a fraction
* Restrict dot dropping just to dots, as it should be
* 50% mix of sparse and uniform point preservation
* Update version, changelog, test, and documentation
* 60% sparse, 40% dense
* Didn't update the test after the last change
* Make the drop-denser rate configurable without recompiling
* Generate labels points after simplification, not before.
Previously there were some cases where dropping the smallest
polygons would never reduce the number of labels.
* Also wait until after polygon cleaning to make labels
* Remove the extra newlines after the TILES ONLY COMPLETE message
* Add an option to generate label points in place of polygons
* Change all these places where I said "extent" but really meant "area"
* Revert "Change all these places where I said "extent" but really meant "area""
This reverts commit 403828d2f7.
* Add --order-smallest-first and --order-largest-first options
* Use Turf's center-of-mass algorithm for polygon label points
* If the label point isn't within the polygon, find one that is
* Don't choose a label point that is too close to a border
* Try a little harder to find an optimal label point
* Checkerboard which tiles labels are generated in, to reduce adjacency
* Use a label point for the general representative point for polygons
(Skipping the iteration to find one that is as far as possible from
the borders)
This makes the labels look better in many cases (like France at z1)
but unfortunately ripples into changing the sequence of polygons in
many tests, so the diff is big.
* Revert "Use a label point for the general representative point for polygons"
This reverts commit 2261adf05e.
* Checkpoint work on spiral labels
* Clip label spirals to the feature bounds
* Fix label test
* Be careful not to place spiral labels too close to borders either
* For spiral anchors, only check tile scale, not feature size
* Update test
* Only try to find a central label point for the largest ring
* In tiny polygon dust, keep the attributes of the largest feature
* Improve precision of get_area by using long double
* Trying to get consistent polygon area results between ARM and x86
* Calculate polygon area closer to the origin for better precision
* Update changelog
* Also exercise tiny polygon dust in the ring area test
They previously behaved differently here between x86 and ARM
* On M1 Macs, long double is just double anyway, so don't use it
* Be more careful about overflow: scale the polygon ring down into range
* Fix the bug I just introduced in the scaled area calculation
* Use only the sign from the scaled-down area calculation
Co-authored-by: Roman Karavia <47303530+romankaravia@users.noreply.github.com>
* First attempt at porting Paul Mach's Visvalingam implementation
* Make indent
* Mostly working
* Approximate equivalence from Douglas-Peucker to Visvalingam levels
* Don't simplify away tile boundary crossing points
* Add a command-line option and test for Visvalingam simplification
* Update changelog
* Cleanup in response to review feedback
* Include <stdio.h> to fix compiler warnings
* Stop adding features to a tile if it can't possibly work
* Add --integer and --fraction options to tippecanoe-decode
* Carry the strategies field from tileset metadata through tile-join
* Update changelog
* Assign different codes to different kinds of error exits
* Add an option to retain extra coordinate precision at maxzoom
* Make sure not to shift away the extra detail from coordinates
* Add an option to convert double-precision attributes to single
* Sort attribute values in tiles to make them compress a little better
* Slightly improve polygon simplification
By choosing a point that would be retained after simplification
to be the start/end point that always gets retained
* I regret making all of these tests involve polygons
* Add an option to specify the size of tiny polygons
* Fix accidental requiring of argument for --single-precision
* Guard against duplicate points when generating "sizes" for them
* Restore the intended behavior that tiny polygons don't get simplified
* Make the extra detail settable rather than always maximizing it
* Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)"
This reverts commit fec5e8354c.
* Add an option to prevent choosing a base zoom higher than the maxzoom
* Keep the drop rate high enough when the basezoom gets constrained
* Revert "Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)""
This reverts commit db6bc27d9e.
* Add --order-by and --order-descending options
* Accept multiple --order-by and --order-descending-by sort keys
An instance of catching an exception by value:
https://blog.knatten.org/2010/04/02/always-catch-exceptions-by-reference/
In jsontool, there's a warning about writing 8 bytes into a 7
byte buffer, potentially truncating or losing the nul-terminator.
There are a couple of instances of an unused bool variable, not
really a big deal.
Erica Fischer