Tippecanoe formatted every double it wrote through milo::dtoa_milo, a
vendored Grisu2. Grisu2 is fast, but it guarantees neither the shortest
digit string nor the correctly rounded one: it only guarantees that what
it prints parses back to the value it came from. In practice it prints a
digit more than necessary about 0.16% of the time, and picks a neighbor
of the correctly rounded digits about 32% of the time.
This ports Russ Cox's fpfmt (https://github.com/rsc/fpfmt) to C++ in
fpfmt/ and formats through it instead. fpfmt is both shortest and
correctly rounded, and it is faster:
full std::string formatting Grisu2 fpfmt speedup
random bit patterns 156.62 ns 66.83 ns 2.34x
geo coordinates 124.07 ns 58.62 ns 2.12x
short decimals 69.37 ns 49.16 ns 1.41x
small integers 44.18 ns 38.06 ns 1.16x
digit generation only Grisu2 fpfmt speedup
random bit patterns 90.07 ns 20.81 ns 4.33x
geo coordinates 80.64 ns 20.18 ns 4.00x
short decimals 55.61 ns 21.90 ns 2.54x
small integers 40.23 ns 22.50 ns 1.79x
(Intel Xeon @ 2.80GHz, g++ 13.3 -O3. `make fpfmt-bench` reproduces this,
and `./fpfmt-bench -check` reruns the correctness sweep, which is why
milo/dtoa_milo.h is kept even though nothing links it any more.)
The port is deliberately literal, so it can be diffed against fpfmt.go.
Its Short() agrees bit for bit with the Go original's on 445,640 values
covering powers of ten, small integers and reciprocals, subnormals, and
random bit patterns. Over 38.5 million values, fpfmt::dtoa always round
trips, is never longer than Grisu2's output, and is shorter 61,329 times.
Output is otherwise formatted exactly as before, including the choice
between plain and exponential notation, so 26 expected test outputs
change: some numbers lose digits (-26.170044999999999 becomes
-26.170045), and some have a corrected final digit (9.823748927348929e+55
becomes 9.823748927348928e+55). Every changed token was checked to parse
back to the identical double; none of the values themselves moved.
milo/milo.h, whose only job was to declare the C shim jsonpull calls, is
replaced by fpfmt/fpfmt.h, and the shim is renamed dtoa_shortest.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wJRAuhMninQE4wK2TUfuZ
* Add a tippecanoe-decode option to restrict which attributes to decode
* Plumb buffer and feature limit around
* Check the feature limit
* Clarifying cases where output detail can be unspecified
* Clip bins to the tile buffer instead of just passing them through
* Add missing include
* Missed some tests
* Add --no-tile-compression option to tippecanoe-overzoom
* Update version and changelog
* Prep to track conditions other than just "dropped" or "kept"
* Count up instead of down
* Drop or retain whole multiplier clusters based on their first feature
* Calculate a global feature dropping sequence
* Switch over to using the drop sequence for drop-fraction
* Remove unused arguments for the old drop-fraction implementation
* Fix copy-and-paste bugs, update tests
* Properly incorporate feature_minzoom into the drop sequence, I hope
* Rename drop_by to drop_sequence
* See if sorting within clusters fixes filter stability between zooms
* Remove very chatty debug print
* Update changelog and version
* Use named constants instead of numbers for feature dropping/keeping
* Add comment to explain purpose and method of bit reversal
* Speed up mvt_value comparison
* Converting repetitive ifs to cases
* More conversions from ifs to cases
* Optimize the always-true filter case
* Don't convert types of attributes without accumulators
* Unordered map seems to be faster than map
* Add missing header
* Fix some warnings
* Fix the warnings better
* Avoid an int->string->int conversion
* Lazily initialize layer key and values maps when actually needed
* More switches from maps to unordered_maps
* Sure, I'll take the microoptimization
* More emplacement
* Save some copies
* Emplaces and moves
* Lazy linear scan of attributes instead of building a map
* Extra printfs, missing header
* Avoid clipping if the input and output tiles are the same
* But do clip if the tile extent is being reduced
* Make sure I'm not constructing std::strings here at runtime
* More worrying about runtime string construction
* A couple more std::moves
* Const references!
* More const references
* Another std::move
* Make the string_value of mvt_value std::optional
* Reserve storage when decoding
* Provision for different mvt_values to share a string pool
* Use the string pool when decoding
* Avoid another string construction
* Try limiting the depth of the search for duplicate attributes
* Revert "Try limiting the depth of the search for duplicate attributes"
This reverts commit 9ec94a15ff.
* Update changelog
* Fix typo noticed during code review
* add pmtiles.hpp from github.com/protomaps/PMTiles [#10]
* tippecanoe main writes pmtiles output. [#10]
* detect output format using suffix
* after mbtiles is done writing, replace with pmtiles based on map/image tables.
* add method to write_json for writing json sub-object.
* tippecanoe-decode reads pmtiles input. [#10]
* tile-join reads and writes pmtiles. [#10]
* pmtiles test suite for decode and tile-join [#10]
* add base GitHub CI action for compiling and test suite.
* update pmtiles.hpp with z>15 fix
* Fix some ordering problems with pmtiles decode
* Pmtiles should also pass the raw tiles tests
* Eradicate spaces from tileset metadata JSON fields
* Eradicate spaces from more test fixtures
* Update more tests
* Pmtiles tests pass now too
* Remove unnecessary sort (and make indent)
* Update changelog
* The allow-existing test for pmtiles needs -o, not -e
* Declare --allow-existing to be unsupported for pmtiles.
It was always a bad idea even for mbtiles.
Co-authored-by: Brandon Liu <bdon@bdon.org>
* Improve precision of get_area by using long double
* Trying to get consistent polygon area results between ARM and x86
* Calculate polygon area closer to the origin for better precision
* Update changelog
* Also exercise tiny polygon dust in the ring area test
They previously behaved differently here between x86 and ARM
* On M1 Macs, long double is just double anyway, so don't use it
* Be more careful about overflow: scale the polygon ring down into range
* Fix the bug I just introduced in the scaled area calculation
* Use only the sign from the scaled-down area calculation
Co-authored-by: Roman Karavia <47303530+romankaravia@users.noreply.github.com>
* Stop adding features to a tile if it can't possibly work
* Add --integer and --fraction options to tippecanoe-decode
* Carry the strategies field from tileset metadata through tile-join
* Update changelog
* Assign different codes to different kinds of error exits
The first feature in a tile can never be dropped, since there is
no previous feature to attach its properties to.
Remove the previous special case that reset the dropping counter
at the first feature within each tile proper (as opposed to the
first feature in each tile, including its buffer, which is now
the one that is guaranteed to be preserved).
Claude