main moved each tool's option table to file scope so that usage.cpp can
generate the usage message from it (#409), which conflicted with the
tables the MLT options had been added to. The three MLT options move to
the file-scope tables, under the same headings they were listed with
before: "Setting or disabling tile size limits" in tippecanoe and
tile-join, and "Output tile" in tippecanoe-overzoom.
Also combine the new radix-sort-test with the MLT tests in the test
target, add usage.o to the tools that gained MLT objects, keep both the
MLT=0 and the new docs CI jobs, and regenerate the man page for the
README changes, which the docs job now checks.
* Fix variable-length-array and uninitialized-union compiler warnings
Clang warns about every variable-length array in C++ (-Wvla-cxx-extension,
on by default), since VLAs are a compiler extension rather than standard
C++. Replace all 57 of them with std::vector, or with std::string for the
mkstemp() template buffers built from tmpdir. Add -Wvla to WARNING_FLAGS so
new ones don't creep back in.
Separately, mvt_value's numeric_value union is 16 bytes wide (the size of
string_value), but both constructors only wrote the 8 bytes of the member
they were setting, leaving the rest indeterminate. The implicit copy
constructor copies the union as a whole, so copying any non-string value
read uninitialized bytes, which GCC reports as
mvt.hpp:83:8: warning: 'v.mvt_value::numeric_value. ... .len' may be
used uninitialized [-Wmaybe-uninitialized]
Give string_value, the widest member, a default member initializer so the
union's full width is initialized however it is later used.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D8gsGMjK78TQiCGKTZ2PyR
* Fix remaining float-conversion and format-truncation warnings
Clang's -Wimplicit-const-int-float-conversion flagged two comparisons
against LLONG_MAX, which is not representable as a double and rounds up
to 2^63.
In serial.cpp this was a real latent overflow, not just noise: the guard
`extent <= LLONG_MAX` was really `extent <= 2^63`, so an extent of exactly
2^63 passed it and then hit `(long long) extent`, which is undefined for
that value and yields LLONG_MIN in practice -- the opposite of the clamp
the else branch intends. Make the bound exclusive so the conversion is
always in range. Requires a polygon area at the very top of the double
range to reach, but the clamp now behaves as written.
In mbtiles.cpp the value is only a stand-in for infinity on its way into
JSON, so cast explicitly; the emitted number is unchanged.
Separately, g++ at -O0 warned that `char abbrev[20]` can be truncated by
"%lld", which is correct: the most negative long long needs 21 bytes with
the NUL. That branch is only reached when point_count < 1000, so it cannot
happen today, but size the buffer to fit rather than rely on that, and
replace the garbled comment about how the size was derived.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D8gsGMjK78TQiCGKTZ2PyR
* Clamp the low end of extent before converting to long long too
The upper bound was fixed in the previous commit; the same overflow exists
on the negative side. get_area() returns a signed shoelace area, so inner
rings contribute negatively, and a polygon whose holes outweigh its rings
drives extent below zero. Far enough below and `(long long) extent` is
undefined again.
The bounds are asymmetric, so this is not simply the mirror of the upper
one: LLONG_MIN is exactly -2^63 and converts exactly, so unlike LLONG_MAX
it can be an inclusive bound.
Verified with -fsanitize=float-cast-overflow that the previous form traps
on 2^63 and on doubles just below -2^63, and that this one is clean across
both boundaries, the infinities, and NaN (which falls to LLONG_MAX, as it
did before).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D8gsGMjK78TQiCGKTZ2PyR
* Add CHANGELOG entries for 2.81.0 and bump the version
CHANGELOG.md was last updated for 2.80.0 (#361), and version.hpp has not
moved since. Twelve PRs have landed in the meantime with no entry: #365,
#368, #375, #382, #384, #385, #391, #395, #397, #399, #400, and #401.
Document all of them, plus this PR, under a single 2.81.0 heading. They are
not given separate version numbers because none of them was ever released
under one -- version.hpp read v2.80.0 throughout -- so assigning a version
per PR would invent release history. 2.81.0 is the version that will
actually carry them.
Where an unreleased PR was corrected by a later one (#384 by #385, #397 by
#399), the pair is described as the single behavior that ships, since the
intermediate behavior was never in a release.
Minor rather than patch bump: the batch adds command-line options.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D8gsGMjK78TQiCGKTZ2PyR
* Review feedback: enforce the union-width assumption, describe both clamp ends
The comment on mvt_value's union claimed string_value is the widest member.
That is true on LP64 (16 bytes against 8) but not on ILP32, where size_t is
4 and it ties with double and long long. The default member initializer still
covers the full union either way, so the fix held, but the justification did
not travel. Replace the claim with a static_assert that checks it on whatever
target is being built, so a platform where it stops holding is a compile
error rather than silently indeterminate bytes. Verified the assert is not
vacuous by widening the union in a scratch copy and watching it fail.
The changelog described only the upper end of the extent clamp. Describe both:
the old guard admitted everything below LLONG_MIN too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D8gsGMjK78TQiCGKTZ2PyR
* Add 2.81.0 changelog entries for the four PRs merged from main
#404, #408, #409, and #410 landed while this branch was open. None of them
bumped version.hpp, so they belong under the same 2.81.0 heading as the rest
of the unreleased work rather than getting versions of their own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D8gsGMjK78TQiCGKTZ2PyR
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Don't write an extra byte when the radix sort writes a bucket directly
radix1() writes out a sorted bucket in two places. merge() writes all but
the last byte of each serialized feature and then appends the byte for the
feature minzoom, since the minzoom is the last byte of the feature. The path
taken when a bucket holds only one feature, or when the recursion has
consumed every bit of the index, instead writes the feature's whole
serialized length and then appends another minzoom byte, which is one byte
more than the feature's length prefix says it is. Everything read from the
geometry afterward is then misaligned by a byte.
--prefer-radix-sort lowers the memory limit to 8K so that this code gets
exercised, and any bucket that has to be written directly is enough to
desynchronize the stream, so it fails on several of the existing test
inputs:
$ ./tippecanoe -q -f -o out.mbtiles -z4 -aR tests/ne_110m_ocean/in.json
wrong length decoding feature: used 10, len is 33
Write one byte less here too, as merge() does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wLk2itWETPBAS9a9yE8zu
* Keep the radix sort from recursing forever when it runs out of files
radix1() subdivides a bucket by the next splitbits bits of the index, and
stops recursing once prefix + splitbits reaches the width of the index.
The number of buckets comes from the number of files still available, which
shrinks at every level, so deep enough recursion reaches availfiles / 4 == 1
and therefore splitbits == 0. At that point the recursion consumes no bits
of the index and availfiles stops shrinking, so prefix never advances and
the recursion has no way to terminate.
A splitbits of 0 also makes the shift that chooses a feature's bucket a
shift by the full width of the index, which is undefined. In practice it
leaves the shift count masked to zero, so the bucket number is the whole
index rather than 0, and writing to that bucket runs off the end of the
arrays of open files.
Require at least two buckets so that each subdivision always consumes at
least one bit of the index and the shift is always in range, and don't
recurse at all when the next level would not have enough files to split
with: sort that bucket in memory instead, even though it is larger than
the memory limit asked for, since that is the only way left to get it
sorted.
--prefer-radix-sort, which lowers the memory limit to 8K so that this code
gets exercised, segfaults on tests/feature-filter/in.json without this:
$ ./tippecanoe -q -f -o out.mbtiles -z0 -aR tests/feature-filter/in.json
Segmentation fault
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wLk2itWETPBAS9a9yE8zu
* Check that the radix sort and the in-memory sort agree
The result of a sort shouldn't depend on how the sort was performed, so
rather than checking the sorted output against a committed copy of it,
check that --prefer-radix-sort, which lowers the memory limit to 8K to
force the radix subdivision to recurse, produces the same tiles as sorting
in memory. Nothing new has to be kept up to date, and the comparison holds
regardless of how deeply the subdivision recurses on a given machine, which
depends on how many files it will let us open at once.
What sends the sort down the paths that are otherwise almost never taken is
the shape of the input rather than the size of it, so two small inputs are
generated for the purpose: several well-separated features that are each
too big to sort in memory, which are each written out as a bucket of their
own, and many features at one location, which have to be subdivided until
there are no index bits left. Between them and tests/feature-filter, all
three of radix1()'s branches are covered, including sorting in memory
because there are no files left to subdivide with.
Both of these inputs fail without the two preceding commits, and every
input here failed before them.
The whole target runs in about ten seconds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wLk2itWETPBAS9a9yE8zu
* Record what the shift by the full index width actually did
Say in the comment that masking the shift count to zero makes the bucket
number come out as the whole shifted index, so the writes go somewhere
past the end of the arrays of buckets, rather than only that the shift is
undefined.
Also correct the note on the test: --prefer-radix-sort sets the memory
limit to 8K, but radix() halves it again, so the subdivision is working
against 4K.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011wLk2itWETPBAS9a9yE8zu
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Generate the man page with go-md2man instead of md2man-roff
md2man-roff is distributed only as a Ruby gem -- it is in neither Homebrew
nor apt -- so in practice nobody has it installed and man/tippecanoe.1
drifts away from README.md. It was stale again as of #400: the man page
still had the dead All Streets link that commit fixed.
Switch to go-md2man, the maintained Go port of the same converter (it is
what Docker, podman and runc use). It is packaged as a single static
binary for Homebrew, apt, Fedora and Alpine, and it renders inline code as
bold the same way md2man-roff did, so the man page still reads the way it
used to.
It also emits valid roff, which md2man-roff did not. `mandoc -T lint` goes
from 621 errors and warnings to 1 (an empty .TH date, left empty on
purpose so that generation stays reproducible). 590 of those were
`invalid escape sequence: \fC`, from md2man-roff wrapping every inline
code span in `\fB\fC` -- `\fC` is not a font escape.
md2man-roff was losing content, too:
README: 1/(2^32) of the size of Earth
md2man-roff: 1/(2 of the size of Earth
go-md2man: 1/(2^32) of the size of Earth
README: '{"attr": "operation", "attr2": "operation2"}'
md2man-roff: '{"attr": "operation", "attr2", "operation2"}'
go-md2man: '{"attr": "operation", "attr2": "operation2"}'
Prepend a title block and a NAME section during generation rather than
adding them to README.md, where they would render as noise on GitHub.
The man page had neither, so its header rendered as "tippecanoe()" with no
section, and `man -k tippecanoe` and `whatis tippecanoe` found nothing.
It now renders as TIPPECANOE(1) and is indexed.
Finally, add a CI job that regenerates the man page and fails if the
committed copy differs, so a README edit that needs `make docs` gets
caught rather than sitting stale until someone notices. This is what
makes the missing-tool problem stop mattering: contributors no longer
need go-md2man installed to keep the man page current, since CI will
tell them when it needs regenerating. The go-md2man version is pinned
there because different versions produce different roff for the same
input.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015B6PcMbNY779iaczo6S6Pu
* Strip the redundant blank lines go-md2man puts between paragraphs
go-md2man separates paragraphs with a blank line as well as a .PP macro.
A blank line is itself a break in roff, so the two together double-space
the page: every paragraph was followed by two blank lines rather than one.
md2man-roff did not do this, so it showed up as a regression -- the source
went from 13 blank lines to 200.
Filter them out after generation. Blank lines inside .EX and .TS blocks
are kept, since there they are part of the example or the table rather
than spacing around it; that is all 13 of the ones md2man-roff emitted.
The rendered page loses 186 blank lines and the source loses 187, with
byte-identical non-blank output under both groff -t -man and mandoc.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015B6PcMbNY779iaczo6S6Pu
* Decouple the man page from version.hpp, and name the first section
Review feedback on #408.
Making man/tippecanoe.1 depend on version.hpp turned the docs job into a
hard CI failure on any release commit that bumps the version without
regenerating -- #406, which is open and moves version.hpp to v2.81.0
without touching the man page, would have tripped it as soon as either
merged. The only thing the dependency bought was the version in the page
footer, so every release would have had to regenerate the whole file to
rewrite that one line, gated by CI. Drop it: the source field is now just
"tippecanoe", and the page depends on README.md alone.
Separately, README.md's own title heading became the second .SH, directly
below the NAME section this branch adds, so the page opened with a stray
"tippecanoe" section. Rename it to DESCRIPTION, which is where that text
belongs and what a reader expects after NAME.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015B6PcMbNY779iaczo6S6Pu
---------
Co-authored-by: Claude <noreply@anthropic.com>
* docs: correct README statements that don't match the code
Cross-checked README.md against the option tables in main.cpp,
tile-join.cpp, decode.cpp, jsontool.cpp and overzoom.cpp, plus
options.hpp for the -pX/-aX letter assignments.
Incorrect:
* -aD and -aS were swapped. options.hpp assigns 'D' to
A_COALESCE_FRACTION_AS_NEEDED and 'S' to
A_COALESCE_DENSEST_AS_NEEDED, the opposite of what was documented.
* --limit-base-zoom-to-maximum-zoom was given as -Pb. It is a
prevent flag (P_BASEZOOM_ABOVE_MAXZOOM = 'b'), so it is -pb; -P
is --read-parallel and takes no letters.
* --retain-points-multiplier referred to --tile-size-limit, which
is not an option. The limit it extends is --maximum-tile-bytes.
* The dot-dropping description said tippecanoe "drops 1/2.5 of the
dots for each zoom level above the point base zoom". It keeps
1/2.5 of them, at zooms below the base zoom (prep_drop_states
sets interval only where i < basezoom).
* The default tileset name was given as "file.json". make_metadata
sets both name and description from the output file or directory
name.
* tile-join -r/--read-from was described as a "list of input
mbtiles"; it names a file to read that list from, one per line.
* tippecanoe-decode's -I and -F were given as --integer and
--fraction. Those work only as getopt abbreviations; the real
names are --integer-coordinates and --fractional-coordinates.
* Development notes said C++11 and suggested g++-5. The Makefile
builds with -std=c++17.
* Malformed references: "-quiet" and "no-simplification-of-shared-nodes".
Undocumented options now covered:
* tippecanoe: -aa/--keep-point-cluster-position,
--preserve-multiplier-density-threshold, -H/--help, the count
operation for --accumulate-attribute, and the
point_count_abbreviated cluster attribute.
* tile-join: -O as the short form of --overzoom, -q/--quiet,
--exclude-all-tile-attributes, --exclude-all-tile-geometries.
* tippecanoe-decode: -y/--include, -x/--exclude-metadata-row.
* tippecanoe-overzoom: -x/--exclude, --exclude-prefix, -J,
-S/--line-simplification, --tiny-polygon-size,
--deduplicate-by-id, --no-tile-compression, -t/--source-tile,
-o/--output, and the long names for -b, -d, -y, -j, -m and -E.
Also noted that CSV latitude/longitude columns are matched
case-insensitively as substrings, added file.csv to the usage
synopsis, and explained the -a/-p letter-bundle syntax that the
short forms throughout the document rely on.
Every newly documented flag was run against a built binary. The
man page is regenerated from README.md per the Makefile rule; that
also picks up the All Streets link fix from #400, which had not
been regenerated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MvSCpD1yQZhRT5iMU9yufQ
* Hide --unidecode-data from the generated usage messages
The option has done nothing since 533e000 removed the only caller of
unidecode_smash(), so listing it advertises behavior the tools don't
have. Move it after the empty-name entry that ends the usage listing,
the same place --no-polygon-splitting and the debug options sit, so it
is still accepted but no longer offered.
This is the situation #409 already fixed for tile-join's
--use-attribute-for-id, but it applies to all three tools that take
--unidecode-data, not just tile-join: main.cpp listed it under
"Filtering features by attributes" and overzoom.cpp under "Modifying
feature attributes", both ahead of the terminator.
tile-join's "Modifying feature attributes" heading covered only this
option, so it goes too rather than being left empty. overzoom.cpp had
no hidden group at all, so one is added. In main.cpp and overzoom.cpp
the heading keeps its other options and stays.
strip_usage_headings() copies every entry with a non-zero val, so the
moved option still reaches getopt_long(); confirmed by running each
tool with --unidecode-data and checking it is absent from --help.
make test passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MvSCpD1yQZhRT5iMU9yufQ
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Generate the usage message of each tool from its long_options
The usage messages of tile-join, tippecanoe-overzoom,
tippecanoe-json-tool, tippecanoe-decode, and tippecanoe-enumerate were
hand-written lists of options that had drifted years out of date, since
nothing tied them to the options that are really accepted. Move the
option-list printing that tippecanoe already does into a shared
print_usage(), and use it in all the tools, so that the message is
derived from the same long_options table that getopt_long() gets and
can't fall behind it again.
The tables now carry section headings, as tippecanoe's does, and the
options that were only reachable by their short names (tile-join's -O,
-b, -R, and -r among them) are listed for the first time.
Also state the non-option arguments the way each tool really treats
them: tile-join takes source tilesets unless --read-from names a file to
read them from, tippecanoe-decode takes a tileset either alone or with a
zoom/x/y, tippecanoe-json-tool reads standard input when no files are
named, and tippecanoe-overzoom's two forms are the ones its argument
parsing recognizes. tippecanoe-overzoom now reports the missing -o
instead of passing NULL to fopen(), and tippecanoe-enumerate goes
through getopt_long() so that it will pick up any options added later.
The shared getopt_string() replaces the identical loop that four of the
tools each had for building the short option string, and strip_usage_headings()
the one for dropping the headings before getopt_long() sees them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016frkRY1xXtiWjxYuCJ8vZY
* Print the usage message when tippecanoe is run with no arguments
Running `tippecanoe` with nothing at all reported the missing output
file, which is true but is not what someone who typed the bare command
needs to know. Check for the empty command line before parsing and print
the general usage message instead, and leave the specific complaint for
the case where an input file was named but an output file wasn't.
To make the message reachable from there, the options table and the
usage printing move out of main() into a usage() function, as in the
other tools.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016frkRY1xXtiWjxYuCJ8vZY
* Address review: alternation, the dead tile-join option, and --version
Four fixes from review of the generated usage messages:
* `--output` and `--output-to-directory` are one-of, not one required and
one optional, in both tippecanoe and tile-join. A `usage_required_option`
can now name an alternation that it belongs to, and the options in one
are listed together as `(--output=... | --output-to-directory=...)`,
which is what the runtime check enforces.
* tile-join's `--use-attribute-for-id` has had no implementation since
533e000 removed it; only the table entry was left behind, so the option
parsed and then exited with "Unrecognized option". Generating the usage
message from the table turned that into a documented option that doesn't
work, so remove the leftover entry too.
* `--version` was grouped under "Progress indicator", in the options table
and in the README both. Give it a heading of its own now that the
headings are something users see.
* print_usage() left `width` holding the length of the last synopsis line,
and only got away with it because every table so far begins with a
heading, which resets it. Start the option list on a line of its own
instead of depending on that.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016frkRY1xXtiWjxYuCJ8vZY
---------
Co-authored-by: Claude <noreply@anthropic.com>
The MLT build fails on aarch64: FastPFOR's cmake passes -msse4.2 to the
compiler in every SIMD mode, including the portable one that MLT selects,
so the build dies with "unrecognized command-line option '-msse4.2'" on
ubuntu-24.04-arm, and on Apple Silicon for the same reason. Nothing about
the MLT integration is architecture-specific, so this is as true of the
output format as of the decoder; it went unnoticed because CI never ran
on the branch that added it.
Upstream fixed it in maplibre/maplibre-tile-spec#1492, by vendoring a
FastPFOR snapshot with the SIMD codecs removed -- which is no loss, since
the SIMD encoder's output isn't compatible with the non-SIMD decoder and
was never used. Move the submodule forward four commits to pick that up
along with the platform fixes in #1497.
FastPFOR is no longer a cmake option there, so drop the two -D flags for
it from the configure step and follow the library's rename from FastPFOR
to fastpfor-lib. FastPFOR decoding is still compiled in, so no tile
becomes unreadable, and the checked-in standards are unchanged. The
submodule also no longer needs its own fastpfor and simde submodules, so
the recursive checkout gets a little smaller.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxhELiLtpUFwrPSWwTYdHG
MapLibre Tile support pulls in the maplibre-tile-spec submodule and needs
cmake to build it, which is a lot to ask of anyone who only wants to work
with Mapbox Vector Tiles.
`make MLT=0` compiles with -DNO_MLT, skips the submodule and its cmake
build entirely, and drops the MLT tests from `make test`. The result needs
no dependencies beyond the ones MVT already needed, and can be built from
a checkout with no submodules at all.
Everything that touches the MLT library is behind the #ifdef, which is
just the two files that were written for it. The option parsing and the
tile format helpers stay compiled either way, so nothing else needs to
know: --output-format=mlt reports that the build has no MLT support rather
than being an unrecognized value, and a tile that is recognized as MLT
reports the same instead of being misparsed as a protobuf, since the
format sniffing itself doesn't need the library.
CI builds and tests this configuration from a checkout without submodules,
so it can't quietly stop working.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxhELiLtpUFwrPSWwTYdHG
Both tools could read MapLibre Tiles but only ever wrote Mapbox Vector
Tiles, so there was no way to convert a tileset into MLT, or to keep a
tileset in MLT once it had been through either of them.
Give them the same --output-format, --pretessellate, and
--no-mlt-feature-sort options that tippecanoe has. tile-join writes the
chosen format to mbtiles files, PMTiles archives, and tile directories,
naming directory tiles and the tileset metadata format accordingly, and
tippecanoe-overzoom writes it to its output tile. Either tool will read
whichever format its sources are in regardless of what it is writing.
The output format selection and the MLT encoder options now live in
mlt.cpp, shared by all three tools rather than defined in main.cpp for
tippecanoe alone, along with encode_tile() for encoding a tile in the
selected format. overzoom() takes the format as a parameter, since
tile-join uses it internally to rescale tiles that will be re-encoded
afterward, and those intermediate tiles should stay MVT.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxhELiLtpUFwrPSWwTYdHG
Tilesets written with --output-format=mlt could not be read back by any of
tippecanoe's own tools, which made MLT a dead end rather than a tile format.
Build the mlt-cpp decoder from the vendored maplibre-tile-spec submodule
alongside the encoder, and convert a decoded MapLibre Tile back into the
equivalent mvt_tile: geometry, feature ids, and property columns. MLT rings
come back explicitly closed, so the repeated final point becomes an MVT
closepath, and the property columns, which are held in an unordered map,
are sorted by name so that the attributes of a decoded tile come out in a
stable order.
Rather than adding an option to each tool, mvt_tile::decode() detects the
encoding and dispatches, so everywhere tippecanoe already reads a vector
tile can read MLT. An MLT tile begins with a varint layer length followed
by a varint layer tag whose only defined value is 1, while an MVT tile is
a protobuf whose only field is the repeated layer field 3, so it begins
with 0x1a followed by a layer length that can never be as short as the one
byte that would be needed to look like an MLT layer tag.
Tile directories also needed a fix: enumerate_dirtiles() recognized .mlt
file names but still recorded .pbf as the extension to read them back
with, so decoding an MLT directory failed to find any of its tiles.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxhELiLtpUFwrPSWwTYdHG
An MLT property column has one type for the whole layer, while MVT values
are individually typed. Converting each value on its own left the encoder
to reconcile the mismatch, and its fallback for a column holding both
integers and doubles is to encode the whole column as strings, so an
attribute like gdp_md_est in tests/ne_110m_admin_0_countries turned into
values like "904.200000".
Summarize each attribute across the layer first and pick one type that can
hold all of its values, so mixed integer and floating point columns become
doubles. Only columns that mix numbers with strings, or booleans with
numbers, still fall back to strings, which is as close as MLT's typed
columns can get.
Also stop encoding JSON-object-valued attributes as MLT struct columns.
Struct children can only be strings, and a struct column is flattened into
"column name + child name" when it is read back, so an attribute `meta`
holding {"en": "one", "de": "eins"} decoded as separate `metaen` and
`metade` attributes. Nested JSON now stays JSON text, the way MVT
carries it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LxhELiLtpUFwrPSWwTYdHG
* wagyu: drop a hole no remaining ring can parent instead of throwing
correct_tree() throws "Could not properly place hole to a parent" when
topology correction leaves a hole whose parent ring was removed (degenerate
input such as stacked duplicate rings from coalesced tiny-polygon
placeholders). That aborts the entire tiling run over one unrepresentable
sliver. Remove the ring and its points instead, matching how other
unresolvable degeneracies are handled.
* Add a regression test for dropping an unplaceable hole
A fuzzer-minimized pair of mutually reversed self-intersecting rings that
makes wagyu's correct_tree fail to find a parent for a hole — the same
failure reported in mapbox/tippecanoe#761. Before the topology_correction
change, running this test exits with EXIT_IMPOSSIBLE via the polygon
cleaning error handler; with it, the clean returns.
Don't prune variable-depth children while a minzoom-gated feature is pending
The minzoom_feature_pending flag from #397 keeps a variable-depth pyramid
subdividing until explicit per-feature minzooms are satisfied, but two gaps
let features still be dropped:
The flag was only set when tippecanoe_minzoom > z + 1, so a feature whose
minzoom is exactly z + 1 never marked the tile pending, even though a leaf
at z carries only z-visible content.
The early-stop commit never consulted the flag: a tile that succeeded in
stopping early inserted itself into skip_children_out unconditionally,
pruning the children the pending feature needed. The flag only inflated
estimated_complexity_out, which the pruning ignores.
Set the flag for any feature excluded below its minzoom, include it in the
early-stop veto, and skip child pruning while it is set. Adds a fixture
covering the minzoom == z + 1 boundary; make test passes with no diffs to
existing fixtures.
--generate-variable-depth-tile-pyramid decides a tile is a leaf once its
geometry fits at full detail, then prunes the tile's entire subtree. The
guard that prevents leafing while deeper content is still pending consults
only feature_minzoom (the automatic dot-dropping zoom); it does not consult
tippecanoe_minzoom, the explicit per-feature minzoom set via the
tippecanoe.minzoom attribute.
So a feature carrying an explicit minzoom deeper than where its region leafs
is excluded from the leaf tile (z < minzoom) while its children are never
generated. It ends up in no tile at any zoom, silently dropped.
next_feature() excludes such a feature and continues without returning it,
so the leaf-prevention guard in write_tile() never sees it. Carry a flag out
of next_feature() when an excluded feature first appears beyond the next
zoom, and feed it into the same estimated-complexity path feature_minzoom
already uses, so the pyramid keeps subdividing down to the feature's minzoom.
The flag only affects estimated_complexity_out, which is written solely under
--generate-variable-depth-tile-pyramid, so builds without that flag are
unchanged. make test passes with no fixture diffs.
Integrate the C++ MLT encoder from maplibre-tile-spec as a submodule.
Tiles are encoded by converting the existing mvt_tile to
mlt::Encoder::Layer and calling the encoder library directly.
New CLI flags:
--output-format=mlt Encode tiles as MLT instead of MVT
--pretessellate Pre-triangulate polygons (MLT only)
--no-mlt-feature-sort Disable within-tile Hilbert sorting (MLT only)
MLT tiles use .mlt extension in directory output and format=mlt in
mbtiles metadata. Compression and all existing flags work unchanged.
Enhance fqsort function to accept a temporary directory parameter for file handling. Update calls to fqsort in main.cpp, sort.cpp, sort.hpp, and unit.cpp to utilize the new parameter, ensuring temporary files are created in the specified directory.
* Divide-and-conquer polygon cleaning
* Catch the case where the gap can't be increased further
* Catch the case where we try to keep impossibly many features
* Make label points earlier in the tiling process
* Another case where it could try to drop even after already limiting.
* And do not coalesce on impossibly small geometries
* Add missing return
* Update version and changelog
* Improve the memory spike during tile construction
* Remove `need_tilestats`. Add a bunch of debug logging
* Remove debug logging
* Update version and changelog
* Add a flag to use an H3 index for the feature index
* Give mvt_value and serial_val a double-with-count concept
* Switch mean over to internal accumulation state
* Get rid of the attribute accumulation map
* Change vectors of features to vectors of pointers to features
* Fix --coalesce
* Revert "Add a flag to use an H3 index for the feature index"
This reverts commit b9b48f42c9.
* Update version and changelog
* Sketching out sqlite options for tile-join
* Enable sqlite3 serialized multithreading
* Fix some unnecessary round trips from std::string to char * and back
* Gathering join keys for sql query
* Make a query
* Actually open the gpkg. Fix the query quoting.
* Actually do the query and get results back
* Join the attributes onto the feature
* Set matched if the sql join matches
* Add a flag to get the feature ID from the query
* Observe attribute exclusion when joining from sql queries
* Add a flag to exclude all attributes from the tile side of the join
* Make tile-join bounding boxes reflect feature bounds, not tile bounds
* More tests
* An empty tileset has empty bounds at null island
* Gather the results from each thread *after* the thread finishes
* Missed a test
* Fix accidental inclusion of the top left of the tile in the bbox
* Case smashing and prefix trimming in the select
* Add test of sql join
* Make the join column option a join expression option
* Fix antimeridian adjustment. Z0 can't wait until the end of the tile
* Checkpoint on accepting multiple joined rows per tiled feature
* Adding the joined attribute should be per-feature, not per-attribute
* Forgot to update the test. Order of joined attributes has changed.
* Get the attributes back in the right order
* Add test of sql join with limit
* Allow multiple tile features to have the same join key
* Update tests for a country name with two distinct geometries
* Update version and changelog
* Forgot to mention the bounding box improvements
* Make tippecanoe-overzoom accept filters from a file
* Accept clip polygons from a file too
* Add test of clipping by polygon from file
* Add a test of reading an overzoom filter from a file
* Update version and changelog
* Plumb a clip bounding box around through overzoom
* Actually do some clipping
* Add a test
* Fix post-binning clipping
* Factoring out geometry parsing from feature parsing
* Accept a clip polygon argument to tippecanoe-overzoom
* Progress in the direction of polygon clipping
* Fix the wagyu flags. We need intersection, not union
* Remove debug spew
* Clip points to polygon bounds too
* Copy the geometric binning code to serve as intersection-finding code
* Add clipper2 for linestring clipping
* Compiles, but does not actually seem to clip. Hmm.
* Oh, it helps if I actually call the function
* Add clipping tests
* Add missing fixture, and don't crash if it is missing
* Remember to do polygon clipping after binning too
* Fix scaling before post-binning clipping. Add test.
* Remove unused parts of clipper
* Rename for consistency
* Revert accidentally added line
* Clip the clip regions to the tile bounds to reduce their complexity
* Add a test of clipping the clip region down to the tile boundary
* Update version and changelog
Claude