Commit Graph
60 Commits
Author SHA1 Message Date
Erica Fischer 355cc31d5b Indirect lists of mvt_feature through pointers for faster sorting 2025-02-12 11:32:28 -08:00
Erica Fischer 11e3196c9a Raise tippecanoe-decode tile size limit to 250 MB (#299)
* Raise tippecanoe-decode tile size limit to 250 MB

* Update version and changelog
2024-11-21 09:17:22 -08:00
Erica Fischer 794ae2b1eb Tippecanoe-decode and tippecanoe-overzoom changes to improve binning performance (#292)
* Add a tippecanoe-decode option to restrict which attributes to decode

* Plumb buffer and feature limit around

* Check the feature limit

* Clarifying cases where output detail can be unspecified

* Clip bins to the tile buffer instead of just passing them through

* Add missing include

* Missed some tests

* Add --no-tile-compression option to tippecanoe-overzoom

* Update version and changelog
2024-11-08 15:46:04 -08:00
Erica Fischer 7d4d264d50 Speeding up tippecanoe-overzoom (#191)
* Speed up mvt_value comparison

* Converting repetitive ifs to cases

* More conversions from ifs to cases

* Optimize the always-true filter case

* Don't convert types of attributes without accumulators

* Unordered map seems to be faster than map

* Add missing header

* Fix some warnings

* Fix the warnings better

* Avoid an int->string->int conversion

* Lazily initialize layer key and values maps when actually needed

* More switches from maps to unordered_maps

* Sure, I'll take the microoptimization

* More emplacement

* Save some copies

* Emplaces and moves

* Lazy linear scan of attributes instead of building a map

* Extra printfs, missing header

* Avoid clipping if the input and output tiles are the same

* But do clip if the tile extent is being reduced

* Make sure I'm not constructing std::strings here at runtime

* More worrying about runtime string construction

* A couple more std::moves

* Const references!

* More const references

* Another std::move

* Make the string_value of mvt_value std::optional

* Reserve storage when decoding

* Provision for different mvt_values to share a string pool

* Use the string pool when decoding

* Avoid another string construction

* Try limiting the depth of the search for duplicate attributes

* Revert "Try limiting the depth of the search for duplicate attributes"

This reverts commit 9ec94a15ff.

* Update changelog

* Fix typo noticed during code review
2024-01-29 11:33:56 -08:00
Erica Fischer b5ec3c46f0 Reduce excessive progress logging during pmtiles conversion (#111)
* Don't log progress so often during pmtiles conversion

And turn off the `catch` tests, which have bit-rotted

* Fix accidental double-multiplication-by-100

* Reenable catch

* Update changelog
2023-07-10 15:59:52 -07:00
Erica FischerandBrandon Liu 3b2599f587 Add support for pmtiles output format (#55)
* add pmtiles.hpp from github.com/protomaps/PMTiles [#10]

* tippecanoe main writes pmtiles output. [#10]

* detect output format using suffix
* after mbtiles is done writing, replace with pmtiles based on map/image tables.
* add method to write_json for writing json sub-object.

* tippecanoe-decode reads pmtiles input. [#10]

* tile-join reads and writes pmtiles. [#10]

* pmtiles test suite for decode and tile-join [#10]

* add base GitHub CI action for compiling and test suite.

* update pmtiles.hpp with z>15 fix

* Fix some ordering problems with pmtiles decode

* Pmtiles should also pass the raw tiles tests

* Eradicate spaces from tileset metadata JSON fields

* Eradicate spaces from more test fixtures

* Update more tests

* Pmtiles tests pass now too

* Remove unnecessary sort (and make indent)

* Update changelog

* The allow-existing test for pmtiles needs -o, not -e

* Declare --allow-existing to be unsupported for pmtiles.

It was always a bad idea even for mbtiles.

Co-authored-by: Brandon Liu <bdon@bdon.org>
2022-12-29 12:00:13 -08:00
Erica Fischer 9c77029b63 Add a data structure for tileset metadata (#44)
* Progress toward making a tileset metadata structure

* Write metadata from structure to mbtiles

* Write dirtiles metadata.json from metadata structure

* Update changelog
2022-12-12 13:32:02 -08:00
Erica FischerandRoman Karavia 182093bdc7 Improve precision of polygon area calculations (#19)
* Improve precision of get_area by using long double

* Trying to get consistent polygon area results between ARM and x86

* Calculate polygon area closer to the origin for better precision

* Update changelog

* Also exercise tiny polygon dust in the ring area test

They previously behaved differently here between x86 and ARM

* On M1 Macs, long double is just double anyway, so don't use it

* Be more careful about overflow: scale the polygon ring down into range

* Fix the bug I just introduced in the scaled area calculation

* Use only the sign from the scaled-down area calculation

Co-authored-by: Roman Karavia <47303530+romankaravia@users.noreply.github.com>
2022-10-04 10:04:34 -07:00
Erica Fischer af1a7ed7ae Once features can't possibly fit in a tile, stop trying (#9)
* Stop adding features to a tile if it can't possibly work

* Add --integer and --fraction options to tippecanoe-decode

* Carry the strategies field from tileset metadata through tile-join

* Update changelog

* Assign different codes to different kinds of error exits
2022-09-23 20:01:13 -04:00
Eric Fischer 25bf3957d1 Guard against null data in the mbtiles file when decoding 2019-10-31 13:50:29 -07:00
Eric Fischer 25b9c51ac8 Fix -Z and -z for tile directories in tile-join and tippecanoe-decode 2019-03-18 18:01:32 -07:00
Eric Fischer 79363e397b Factor out the GeoJSON parsing loop from feature serialization 2018-08-08 13:11:10 -07:00
Dane Springmeyer 6683939483 catch all exceptions 2018-07-13 18:27:24 -07:00
Eric Fischer 11c6c869aa Include a creator field in tileset metadata with the Tippecanoe version
Also finally eradicates binary diffs from the tests
2018-06-07 11:20:12 -07:00
Eric Fischer 7f843c4d36 Check integrity of sqlite3 file before decoding or tile-joining 2018-03-29 12:02:59 -07:00
Eric Fischer 1c4aa57ffd Change the JSON writing functions to object methods 2018-03-16 13:43:21 -07:00
Eric Fischer 623e0c90fe More regularization of JSON writing 2018-03-15 17:07:26 -07:00
Eric Fischer ddf4a45b33 Use JSON writer functions for tile size stats 2018-03-15 15:19:52 -07:00
Eric Fischer 71df4d3e81 Always include tile and layer in tippecanoe-decode, fixing corrupt JSON. 2018-03-14 16:35:59 -07:00
Eric Fischer a8a342f701 Send dot-dropping through the same pipeline.
The first feature in a tile can never be dropped, since there is
no previous feature to attach its properties to.

Remove the previous special case that reset the dropping counter
at the first feature within each tile proper (as opposed to the
first feature in each tile, including its buffer, which is now
the one that is guaranteed to be preserved).
2018-02-23 17:19:54 -08:00
Eric Fischer d7f44ab082 Factor out code for reading metadata.json 2017-11-30 15:49:38 -08:00
Eric Fischer 1acd771743 Factor out code for enumerating the tiles in a directory 2017-11-30 15:37:46 -08:00
Eric Fischer ba1c343204 Make --allow-existing work on dirs; trim .geojson from layer names 2017-11-29 16:24:48 -08:00
Eric Fischer 20e6e80cb1 Fix signed comparison warning from g++ 2017-11-07 11:38:38 -08:00
Eric Fischer 93a325605c Guard against impossible tile coordinates when decoding 2017-10-30 13:24:31 -07:00
Eric Fischer e2b9b96ba8 Detect impossible zoom levels in mbtiles when decoding 2017-10-30 12:55:22 -07:00
Eric Fischer a3e95db0c3 Guard against decoding tiles with an impossible extent 2017-10-30 12:48:55 -07:00
Eric Fischer d13e08c9b5 Guard against null keys and values in tileset metadata 2017-10-27 17:38:07 -07:00
Eric Fischer 748ef3b1d5 Add a tippecanoe-decode option to report tile size and feature count 2017-10-09 17:05:29 -07:00
Eric Fischer e453e32321 Fix test flakiness for filtered tile-join, and some other warnings. 2017-08-09 11:30:16 -07:00
Eric Fischer c79f19e3ca Merge branch 'master' into plugins 2017-08-08 11:08:10 -07:00
Eric Fischer c164a3e69b Add long options to tippecanoe-decode and tile-join. Add --quiet to tile-join. 2017-07-07 12:51:49 -07:00
Eric Fischer b301512860 Merge branch 'master' into plugins 2017-05-12 17:20:30 -07:00
Eric Fischer 9632c14551 Only warn once about invalid polygon encoding in tippecanoe-decode 2017-05-12 11:15:27 -07:00
Eric Fischer 30d54ff50d Tile-join had not been compressing its tiles. 2017-05-11 12:08:47 -07:00
Eric Fischer 922bef72c8 Fix array bounds error found by -fsanitize=address 2017-05-05 11:22:40 -07:00
Eric Fischer 9eb3a7f7ec Enforce polygon winding and closure rules in tippecanoe-decode 2017-05-05 10:56:50 -07:00
Eric Fischer 834ba19277 Add option for newline-delimited output format to tippecanoe-decode 2017-03-28 16:25:40 -07:00
Eric Fischer 9cbbc4c68d Merge branch 'master' into plugins-merge 2017-03-02 10:45:25 -08:00
Eric Fischer 835fafe30a Add an option to decode only specified layers 2017-02-27 10:10:28 -08:00
Eric Fischer 5c2106c0dc Merge branch 'master' into plugins 2017-01-25 16:38:03 -08:00
Eric Fischer c74a9a8325 Add minzoom and maxzoom options to tippecanoe-decode 2017-01-24 14:15:40 -08:00
Eric Fischer 150e3663e1 Pass the index, sequence, and extent through the prefilter when needed 2017-01-13 14:59:11 -08:00
Eric Fischer d1dc310bbc The GeoJSON-producing part of prefiltering 2016-12-08 15:13:38 -08:00
Eric Fischer 6530e155eb Don't put a comma between features in filter output 2016-12-08 11:14:06 -08:00
Eric Fischer 87d86ecfc9 Set close-on-exec flag for most file descriptors 2016-12-07 10:57:56 -08:00
Eric Fischer 112f451c66 Move JSON-writing again to keep it from requiring all plugin code 2016-12-06 17:19:22 -08:00
Eric Fischer 72478ae13e Be more consistent about checking for errors from close() 2016-12-06 17:19:22 -08:00
Eric Fischer adfceed554 Factor out conversion from vector tile to GeoJSON 2016-12-06 17:19:22 -08:00
Eric Fischer 2bc1b9bd91 Support feature IDs for decoding 2016-07-15 13:58:15 -07:00