Commit Graph
44 Commits
Author SHA1 Message Date
Erica Fischer f957f30f90 Clean up internal naming related to tilestats (#190)
* Get rid of the type_and_string near-synonym for serial_val

* Rename file_keys to the more familiar tilestats

* "tas" (type_and_string) => "sv" (serial_val)

* "fk" (tile_keys) => "ts" (tilestats)

* Revert ""tas" (type_and_string) => "sv" (serial_val)"

This reverts commit 4854c57e22.

* More carefully this time: "tas" (type_and_string) => "sv" (serial_val)
2024-01-23 14:17:19 -08:00
Erica Fischer 5d92a17193 Add point retention multiplier (#179)
* Add an option to retain N times as many points as usual at each zoom

* Tests for point multipler with specified and guessed maxzooms

* Work in progress on inverse spatial ordering

* Fix inverse spatial feature order

* --reorder was depending on a feature index that wasn't being preserved

* Separate ordering by feature_minzoom from ordering inverse-spatially

* Add a test for the inverse spatial ordering

* Store the basezoom/droprate/multiplier decisions in tileset metadata

* Progress on adding filters to tippecanoe-overzoom

* Type promotion for comparison

* Look up the attribute value for ordering

* Add test of thinning and ordering features

* Plumb tippecanoe_decisions metadata through pmtiles

* Be careful not to put infinities in JSON

* Fix accidental dropping in what is meant to preserve sparse points

* Start distinguishing true, false, and null in expressions

* Most of the type conversions

* Add boolean conversions

* Literals and conjunctions

* Add filtering to tippecanoe-overzoom

* Add a test of filtering in overzoom

* Fix boolean conjunctions

* Handle the combination of cluster size and filtering

* Rework dot dropping to reconcile density threshold and multiplier

* Revert "Rework dot dropping to reconcile density threshold and multiplier"

This reverts commit f253a66382.

* Retain points by multiplier within each tile, not in global probability

* Test that intends to verify that the multiplier is reversible

* Get the test to detect the discrepancy

* Mark the start of multiplier clusters with a magic attribute

* Add string-contains

* Add in and ni operators

* Revert "Look up the attribute value for ordering"

This reverts commit 56bc73e49a.

* Revert "Type promotion for comparison"

This reverts commit 6f3256f5af.

* Make number formatting in tippecanoe_decisions consistent

* Revert "Add a test for the inverse spatial ordering"

This reverts commit c8047de9ab.

* Revert "Separate ordering by feature_minzoom from ordering inverse-spatially"

This reverts commit 35b19a223c.

* Revert "Fix inverse spatial feature order"

This reverts commit 5978ecdb44.

* Revert "Work in progress on inverse spatial ordering"

This reverts commit fdf230f632.

* Somehow missed the tests associated with that last revert

* Round-robin assign attributes to partials from across the multiplier

* Count the multiplier separately in each layer

* Fix distribution of accumulated attribute across multiplier features

* Update changelog, version, and docs

* Add "is null" and "isnt null" expressions

* Update interpretation of FSL expressions to pass the tests

* Test to assert that polygons are unaffected by the multiplier

* Clean up and comment

* Remove accidental unused case
2024-01-18 15:56:09 -08:00
Erica Fischer f7dc7faf31 Reduce memory use of polygon shard detection (#139)
* Crunch down memory required per polygon joint

* *Actually* reduce the size of the structure

* Update version and changelog
2023-09-21 10:59:12 -07:00
Erica Fischer afc7ca01a2 Calculate an antimeridian-adjusted bounding box in tileset metadata (#82)
* Calculate a new antimeridian-adjusted bounding box

* Add antimeridian bounding box to pmtiles, dirtiles, and tile-join

* Don't take out-of-bounds latitudes into account in the adjusted bbox

* Update changelog

* Forgot to adjust tests after the last change
2023-03-09 20:26:48 -08:00
Erica FischerandBrandon Liu 3b2599f587 Add support for pmtiles output format (#55)
* add pmtiles.hpp from github.com/protomaps/PMTiles [#10]

* tippecanoe main writes pmtiles output. [#10]

* detect output format using suffix
* after mbtiles is done writing, replace with pmtiles based on map/image tables.
* add method to write_json for writing json sub-object.

* tippecanoe-decode reads pmtiles input. [#10]

* tile-join reads and writes pmtiles. [#10]

* pmtiles test suite for decode and tile-join [#10]

* add base GitHub CI action for compiling and test suite.

* update pmtiles.hpp with z>15 fix

* Fix some ordering problems with pmtiles decode

* Pmtiles should also pass the raw tiles tests

* Eradicate spaces from tileset metadata JSON fields

* Eradicate spaces from more test fixtures

* Update more tests

* Pmtiles tests pass now too

* Remove unnecessary sort (and make indent)

* Update changelog

* The allow-existing test for pmtiles needs -o, not -e

* Declare --allow-existing to be unsupported for pmtiles.

It was always a bad idea even for mbtiles.

Co-authored-by: Brandon Liu <bdon@bdon.org>
2022-12-29 12:00:13 -08:00
Erica Fischer 9c77029b63 Add a data structure for tileset metadata (#44)
* Progress toward making a tileset metadata structure

* Write metadata from structure to mbtiles

* Write dirtiles metadata.json from metadata structure

* Update changelog
2022-12-12 13:32:02 -08:00
Erica Fischer 5c647cdb83 Don't preflight zoom levels for potential as-needed dropping (#40)
* Working on eliminating preflighting

* Adjust for the map/images schema change

* Avoid generating duplicate tiles with the detail reduction strategy

* Do error checking if tiles in a directory can't be written

* Update changelog and version

* Revert unintentional code reordering

* Neglected to add the exit on error here
2022-11-29 12:42:30 -08:00
Erica Fischer c65d6938a4 Drop or coalesce points too when dropping or coalescing smallest (#6)
* Handle points too when dropping or coalescing the "smallest" features.

* Add statistics of tile size reduction strategies to tileset metadata

* Update changelog and version

* Update documentation
2022-08-09 15:29:03 -07:00
Eric Fischer 4f7459c797 Record the command line options in the tileset metadata 2019-04-04 17:03:36 -07:00
Eric Fischer bc84b41246 Add options to change the tilestats limits 2018-08-31 15:12:00 -07:00
Eric Fischer 7f90ba5e1b Include version number in tile-join output too 2018-06-07 21:37:25 -07:00
Eric Fischer bb8b01a702 Add option to specify description for attributes in tileset metadata 2018-05-24 14:18:22 -07:00
Eric Fischer 7128ac44d5 Track layer description as part of the tilestats 2018-05-24 10:27:43 -07:00
Eric Fischer 3c827b1d68 Use the standard JSON writer when writing metadata.json 2018-03-16 15:35:26 -07:00
Eric Fischer b03dabe098 Add an option for clustering features 2017-12-14 14:30:08 -08:00
Eric Fischer 4f974b3dc6 Less verbose initializer syntax 2017-11-07 15:25:54 -08:00
Eric Fischer ba62ab8596 More structure initializers 2017-11-07 15:20:17 -08:00
Eric Fischer 372194cee9 Add an option not to produce tilestats 2017-08-15 10:35:41 -07:00
Eric Fischer 9ec52f2c90 Merge branch 'master' into tilestats 2017-07-21 13:08:33 -07:00
Eric Fischer 7e6d10c2fa In tilestats, truncate and merge layers and attributes with long names 2017-07-21 12:56:30 -07:00
Eric Fischer 855e344552 Don't keep tilestats for features that are filtered out in tile-join 2017-07-19 16:53:53 -07:00
Eric Fischer 97d0b2a1b5 Retain attributes in alphabetical order, for consistency 2017-07-19 16:40:06 -07:00
Eric Fischer 3cafef89f1 Factor out duplicated tilestats code 2017-07-19 14:45:15 -07:00
Eric Fischer 18f9a8dbb7 Close mbtiles properly when exiting because there are no features 2017-07-18 09:54:59 -07:00
Eric Fischer 71f57793fd Derive tilejson attribute types from sample attribute values 2017-07-17 17:31:46 -07:00
Eric Fischer 77bf14bfb0 Track sample values for each attribute in each layer 2017-07-17 13:28:46 -07:00
Eric Fischer b1771a3365 Make layer attributes a map to sample values 2017-07-17 11:58:37 -07:00
Eric Fischer 7445feb845 Work in the direction of tracking representative attribute values 2017-07-14 17:59:24 -07:00
Eric Fischer 62ee53992b Track how many features of each type are in each layer 2017-07-14 17:23:41 -07:00
Eric Fischer 65c095cc2b Clean up #includes and add fields for counting attributes 2017-07-14 16:56:23 -07:00
Eric Fischer f558b78380 Include tileset metadata when writing either to directory or mbtiles 2017-04-07 13:25:05 -07:00
Eric Fischer 1362ac448f Add --description option to set mbtiles description 2017-03-21 11:07:15 -07:00
Eric Fischer fa56adc530 Clean up utility functions that are also used in other projects 2017-03-16 15:06:58 -07:00
Eric Fischer 021d792d33 Getting ready for multithreaded tile-joining 2016-09-20 11:04:24 -07:00
Eric Fischer 437152e02b Track the minzoom and maxzoom for each layer separately 2016-09-19 17:53:31 -07:00
Eric Fischer d490d8475e Remove unused layer count and layer name list 2016-08-30 14:17:28 -07:00
Eric Fischer 31d2a3738a Get rid of the old file_keys 2016-08-30 13:46:37 -07:00
Eric Fischer 7f49ce5caa Merge per-thread layer names and file keys 2016-08-29 16:38:57 -07:00
Eric Fischer c26fa23564 Per-thread layermap will contain the file keys, not just a layer ID 2016-08-29 14:59:28 -07:00
Eric Fischer d712edcdc9 Stop using strdup for tile-join matching 2016-05-03 11:14:09 -07:00
Eric Fischer 271ec3d154 Stop using malloc for layer names 2016-05-03 10:52:49 -07:00
Eric Fischer 023ce03672 Fix indentation and bump version number 2016-04-28 15:11:57 -07:00
Eric Fischer adc70341ad Use std::set to track the layer-wide feature attribute types.
Track them during parsing, not tiling.  Remove the old string pool code.
2016-04-28 14:59:58 -07:00
Eric Fischer 65253cba50 Drag header files into C++ 2016-04-27 14:00:14 -07:00