Commit Graph
100 Commits
Author SHA1 Message Date
Erica Fischer 2b7bbbdbca Report JSON object context on the same line as the error message (#49)
* Report JSON object context on the same line as the error message

* Update changelog
2022-12-14 12:34:21 -08:00
Erica Fischer 9c77029b63 Add a data structure for tileset metadata (#44)
* Progress toward making a tileset metadata structure

* Write metadata from structure to mbtiles

* Write dirtiles metadata.json from metadata structure

* Update changelog
2022-12-12 13:32:02 -08:00
Erica Fischer f1df09f147 Generate fewer duplicate label points at high zoom levels (#42) 2022-12-02 11:02:33 -08:00
Erica Fischer 5c647cdb83 Don't preflight zoom levels for potential as-needed dropping (#40)
* Working on eliminating preflighting

* Adjust for the map/images schema change

* Avoid generating duplicate tiles with the detail reduction strategy

* Do error checking if tiles in a directory can't be written

* Update changelog and version

* Revert unintentional code reordering

* Neglected to add the exit on error here
2022-11-29 12:42:30 -08:00
Erica Fischer 3095adb467 Simplify geometry earlier when the in-memory representation of a tile gets large, to reduce peak memory usage (#38)
* Simplify geometries earlier if the in-memory tile starts getting big

* Update changelog
2022-11-22 14:57:07 -08:00
Erica Fischer 8eec2be462 Improve quality of coalescing-as-needed; add options to limit feature count (#25)
* Add an option to limit geometry vertex count

* Update docs and changelog

* Track desired feature count and geometry size in strategies

* Remove dead code for a long-forgotten inaccessible option

* Fix the option name in changelog

* Refine coalesce-smallest to only coalesce onto other small features

* Clean polygons before coalescing-as-needed

* Don't accumulate tiny polygon holes as negative dust

* Add the option not to limit the feature count at maxzoom

* Add options to limit feature count more abruptly in each tile

* Revert "Add the option not to limit the feature count at maxzoom"

This reverts commit ace173ab7a.

* Revert "Fix the option name in changelog"

This reverts commit 38dbce8405.

* Revert "Add an option to limit geometry vertex count"

This reverts commit d379fdf06b.

* Remove test for reverted option

* Remove more leftovers from geometry size limiting

* Update changelog
2022-11-17 13:09:10 -08:00
Erica Fischer 7733cced43 Add an option to drop more points in dense areas than in sparse areas (#35)
* Add an option to drop the densest features by rate, not a fraction

* Restrict dot dropping just to dots, as it should be

* 50% mix of sparse and uniform point preservation

* Update version, changelog, test, and documentation

* 60% sparse, 40% dense

* Didn't update the test after the last change

* Make the drop-denser rate configurable without recompiling
2022-11-16 10:55:07 -08:00
Erica Fischer 622084a507 Limit guessed maxzoom to avoid spending too many tiles on polygon fill (#23)
* Change sqlite3 schema to deduplicate identical tiles

* Limit guessed maxzoom to avoid spending too many tiles on polygon fill

* Fix test.

These dust polygons now have their area calculated because their
maxzoom is being guessed, so the attributes from the largest one
rather than the last one are preserved.

* Increase polygon limit to a million tiles

* Two million tiles ought to be enough for anyone, right?

* Add explanatory comments for mysterious numbers
2022-11-07 10:14:17 -08:00
Erica Fischer 1584543224 Fix the flatgeobuf build (#30) 2022-11-03 14:43:00 -07:00
Erica Fischer 5b03b18f31 Generate label points after simplification, not before. (#27)
* Generate labels points after simplification, not before.

Previously there were some cases where dropping the smallest
polygons would never reduce the number of labels.

* Also wait until after polygon cleaning to make labels

* Remove the extra newlines after the TILES ONLY COMPLETE message
2022-10-21 13:59:36 -07:00
Erica Fischer b763625862 Add an option to generate label points in place of polygons (#20)
* Add an option to generate label points in place of polygons

* Change all these places where I said "extent" but really meant "area"

* Revert "Change all these places where I said "extent" but really meant "area""

This reverts commit 403828d2f7.

* Add --order-smallest-first and --order-largest-first options

* Use Turf's center-of-mass algorithm for polygon label points

* If the label point isn't within the polygon, find one that is

* Don't choose a label point that is too close to a border

* Try a little harder to find an optimal label point

* Checkerboard which tiles labels are generated in, to reduce adjacency

* Use a label point for the general representative point for polygons

(Skipping the iteration to find one that is as far as possible from
the borders)

This makes the labels look better in many cases (like France at z1)
but unfortunately ripples into changing the sequence of polygons in
many tests, so the diff is big.

* Revert "Use a label point for the general representative point for polygons"

This reverts commit 2261adf05e.

* Checkpoint work on spiral labels

* Clip label spirals to the feature bounds

* Fix label test

* Be careful not to place spiral labels too close to borders either

* For spiral anchors, only check tile scale, not feature size

* Update test

* Only try to find a central label point for the largest ring

* In tiny polygon dust, keep the attributes of the largest feature
2022-10-13 14:21:36 -07:00
Erica FischerandRoman Karavia 182093bdc7 Improve precision of polygon area calculations (#19)
* Improve precision of get_area by using long double

* Trying to get consistent polygon area results between ARM and x86

* Calculate polygon area closer to the origin for better precision

* Update changelog

* Also exercise tiny polygon dust in the ring area test

They previously behaved differently here between x86 and ARM

* On M1 Macs, long double is just double anyway, so don't use it

* Be more careful about overflow: scale the polygon ring down into range

* Fix the bug I just introduced in the scaled area calculation

* Use only the sign from the scaled-down area calculation

Co-authored-by: Roman Karavia <47303530+romankaravia@users.noreply.github.com>
2022-10-04 10:04:34 -07:00
Erica Fischer a6abb0bc30 Add the option to use a different simplification level at maxzoom (#17)
* Add an option to specify a different simplification at maxzoom

* Add test
2022-10-03 12:18:02 -07:00
Erica Fischer 4ef430b913 Add an option for the Visvalingam simplification algorithm (#16)
* First attempt at porting Paul Mach's Visvalingam implementation

* Make indent

* Mostly working

* Approximate equivalence from Douglas-Peucker to Visvalingam levels

* Don't simplify away tile boundary crossing points

* Add a command-line option and test for Visvalingam simplification

* Update changelog

* Cleanup in response to review feedback

* Include <stdio.h> to fix compiler warnings
2022-09-30 12:00:07 -07:00
Erica Fischer 6de00c15ec Fix tests that I missed updating in 2.6.2 (#15) 2022-09-27 10:37:02 -07:00
Erica Fischer 3086838a10 Fix wrong-way comparison that was causing buffer overflow crash in tile-join (#14)
* Fix wrong-way comparison that was causing buffer overflow in tile-join

* Update changelog
2022-09-27 09:15:54 -07:00
Erica Fischer af1a7ed7ae Once features can't possibly fit in a tile, stop trying (#9)
* Stop adding features to a tile if it can't possibly work

* Add --integer and --fraction options to tippecanoe-decode

* Carry the strategies field from tileset metadata through tile-join

* Update changelog

* Assign different codes to different kinds of error exits
2022-09-23 20:01:13 -04:00
Erica Fischer 8e46650da7 Upgrade to protozero 1.7.1 (#5)
* Upgrade to protozero 1.7.1

* Bump version number
2022-09-13 11:24:01 -07:00
Erica Fischer 073700aa38 Reduce maxzooms being guessed a little: (#2)
* Reduce maxzooms being guessed a little:

* Use 1.5 standard deviations, not 2, as the minimum distinguishable
* Give overlapping polygons and linestrings more distinct indices

* Add another drop rate guessing options, from the same metrics -zg uses

* Guard against using -rp without -zg
2022-09-08 15:47:32 -07:00
Erica Fischer a447dfc089 Extra coordinate precision; feature ordering; compression improvements
* Add an option to retain extra coordinate precision at maxzoom

* Make sure not to shift away the extra detail from coordinates

* Add an option to convert double-precision attributes to single

* Sort attribute values in tiles to make them compress a little better

* Slightly improve polygon simplification

By choosing a point that would be retained after simplification
to be the start/end point that always gets retained

* I regret making all of these tests involve polygons

* Add an option to specify the size of tiny polygons

* Fix accidental requiring of argument for --single-precision

* Guard against duplicate points when generating "sizes" for them

* Restore the intended behavior that tiny polygons don't get simplified

* Make the extra detail settable rather than always maximizing it

* Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)"

This reverts commit fec5e8354c.

* Add an option to prevent choosing a base zoom higher than the maxzoom

* Keep the drop rate high enough when the basezoom gets constrained

* Revert "Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)""

This reverts commit db6bc27d9e.

* Add --order-by and --order-descending options

* Accept multiple --order-by and --order-descending-by sort keys
2022-09-06 13:08:11 -07:00
Eric Fischer 80eeacc98e Forgot to increment the version number 2020-08-26 12:04:44 -07:00
Eric Fischer ea52e88a58 Update changelog 2020-08-26 10:44:27 -07:00
Eric Fischer 9f8c8508ff Exclude mapbox header libraries from code coverage 2020-08-26 10:11:56 -07:00
Eric Fischer c678f784b7 Also update the Wagyu license 2020-08-26 08:31:12 -07:00
Eric Fischer e0c7f1b91a Upgrade Wagyu to version 0.5.0 2020-08-25 17:00:50 -07:00
Eric Fischer ed9f844cef Fix calculation of mean when accumulating attributes in clusters 2020-02-06 15:28:10 -08:00
Eric Fischer 25bf3957d1 Guard against null data in the mbtiles file when decoding 2019-10-31 13:50:29 -07:00
Eric Fischer 9e81d5a3b2 Explain why you might want uncompressed tiles 2019-10-01 10:09:59 -07:00
Eric Fischer fc810d4156 Fix line numbers in GeoJSON feature parsing error messages 2019-08-30 15:34:24 -07:00
Eric Fischer 78853baf87 Extend the table of precisions 2019-08-19 12:08:01 -07:00
Eric Fischer 7bba6bc7e1 Make the documentation match the code about the "clustered" attribute 2019-07-22 09:59:37 -07:00
Eric Fischer 9e3fed8834 Merge pull request #768 from mapbox/undefined-shifts
Be careful to avoid undefined behavior from shifting negative numbers
2019-06-18 14:57:49 -07:00
Eric Fischer fc335e2221 Be careful to avoid undefined behavior from shifting negative numbers 2019-06-17 17:50:25 -07:00
Eric Fischer b550c7b4a6 Merge pull request #760 from mapbox/preserve-shared-nodes
Add an option to keep intersection nodes from being simplified away
2019-05-16 10:38:56 -07:00
Eric Fischer dcdfb94ba9 Bump version number for --no-simplification-of-shared-nodes 2019-05-16 10:13:43 -07:00
Eric Fischer 42ee195bc3 Add an option to keep intersection nodes from being simplified away 2019-05-13 17:39:07 -07:00
Eric Fischer 9dc92a452a Merge pull request #757 from mapbox/map-scale
Add map scale to table of zoom level equivalences
2019-05-09 16:48:20 -07:00
Eric Fischer 1b14034cb5 Add map scale to table of zoom level equivalences 2019-05-08 13:47:53 -07:00
Eric Fischer 4d3e30773f Be more consistent about when longitudes beyond 180 are allowed 2019-04-12 12:12:14 -07:00
Eric Fischer 7be3710b40 Rebuild documentation 2019-04-08 16:33:11 -07:00
Eric Fischer fcab1da952 Merge branch 'master' into raw-tiles-zoom-enumerate 2019-04-08 16:33:03 -07:00
Eric Fischer a8a183df2d Update changelog and version 2019-04-08 16:32:39 -07:00
Eric Fischer ffab876002 Don't run shell filters if the current zoom is below the minzoom 2019-04-08 16:30:19 -07:00
Eric Fischer 99dc019d92 Fix tests after merge 2019-04-08 16:29:56 -07:00
Eric Fischer e89bdecb6d Merge branch 'master' into raw-tiles-zoom-enumerate 2019-04-08 16:28:12 -07:00
Eric Fischer 382563a5b1 Sort wildcards in Makefile to provide test stability 2019-04-05 13:15:46 -07:00
Eric Fischer 26a07b2167 Forgot to update this test standard 2019-04-05 12:05:03 -07:00
Eric Fischer f95b9bcc65 Use single-quote quoting instead of backslash quoting 2019-04-05 11:30:57 -07:00
Eric Fischer 4f7459c797 Record the command line options in the tileset metadata 2019-04-04 17:03:36 -07:00
Eric Fischer e051c53051 Fix duplicated error message for unknown option 2019-03-19 10:16:14 -07:00
Eric Fischer ac12932089 Return a successful error status for --help and --version 2019-03-19 10:13:44 -07:00
Eric Fischer 7419d64aab Suggest https instead of ssh for "git clone"
to avoid the need for authentication
2019-03-19 10:06:46 -07:00
Eric Fischer 25b9c51ac8 Fix -Z and -z for tile directories in tile-join and tippecanoe-decode 2019-03-18 18:01:32 -07:00
Eric Fischer 945d80400b Merge pull request #725 from mapbox/geobuf-multilinestring
Fix reading of MultiLineStrings in Geobuf input
2019-03-01 16:22:00 -08:00
Eric Fischer 1bc475b789 Fix reading of MultiLineStrings in Geobuf input 2019-03-01 15:52:24 -08:00
Eric Fischer 907ddeb7ba Merge pull request #718 from mapbox/decode-mvt
Make tile-join and tippecanoe-decode more flexible about directories
2019-02-20 14:08:22 -08:00
Eric Fischer 90afdf8fb2 Merge branch 'master' into decode-mvt 2019-02-20 12:06:06 -08:00
Eric Fischer 8df0455230 Make tile-join and tippecanoe-decode more flexible about directories:
* Accept .mvt as well as .pbf in directories of tiles
* Allow tippecanoe-decode and tile-join of directories with no metadata
2019-02-20 12:04:10 -08:00
Eric Fischer 454f75ef91 Merge pull request #716 from mapbox/keep-id-attribute
Don't exclude the attribute that is specified to be the feature ID
2019-02-14 12:40:22 -08:00
Eric Fischer 435a788c3f Don't exclude the attribute that is specified to be the feature ID 2019-02-14 11:40:58 -08:00
Eric Fischer 9387fe6546 Current behavior if the ID attribute is also excluded 2019-02-14 10:48:22 -08:00
Eric Fischer b1ae63ab86 Merge pull request #707 from mapbox/unused-filter
Remove now-unused filter argument during parsing
2019-02-14 10:43:11 -08:00
Eric Fischer 451175dd35 Merge pull request #715 from mapbox/multiple-include
Clarify use of multiple -x or -y options
2019-02-14 10:42:43 -08:00
Eric Fischer 8c2df5f7b4 Clarify use of multiple -x or -y options 2019-02-13 16:45:11 -08:00
Eric Fischer 87bdaa1a25 Merge pull request #711 from andrewharvey/patch-3
update broken links in README.md
2019-02-04 10:06:00 -08:00
Eric Fischer e805d113d0 Remove now-unused filter argument during parsing 2019-01-22 14:26:06 -08:00
Eric Fischer 610afc2332 Merge pull request #702 from mapbox/minzoom-0
Fix a bug that disallowed a per-feature minzoom of 0
2019-01-16 13:08:54 -08:00
Eric Fischer 6da2090960 Fix a bug that disallowed a per-feature minzoom of 0 2019-01-16 11:56:25 -08:00
Eric Fischer 454ae96242 Merge pull request #697 from wsw0108/typo
fix some typo
2019-01-02 10:16:06 -08:00
Eric Fischer 5421dbf8f9 Merge pull request #695 from mapbox/cookbook-replace-features
Add cookbook example for adding and removing features with tile-join
2018-12-20 15:31:34 -08:00
Eric Fischer 0505e0e7d5 Add cookbook example for adding and removing features with tile-join 2018-12-20 15:06:44 -08:00
Eric Fischer 255f795c91 Merge pull request #691 from mapbox/detail-limit
Limit detail to 30 and buffer 127 to prevent coordinate delta overflow
2018-12-18 17:31:51 -08:00
Eric Fischer 32b14b1269 Limit detail to 30 and buffer 127 to prevent coordinate delta overflow 2018-12-18 15:53:11 -08:00
Eric Fischer b31b0b04b6 Merge pull request #684 from mapbox/metadata-exists
Better error message if the output tileset already exists
2018-12-13 15:07:31 -08:00
Eric Fischer 28f773a933 Better error message if the output tileset already exists 2018-12-13 11:20:06 -08:00
Eric Fischer 45fbdf43c5 Merge pull request #683 from mapbox/coalesce-points
Point features may now be coalesced into MultiPoint features with --coalesce
2018-12-12 16:58:30 -08:00
Eric Fischer 6376c283ba Add Hilbert Curve feature sequencing as an option 2018-12-12 10:51:00 -08:00
Eric Fischer 5f1c0e151f Add a table of zoom level precisions to the documentation 2018-12-11 17:52:51 -08:00
Eric Fischer ca4aad0c64 Point features may now be coalesced into MultiPoint features with --coalesce 2018-12-11 17:09:13 -08:00
Eric Fischer 4b20948369 Merge pull request #681 from Plantain/master
Update coalesce-smallest-as-needed documentation
2018-12-04 12:49:23 -08:00
Eric Fischer 9c708aa3c4 Merge pull request #676 from mapbox/empty-coordinate
Make it an error, not a warning, to have missing coordinates for a point
2018-11-26 15:34:11 -08:00
Eric Fischer cf28e6b88b Make it an error, not a warning, to have missing coordinates for a point 2018-11-26 14:50:45 -08:00
Eric Fischer 506526ddfa Merge pull request #657 from mapbox/drop-features-earlier
Drop LineStrings and Polygons early in tiling if they are smaller than a pixel
2018-11-26 10:57:30 -08:00
Eric Fischer 4fec7a162e Reduce coordinate precision so GeoJSON and geobuf have the same results 2018-11-26 10:22:05 -08:00
Eric Fischer 5229990523 Fix gridding of small squares; use less memory for small features 2018-11-21 15:34:43 -08:00
Eric Fischer 739445cb6c Merge pull request #665 from mapbox/coerce-id-leading-zero
Ignore leading zeroes when converting string attributes to feature IDs
2018-11-07 16:31:35 -08:00
Eric Fischer f9c9ff3853 Ignore leading zeroes when converting string attributes to feature IDs 2018-11-07 15:46:44 -08:00
Eric Fischer 949468e6f5 Merge pull request #658 from mapbox/coerce-feature-ids
Add an option to convert stringified number feature IDs to numbers
2018-11-07 14:45:08 -08:00
Eric Fischer a3150642d3 Merge pull request #663 from jingsam/ignore
chore: ignore geobuf tests files and node_modules
2018-11-07 10:36:04 -08:00
Eric Fischer cc7c0cc6c9 Don't test string ID conversion with geobuf, because it fails 2018-11-02 15:28:51 -07:00
Eric Fischer bd2423a5ea Add an option to use a specified feature attribute as the feature ID 2018-11-02 15:21:52 -07:00
Eric Fischer 53de8a2014 Add an option to convert stringified number feature IDs to numbers 2018-11-01 17:07:33 -07:00
Eric Fischer b8ffdda32e Merge pull request #656 from mapbox/tile-join-maxzoom
Warn in tile-join if tilesets being joined have different maxzooms
2018-10-31 16:09:11 -07:00
Eric Fischer 9c79cf7162 Warn in tile-join if tilesets being joined have different maxzooms 2018-10-31 14:25:25 -07:00
Eric Fischer d3ae48534e Merge pull request #650 from mapbox/filter-crash
Fix null pointer crash/add bounding box option
2018-10-23 11:59:28 -07:00
Eric Fischer 1cf3b0accc Fix the feature bounding box after clipping 2018-10-23 10:55:07 -07:00
Eric Fischer 5dfbc4a281 Add --clip-bounding-box option to clip input geometry 2018-10-22 16:49:33 -07:00
Eric Fischer 0acdeef525 Fix null pointer crash when looking for feature extent in filter output 2018-10-22 13:26:53 -07:00
Eric Fischer e18748e29b Merge pull request #647 from mapbox/attribute-compare
Bug fix: mismatched features could be coalesced if from a prefilter
2018-10-10 15:12:38 -07:00
Eric Fischer 1b75ca33b4 Bug fix: mismatched features could be coalesced if from a prefilter 2018-10-10 14:35:46 -07:00