Commit Graph
1959 Commits
Author SHA1 Message Date
Erica Fischer 47124087ca Unit test coordinate encoding and decoding 2024-04-05 12:23:37 -07:00
Erica Fischer bc5183c075 Found the place that expected to be able to mask bits off the top 2024-04-04 16:37:06 -07:00
Erica Fischer dcda6bcdd5 Work in progress on a new feature sort 2024-04-04 15:29:45 -07:00
Erica Fischer da94dbf9c4 Getting a little closer to 128-bit indices 2024-04-03 14:21:32 -07:00
Erica Fischer 6c148a63f7 Use a typedef for variables representing quadkey/hilbert indices 2024-04-03 13:50:03 -07:00
Erica Fischer 9c7a1b66f5 Revert "Progress on 128-bit indices"
This reverts commit 2abbcc7099.
2024-04-03 12:48:32 -07:00
Erica Fischer 3e1848c67c Merge remote-tracking branch 'origin/main' into more-detail 2024-04-03 12:46:35 -07:00
Erica Fischer bb4f220678 Reduce tiling memory (#227)
* Trying to reduce memory in tiling

* Let the simplification workers go out of scope earlier

* Bail out quickly once the maximum feature count is reached

* Update changelog and version
2.53.0
2024-04-03 07:12:26 -07:00
Erica Fischer 2ea364d32a Make it further into the tests 2024-04-01 12:25:08 -07:00
Erica Fischer 2abbcc7099 Progress on 128-bit indices 2024-03-29 21:55:25 -07:00
Erica Fischer 054becc58f Merge remote-tracking branch 'origin/main' into more-detail 2024-03-29 13:56:41 -07:00
Dane Springmeyer 266ef3c286 Add tippecanoe-overzoom to .gitignore (#224)
add tippecanoe-overzoom to .gitignore
2024-03-25 11:00:18 -07:00
Dane Springmeyer 1519b50791 Upgrade catch from v1.5.7 to v2.13.10 (#223)
upgrade catch from v1.5.7 to v2.13.10
2024-03-25 10:39:11 -07:00
Erica Fischer bd48ba8ea1 Fix accidental loss (at all zooms) of features with an explicit minzoom (#221)
* Fix accidental loss (at all zooms) of features with an explicit minzoom

* Try to stabilize centers and bounding boxes between architectures
2.52.0
2024-03-22 10:20:10 -07:00
Erica Fischer 5dd72da3f8 Work around overflow in distance calculations 2024-03-18 12:47:07 -07:00
Erica Fischer 686ab56c4a Bringing back reverted overflow changes, one at a time 2024-03-18 12:37:24 -07:00
Erica Fischer cf63945c37 Revert "Fix more overflows"
This reverts commit cc6135dd02.
2024-03-18 12:28:19 -07:00
Erica Fischer ccc49bc870 Revert "Use 128-bit arithmetic to avoid overflow"
This reverts commit 3b948e0f20.
2024-03-18 12:28:11 -07:00
Erica Fischer 77431db47d Don't let bounds stick off the edge of the plane 2024-03-18 11:47:46 -07:00
Erica Fischer 3b948e0f20 Use 128-bit arithmetic to avoid overflow 2024-03-18 11:37:47 -07:00
Erica Fischer cc6135dd02 Fix more overflows 2024-03-18 10:17:18 -07:00
Erica Fischer 1c97d476c5 Fix tileset bounds 2024-03-18 09:54:21 -07:00
Erica Fischer 331bd662bc Fix cluster-distance, which works by indices, not coordinates 2024-03-15 15:38:39 -07:00
Erica Fischer 3f45b5c51c Reduce test diff noisiness with intermediate rounding 2024-03-15 12:17:32 -07:00
Erica Fischer db9a4feec2 Trying to figure out what is throwing off this point's location 2024-03-15 10:16:28 -07:00
Erica Fischer fbc7ef52ac 32-bit worlds as hex strings 2024-03-15 08:57:37 -07:00
Erica Fischer accdb96409 Trying to fix tileset bounding boxes 2024-03-14 23:40:56 -07:00
Erica Fischer 96ba174473 Fix higher zooms 2024-03-14 23:27:15 -07:00
Erica Fischer dcf760a439 Fix z1 at least 2024-03-14 23:15:04 -07:00
Erica Fischer c114d17eb4 Change initial_x and initial_y to be 64-bit 2024-03-14 18:43:12 -07:00
Erica Fischer 4632b15b47 Eradicate more 32s 2024-03-14 14:54:54 -07:00
Erica Fischer a12bea7129 Straighten out hilbert and quadkey encoding 2024-03-14 14:36:51 -07:00
Erica Fischer a852c04ed4 More 32s 2024-03-14 13:43:42 -07:00
Erica Fischer cea16d3f7e Checkpoint on parameterizing 32s 2024-03-14 13:30:38 -07:00
Erica Fischer e796400377 Fix null behavior in in/ni expressions (#214)
* Fix null behavior in in/ni expressions

* Add test for null attributes in "ni" filters

* Remove unidecode from "in" expression evaluation again
2.51.0
2024-03-08 13:01:12 -08:00
Erica Fischer 655eccfcc3 Restore the use of unidecode for in/ni operators (#212)
* Restore the use of unidecode for in/ni operators

* Update changelog and version
2.50.0
2024-03-08 09:22:55 -08:00
Erica Fischer 021bb96aeb Allow non-string types to be used in "in" expressions (#211)
* Allow non-string types to be used in "in" expressions

* No unidecode smashing in "in" expressions, though!

* Update version and changelog
2.49.0
2024-03-04 10:16:48 -08:00
Erica Fischer 6a8f1b83d8 Fix some undefined behavior (#209)
* Fix some undefined behavior

* Avoid overflow in line simplification calculations

* Oops, missed a test

* Didn't mean to add that to the Makefile

* Revert "Revert "[ci] test in debug mode (#202)""

This reverts commit c95c328e47.

* Fix reference to out-of-scope pointer

* Fix invalid shift and out-of-bounds vector element reference

* Update changelog and version
2.48.0
2024-03-01 10:11:41 -08:00
Erica Fischer 312e1560a3 Stabilize feature order in overzoom (#210)
* Stabilize feature order in overzoom

* I want my sorts to be stable, please

* Revert "[ci] test in debug mode (#202)"

This reverts commit 853ada87b5.

* No need to reinitialize here
2.47.0
2024-02-29 10:39:10 -08:00
Pal Szabo d9a5c2eed7 highz precision values (#208) 2024-02-28 14:08:34 -08:00
Dane Springmeyer 853ada87b5 [ci] test in debug mode (#202) 2024-02-28 12:54:22 -08:00
Erica Fischer a987197eed Don't swap attributes when reducing tiny polygon dust (#207)
* Don't swap attributes when reducing tiny polygon dust

Because the dust placeholder may be misleadingly far from the feature
that contributed the most area to it

* Remove unused arguments; update changelog
2.46.0
2024-02-26 15:41:26 -08:00
Erica Fischer 2b6630c42c Allow features that would be dropped dynamically to become multiplier features (#199)
* Postpone tagging features as being the first of a multiplier cluster

* Upgrade some dynamically dropped features to multiplier features

* Still don't let it put more features in a cluster than is allowed

* Update tests

* Make the current multiplier cluster size per-layer

* Make sure the first non-empty-geometry in the layer is marked as primary

* Improve comments

* Update changelog and version

* Remove commented out debugging printf

* Factor out duplicated code
2.45.0
2024-02-15 11:34:09 -08:00
Erica Fischer 96f126dd59 FSL-style expressions can use unidecode data to smash case and diacritics (#197)
* Read unidecode data, do some plumbing of it

* More unidecode plumbing

* Do the unidecode smashing, but it doesn't seem to be working

* Ah, that's better!

* Add missing header

* And reorder the includes too

* Shortcut when there is no unidecode data to work with

* Update version and changelog

* Avoid repeated unidecode smashing of the same constant string
2.44.0
2024-02-13 14:18:30 -08:00
Erica Fischer 4e52cbd957 Drop or retain whole multiplier clusters when dropping as needed (#198)
* Prep to track conditions other than just "dropped" or "kept"

* Count up instead of down

* Drop or retain whole multiplier clusters based on their first feature

* Calculate a global feature dropping sequence

* Switch over to using the drop sequence for drop-fraction

* Remove unused arguments for the old drop-fraction implementation

* Fix copy-and-paste bugs, update tests

* Properly incorporate feature_minzoom into the drop sequence, I hope

* Rename drop_by to drop_sequence

* See if sorting within clusters fixes filter stability between zooms

* Remove very chatty debug print

* Update changelog and version

* Use named constants instead of numbers for feature dropping/keeping

* Add comment to explain purpose and method of bit reversal
2.43.0
2024-02-12 10:58:49 -08:00
Erica Fischer e2a7a409c7 Improve tiling speed (#195)
* Add a way to run tippecanoe single-threaded for profiling

* Do less work when the tilestats sample values list is already full

* Save a copy when retrieving the attribute key

* Fewer atomic operations

* Move string hashing from mbtiles to text

* Only do approximate attribute deduplication when writing tiles

* Feature dropping tests are sensitive to exact tile size

* All tile creators now create a string pool for the tile

* Features clipped away to nothing should not participate in that tile

* Revert "Only do approximate attribute deduplication when writing tiles"

This reverts commit c42b34b498.

* Also revert the related test changes

* Revert "Revert "Only do approximate attribute deduplication when writing tiles""

This reverts commit 18509876c3.

* Be more specific about the string hash function

* Use fnv1a instead of std::hash for everything

* Reduce the chance of hash collisions

* Stick a hash search on the front of the tree search in addpool

* Eliminate repeated hashing of the same string

* Switch instead of ifs in json parsing

* A few more cases to populate the hash in addpool

* Store the hash in the tree instead of recalculating

* Add explanatory comment for mysterious argument

* Fewer copies in attribute stringification

* Clean up ancient weirdness in JSON attribute stringification

* More serial_val cleanup

* Pass a serial_feature to rewrite instead of many broken-down arguments

* Get rid of the multiple geometries within `partial`

* Revert "Pass a serial_feature to rewrite instead of many broken-down arguments"

This reverts commit 6f4ab9b725.

* Goodbye, struct coalesce

* Revert "Features clipped away to nothing should not participate in that tile"

This reverts commit 124462fbdc.

* Migrating fields from partial to serial_feature

* Name reconciliation between serial_feature and partial

* Replace struct partial with an augmented serial_feature

* Fix some overzealous search-and-replace renaming

* Don't say struct so often

* Remove more of the former partial construction

* Commenting and cleaning up

* Trying again to avoid all these arguments to rewrite

* I swear I did this same thing before and it didn't work.

* More rewrite cleanup

* Exile --detect-shared-borders to its own file

* Add missing headers

* More commenting and cleanup

* More comments

* Sprinkle consts around

* Emplacing and std::moving

* More cleanup

* That shouldn't have worked after a std::move

* Don't need to allocate memory to compare keys

* Reduce use of the global string pool in tiling

* Another avoidable mvt_value construction

* Further reduction to explicit string pool passing

* These reverses are no longer optimizations

* These layernames can all be references

* Don't drag an unused layername string around with every feature

* Heed a compiler warning about potential buffer overflow

* Fix my confusion about which feature's string pool is relevant

* Avoid some unnecessary allocations in attribute accumulation

* Maybe faster serialization?

* Eliminate a comparison

* Do the same here

* Save a couple of allocations when parsing numbers in JSON

* Immediately assign features to layers instead of subdividing later

* Maintain tilestats for tippecanoe:retain_points_multiplier_sequence

* Crunch out more duplicate attribute values when writing out the tile

* Do tilestats for tippecanoe:retain_points_multiplier_first too

* Shell filters need to be real threads, even if nothing else does

* Simplify tippecanoe_minzoom/maxzoom representation

* Update version and changelog
2.42.0
2024-02-07 17:50:35 -08:00
Can Durukandsnyk-bot c5c4698161 [Snyk] Security upgrade centos from 7 to centos8 (#196)
fix: Dockerfile.centos7 to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-CENTOS7-EXPAT-2408807
- https://snyk.io/vuln/SNYK-CENTOS7-EXPAT-3035862
- https://snyk.io/vuln/SNYK-CENTOS7-NSS-2308137
- https://snyk.io/vuln/SNYK-CENTOS7-NSSSYSINIT-2308141
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2308139

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-02-01 14:38:43 -08:00
Erica Fischer 6a2bce8164 Scale the tile size limit up with the multiplier at low zooms (#192)
* Scale the tile size limit up with the multiplier at low zooms

* Add a test to demonstrate that high zoom tiles can't be extra large

* Update changelog and version

* Fail more cleanly when a tile can't be made small enough

* Guard against a cluster where the start marker has been dropped

* Look harder for a working feature interval instead of giving up

* That change to the drop-smallest logic changed a test output

* Update changelog

* Add explanatory comment
2.41.3
2024-01-31 10:23:05 -08:00
David Haberthür ca1aa58a11 Prettify one code line (#187) 2024-01-29 11:34:47 -08:00
Erica Fischer 7d4d264d50 Speeding up tippecanoe-overzoom (#191)
* Speed up mvt_value comparison

* Converting repetitive ifs to cases

* More conversions from ifs to cases

* Optimize the always-true filter case

* Don't convert types of attributes without accumulators

* Unordered map seems to be faster than map

* Add missing header

* Fix some warnings

* Fix the warnings better

* Avoid an int->string->int conversion

* Lazily initialize layer key and values maps when actually needed

* More switches from maps to unordered_maps

* Sure, I'll take the microoptimization

* More emplacement

* Save some copies

* Emplaces and moves

* Lazy linear scan of attributes instead of building a map

* Extra printfs, missing header

* Avoid clipping if the input and output tiles are the same

* But do clip if the tile extent is being reduced

* Make sure I'm not constructing std::strings here at runtime

* More worrying about runtime string construction

* A couple more std::moves

* Const references!

* More const references

* Another std::move

* Make the string_value of mvt_value std::optional

* Reserve storage when decoding

* Provision for different mvt_values to share a string pool

* Use the string pool when decoding

* Avoid another string construction

* Try limiting the depth of the search for duplicate attributes

* Revert "Try limiting the depth of the search for duplicate attributes"

This reverts commit 9ec94a15ff.

* Update changelog

* Fix typo noticed during code review
2024-01-29 11:33:56 -08:00