Commit Graph
10 Commits
Author SHA1 Message Date
Erica Fischer c5f2f0da34 More work on plumbing attribute accumulation through (#263)
* Plumb bounding boxes through potential intersections

* Quick bbox reject for bins that can't possibly intersect

* Inching toward attribute accumulation in megatile handling

* Some sort of test for how all these things interact with each other.

Automatic numeric attribute accumulation does *not* apply to attributes
that have an explicit attribute accumulator set, because the order of
operations is too messy and weird

* More sketching

* More sketching

* Actually do some accumulation

* Put all that behind an --accumulate-numeric flag

* Use the same attribute accumulation logic in binning as in megatiles

* Fix backwards conditional

* Add means, but somehow I have some counts of 0

* Handle aggregated attributes with no base attribute in the feature

* Checkpoint before I break everything

* Found a flaw, now to debug

* Fix a typo that broke accumulation

* Add binning tests

* Make sure IDs make it through on the bins

* Fix count/mean accumulation

* Make the numeric accumulation prefix configurable

* Make sure the accumulate test still works with a different prefix

* Forgot to update this test

* More testing to make sure cluster sizes make it all the way through

* Fix neglected --accumulate-attribute when binning

* Mark unexercised attribute accumulation cases as "can't happen"

* Factor out numeric preservation

* Attrs with the accumulation prefix are just preserved, not accumulated

* Test behavior of prefixed attributes

* Plumbing for exclude and exclude-prefix

* Implement and test attribute prefix stripping in overzoom

* Update version and changelog

* For debugging, make an attribute list of source feature IDs

* Revert "For debugging, make an attribute list of source feature IDs"

This reverts commit 65fc99c9d1.
2024-09-20 15:30:29 -07:00
Erica Fischer 47e774adc4 Improve the appearance of coalesce-densest-as-needed tiles (#247)
* Start to distinguish fixed cluster density setting from as-needed density

* Make consistent {drop,coalesce}-densest decisions between zooms

* Actually track the previous index instead of just intending to

* Clean up collinearities in coalesced features

* To determine densest, look at actual physical distance, not just index

* Don't actually need the previous index in serial_feature now

* Center of mass of one feature to most distant point of the next

* Add apologetic comment

* Wait, how did the tests pass before?

* Revert "Wait, how did the tests pass before?"

This reverts commit f73c8ee543.

* Add --maximum-string-attribute-length option

* Update version and changelog

* A little more testing to make sure
2024-07-16 13:23:23 -07:00
Erica Fischer 96f126dd59 FSL-style expressions can use unidecode data to smash case and diacritics (#197)
* Read unidecode data, do some plumbing of it

* More unidecode plumbing

* Do the unidecode smashing, but it doesn't seem to be working

* Ah, that's better!

* Add missing header

* And reorder the includes too

* Shortcut when there is no unidecode data to work with

* Update version and changelog

* Avoid repeated unidecode smashing of the same constant string
2024-02-13 14:18:30 -08:00
Erica Fischer 4e52cbd957 Drop or retain whole multiplier clusters when dropping as needed (#198)
* Prep to track conditions other than just "dropped" or "kept"

* Count up instead of down

* Drop or retain whole multiplier clusters based on their first feature

* Calculate a global feature dropping sequence

* Switch over to using the drop sequence for drop-fraction

* Remove unused arguments for the old drop-fraction implementation

* Fix copy-and-paste bugs, update tests

* Properly incorporate feature_minzoom into the drop sequence, I hope

* Rename drop_by to drop_sequence

* See if sorting within clusters fixes filter stability between zooms

* Remove very chatty debug print

* Update changelog and version

* Use named constants instead of numbers for feature dropping/keeping

* Add comment to explain purpose and method of bit reversal
2024-02-12 10:58:49 -08:00
Erica Fischer e2a7a409c7 Improve tiling speed (#195)
* Add a way to run tippecanoe single-threaded for profiling

* Do less work when the tilestats sample values list is already full

* Save a copy when retrieving the attribute key

* Fewer atomic operations

* Move string hashing from mbtiles to text

* Only do approximate attribute deduplication when writing tiles

* Feature dropping tests are sensitive to exact tile size

* All tile creators now create a string pool for the tile

* Features clipped away to nothing should not participate in that tile

* Revert "Only do approximate attribute deduplication when writing tiles"

This reverts commit c42b34b498.

* Also revert the related test changes

* Revert "Revert "Only do approximate attribute deduplication when writing tiles""

This reverts commit 18509876c3.

* Be more specific about the string hash function

* Use fnv1a instead of std::hash for everything

* Reduce the chance of hash collisions

* Stick a hash search on the front of the tree search in addpool

* Eliminate repeated hashing of the same string

* Switch instead of ifs in json parsing

* A few more cases to populate the hash in addpool

* Store the hash in the tree instead of recalculating

* Add explanatory comment for mysterious argument

* Fewer copies in attribute stringification

* Clean up ancient weirdness in JSON attribute stringification

* More serial_val cleanup

* Pass a serial_feature to rewrite instead of many broken-down arguments

* Get rid of the multiple geometries within `partial`

* Revert "Pass a serial_feature to rewrite instead of many broken-down arguments"

This reverts commit 6f4ab9b725.

* Goodbye, struct coalesce

* Revert "Features clipped away to nothing should not participate in that tile"

This reverts commit 124462fbdc.

* Migrating fields from partial to serial_feature

* Name reconciliation between serial_feature and partial

* Replace struct partial with an augmented serial_feature

* Fix some overzealous search-and-replace renaming

* Don't say struct so often

* Remove more of the former partial construction

* Commenting and cleaning up

* Trying again to avoid all these arguments to rewrite

* I swear I did this same thing before and it didn't work.

* More rewrite cleanup

* Exile --detect-shared-borders to its own file

* Add missing headers

* More commenting and cleanup

* More comments

* Sprinkle consts around

* Emplacing and std::moving

* More cleanup

* That shouldn't have worked after a std::move

* Don't need to allocate memory to compare keys

* Reduce use of the global string pool in tiling

* Another avoidable mvt_value construction

* Further reduction to explicit string pool passing

* These reverses are no longer optimizations

* These layernames can all be references

* Don't drag an unused layername string around with every feature

* Heed a compiler warning about potential buffer overflow

* Fix my confusion about which feature's string pool is relevant

* Avoid some unnecessary allocations in attribute accumulation

* Maybe faster serialization?

* Eliminate a comparison

* Do the same here

* Save a couple of allocations when parsing numbers in JSON

* Immediately assign features to layers instead of subdividing later

* Maintain tilestats for tippecanoe:retain_points_multiplier_sequence

* Crunch out more duplicate attribute values when writing out the tile

* Do tilestats for tippecanoe:retain_points_multiplier_first too

* Shell filters need to be real threads, even if nothing else does

* Simplify tippecanoe_minzoom/maxzoom representation

* Update version and changelog
2024-02-07 17:50:35 -08:00
Eric Fischer 4f7459c797 Record the command line options in the tileset metadata 2019-04-04 17:03:36 -07:00
Eric Fischer d1a0e495ce Require that per-feature maxzoom and minzoom be small integers
Also fixes a place in the Makefile that accidentally still used
CFLAGS instead of CXXFLAGS
2018-07-12 11:24:30 -07:00
Eric Fischer 62f135a97d Exclude long attribute values from tilestats output, as specified 2017-07-21 12:28:14 -07:00
Eric Fischer 65c095cc2b Clean up #includes and add fields for counting attributes 2017-07-14 16:56:23 -07:00
Eric Fischer 9806db3c0a Make UTF-8 checking into a unit test with Catch 2016-10-05 14:55:32 -07:00