Commit Graph
22 Commits
Author SHA1 Message Date
Erica Fischer 23667bb8eb Reduce memory consumption from attribute accumulation (#290)
* Progress on plumbing a string pool for full_keys through

* More plumbing for key_pool

* Don't keep features with identical locations as multiplier features

* Revert "Don't keep features with identical locations as multiplier features"

This reverts commit 413f0c8024.

* Adjust calculated maxzoom to account for duplicate feature locations

* Update changelog and version

* Add a test affected by the maxzoom change with duplicate locations

* Round the drop rate a little for cross-platform test consistency
2024-11-05 14:39:41 -08:00
Patrick M YoungandPatrick Young 120b8f4e41 Fixes bool atrributes coming from FGB files (#289)
Fixes https://github.com/felt/tippecanoe/issues/180.

Co-authored-by: Patrick Young <pyoung@snapchat.com>
2024-11-01 12:27:05 -07:00
Rizki Alhamid f44230f5f5 Fix a bug when reading MultiLineString from .fgb file (#241)
fix logic
2024-08-28 10:34:33 -07:00
Erica Fischer e2a7a409c7 Improve tiling speed (#195)
* Add a way to run tippecanoe single-threaded for profiling

* Do less work when the tilestats sample values list is already full

* Save a copy when retrieving the attribute key

* Fewer atomic operations

* Move string hashing from mbtiles to text

* Only do approximate attribute deduplication when writing tiles

* Feature dropping tests are sensitive to exact tile size

* All tile creators now create a string pool for the tile

* Features clipped away to nothing should not participate in that tile

* Revert "Only do approximate attribute deduplication when writing tiles"

This reverts commit c42b34b498.

* Also revert the related test changes

* Revert "Revert "Only do approximate attribute deduplication when writing tiles""

This reverts commit 18509876c3.

* Be more specific about the string hash function

* Use fnv1a instead of std::hash for everything

* Reduce the chance of hash collisions

* Stick a hash search on the front of the tree search in addpool

* Eliminate repeated hashing of the same string

* Switch instead of ifs in json parsing

* A few more cases to populate the hash in addpool

* Store the hash in the tree instead of recalculating

* Add explanatory comment for mysterious argument

* Fewer copies in attribute stringification

* Clean up ancient weirdness in JSON attribute stringification

* More serial_val cleanup

* Pass a serial_feature to rewrite instead of many broken-down arguments

* Get rid of the multiple geometries within `partial`

* Revert "Pass a serial_feature to rewrite instead of many broken-down arguments"

This reverts commit 6f4ab9b725.

* Goodbye, struct coalesce

* Revert "Features clipped away to nothing should not participate in that tile"

This reverts commit 124462fbdc.

* Migrating fields from partial to serial_feature

* Name reconciliation between serial_feature and partial

* Replace struct partial with an augmented serial_feature

* Fix some overzealous search-and-replace renaming

* Don't say struct so often

* Remove more of the former partial construction

* Commenting and cleaning up

* Trying again to avoid all these arguments to rewrite

* I swear I did this same thing before and it didn't work.

* More rewrite cleanup

* Exile --detect-shared-borders to its own file

* Add missing headers

* More commenting and cleanup

* More comments

* Sprinkle consts around

* Emplacing and std::moving

* More cleanup

* That shouldn't have worked after a std::move

* Don't need to allocate memory to compare keys

* Reduce use of the global string pool in tiling

* Another avoidable mvt_value construction

* Further reduction to explicit string pool passing

* These reverses are no longer optimizations

* These layernames can all be references

* Don't drag an unused layername string around with every feature

* Heed a compiler warning about potential buffer overflow

* Fix my confusion about which feature's string pool is relevant

* Avoid some unnecessary allocations in attribute accumulation

* Maybe faster serialization?

* Eliminate a comparison

* Do the same here

* Save a couple of allocations when parsing numbers in JSON

* Immediately assign features to layers instead of subdividing later

* Maintain tilestats for tippecanoe:retain_points_multiplier_sequence

* Crunch out more duplicate attribute values when writing out the tile

* Do tilestats for tippecanoe:retain_points_multiplier_first too

* Shell filters need to be real threads, even if nothing else does

* Simplify tippecanoe_minzoom/maxzoom representation

* Update version and changelog
2024-02-07 17:50:35 -08:00
Brandon Liu f00eeb486a Fix narrowing of index_size (#126)
Fix narrowing of index_size, affecting indexed flatgeobufs with lots of features [#125]
2023-08-14 14:06:38 -07:00
Tim FinerandTim Finer 0422d77bbe Check for nullptr in call to properties, fixes segfault. (#106)
* Added -fPIE as g++ 9.4.0 complained and would not link

* Prevent crash by checking for properties before dereferencing.

---------

Co-authored-by: Tim Finer <finert@pixia.com>
2023-05-23 09:47:45 -07:00
Erica Fischer 1584543224 Fix the flatgeobuf build (#30) 2022-11-03 14:43:00 -07:00
Erica Fischer af1a7ed7ae Once features can't possibly fit in a tile, stop trying (#9)
* Stop adding features to a tile if it can't possibly work

* Add --integer and --fraction options to tippecanoe-decode

* Carry the strategies field from tileset metadata through tile-join

* Update changelog

* Assign different codes to different kinds of error exits
2022-09-23 20:01:13 -04:00
Brandon Liu 4adb22b74e suppress flatgeobuf message 2022-07-05 15:32:00 +08:00
Brandon Liu d04bd09d11 fix struct name collision [#12] 2022-04-27 22:14:22 +08:00
Brandon Liu 7c726e4d2c add missing column types [#2] 2022-04-23 13:26:26 +08:00
Brandon Liu 170ddf73ff comments cleanup 2022-04-02 12:28:28 +08:00
Brandon Liu 7f678502af FlatGeobuf: read files with heterogeneous geometry types [#8] 2022-04-02 12:07:22 +08:00
Brandon Liu 19241f8373 FlatGeobuf: if fgb has index, assign feature IDs based on sequence number [#2] 2022-03-30 12:37:55 +08:00
Brandon Liu fe6f21fe55 FlatGeobuf: support Point geometries [#2] 2022-03-28 00:01:08 +08:00
Brandon Liu d2951d3437 FlatGeobuf style fixup, fix starting of queue with few features, buffer verification [#2,#6] 2022-03-27 19:14:35 +08:00
Brandon Liu 6e6cd29399 FlatGeobuf multithreading [#2] 2022-03-27 19:14:27 +08:00
Brandon Liu 245570feff FlatGeobuf multipoints [#2] 2022-03-27 19:13:43 +08:00
Brandon Liu 33a9aef758 Handle indeterminate length FlatGeobuf headers, linestring geometries [#2] 2022-03-27 19:13:22 +08:00
Brandon Liu b350a8dda7 FlatGeobuf MultiPolygon parsing [#2] 2022-03-27 19:13:12 +08:00
Brandon Liu 88cedf54d4 FlatGeobuf column type parsing [#2] 2022-03-27 19:13:12 +08:00
Brandon Liu 8a5a2496c3 Implement FlatGeobuf Polygon type parsing [#2] 2022-03-27 19:13:03 +08:00