Commit Graph
28 Commits
Author SHA1 Message Date
Erica Fischer 4632b15b47 Eradicate more 32s 2024-03-14 14:54:54 -07:00
Erica Fischer e2a7a409c7 Improve tiling speed (#195)
* Add a way to run tippecanoe single-threaded for profiling

* Do less work when the tilestats sample values list is already full

* Save a copy when retrieving the attribute key

* Fewer atomic operations

* Move string hashing from mbtiles to text

* Only do approximate attribute deduplication when writing tiles

* Feature dropping tests are sensitive to exact tile size

* All tile creators now create a string pool for the tile

* Features clipped away to nothing should not participate in that tile

* Revert "Only do approximate attribute deduplication when writing tiles"

This reverts commit c42b34b498.

* Also revert the related test changes

* Revert "Revert "Only do approximate attribute deduplication when writing tiles""

This reverts commit 18509876c3.

* Be more specific about the string hash function

* Use fnv1a instead of std::hash for everything

* Reduce the chance of hash collisions

* Stick a hash search on the front of the tree search in addpool

* Eliminate repeated hashing of the same string

* Switch instead of ifs in json parsing

* A few more cases to populate the hash in addpool

* Store the hash in the tree instead of recalculating

* Add explanatory comment for mysterious argument

* Fewer copies in attribute stringification

* Clean up ancient weirdness in JSON attribute stringification

* More serial_val cleanup

* Pass a serial_feature to rewrite instead of many broken-down arguments

* Get rid of the multiple geometries within `partial`

* Revert "Pass a serial_feature to rewrite instead of many broken-down arguments"

This reverts commit 6f4ab9b725.

* Goodbye, struct coalesce

* Revert "Features clipped away to nothing should not participate in that tile"

This reverts commit 124462fbdc.

* Migrating fields from partial to serial_feature

* Name reconciliation between serial_feature and partial

* Replace struct partial with an augmented serial_feature

* Fix some overzealous search-and-replace renaming

* Don't say struct so often

* Remove more of the former partial construction

* Commenting and cleaning up

* Trying again to avoid all these arguments to rewrite

* I swear I did this same thing before and it didn't work.

* More rewrite cleanup

* Exile --detect-shared-borders to its own file

* Add missing headers

* More commenting and cleanup

* More comments

* Sprinkle consts around

* Emplacing and std::moving

* More cleanup

* That shouldn't have worked after a std::move

* Don't need to allocate memory to compare keys

* Reduce use of the global string pool in tiling

* Another avoidable mvt_value construction

* Further reduction to explicit string pool passing

* These reverses are no longer optimizations

* These layernames can all be references

* Don't drag an unused layername string around with every feature

* Heed a compiler warning about potential buffer overflow

* Fix my confusion about which feature's string pool is relevant

* Avoid some unnecessary allocations in attribute accumulation

* Maybe faster serialization?

* Eliminate a comparison

* Do the same here

* Save a couple of allocations when parsing numbers in JSON

* Immediately assign features to layers instead of subdividing later

* Maintain tilestats for tippecanoe:retain_points_multiplier_sequence

* Crunch out more duplicate attribute values when writing out the tile

* Do tilestats for tippecanoe:retain_points_multiplier_first too

* Shell filters need to be real threads, even if nothing else does

* Simplify tippecanoe_minzoom/maxzoom representation

* Update version and changelog
2024-02-07 17:50:35 -08:00
Erica Fischer af1a7ed7ae Once features can't possibly fit in a tile, stop trying (#9)
* Stop adding features to a tile if it can't possibly work

* Add --integer and --fraction options to tippecanoe-decode

* Carry the strategies field from tileset metadata through tile-join

* Update changelog

* Assign different codes to different kinds of error exits
2022-09-23 20:01:13 -04:00
Erica Fischer 67cd9d8d85 Reduce tippecanoe memory usage (#5)
* Change JSON objects to a union type to use less memory

* Stop storing the string representation of JSON numbers

* Restore the ability to create features with large integer attributes

* Make sure large-integer feature IDs still behave as before

* Add missing #include

* Don't preallocate as much space for arrays and objects

* Treat inability to check free disk space as a warning, not an error

* Update changelog and version
2022-08-09 15:18:51 -07:00
Erica Fischer 6978a0e102 Reindent with clang-format (#3) 2022-07-28 11:19:30 -07:00
Eric Fischer 1bc475b789 Fix reading of MultiLineStrings in Geobuf input 2019-03-01 15:52:24 -08:00
Eric Fischer d1a0e495ce Require that per-feature maxzoom and minzoom be small integers
Also fixes a place in the Makefile that accidentally still used
CFLAGS instead of CXXFLAGS
2018-07-12 11:24:30 -07:00
Eric Fischer 1b26becad9 Clear up some confusion about attribute count and external references
Now the count is always adjacent to whereever the key/value pair is
stored, and is not kept in the serial feature object other than as
the length of the vectors of keys and values.
2018-04-05 15:40:14 -07:00
Eric Fischer fac0ebbf52 All the other places where I used volatile but really wanted atomic 2018-03-13 15:21:21 -07:00
Eric Fischer 2d87059a04 Some more constructors 2017-11-10 13:59:57 -08:00
Eric Fischer dca1ebbbfb Get rid of some C99-style variable-length arrays 2017-11-09 16:54:06 -08:00
Eric Fischer 0152db4a20 More initializers 2017-11-07 15:57:56 -08:00
Eric Fischer ba62ab8596 More structure initializers 2017-11-07 15:20:17 -08:00
Eric Fischer 50910b211b Remove more unused parameters 2017-11-07 11:09:22 -08:00
Eric Fischer c3d23675d1 Also fix arithmetic overflow when reading a Geobuf GeometryCollection 2017-09-07 13:48:09 -07:00
Eric Fischer 7b71f41e43 Fix compiler warnings 2017-09-01 17:36:43 -07:00
Eric Fischer 223d837736 Multithreaded geobuf feature parsing 2017-08-31 13:54:04 -07:00
Eric Fischer 076dfcdfeb Work around differences of opinion about the range of feature IDs
Geobuf uses signed 64-bit ints
Vector tile spec uses unsigned 64-bit ints
Geobuf silently corrupts IDs through use of floating point
2017-08-30 14:34:59 -07:00
Eric Fischer f65faaf3da Support GeometryCollections in geobuf 2017-08-30 14:10:35 -07:00
Eric Fischer 5c5fbe2617 Support top-level geometries 2017-08-30 14:01:09 -07:00
Eric Fischer 5665d08745 Support per-feature minzoom and maxzoom in geobuf 2017-08-30 11:44:57 -07:00
Eric Fischer 4d6ebaa088 Fix nulls and negative integers in geobuf parsing 2017-08-29 16:04:50 -07:00
Eric Fischer f9a007e8c3 Use milo dtoa for consistent string representation of numbers 2017-08-29 14:44:34 -07:00
Eric Fischer 7c031a9796 Memory-map geobuf input instead of reading it into a temporary string 2017-08-28 11:30:30 -07:00
Eric Fischer 8d09f0769e Fix multipolygon and feature ID parsing 2017-08-25 17:40:08 -07:00
Eric Fischer ad89fb88fb Most of the way through geometry decoding 2017-08-25 16:36:53 -07:00
Eric Fischer b98bf6e8c7 Get attribute value decoding working 2017-08-25 15:46:32 -07:00
Eric Fischer 75be013059 Start on geobuf parsing 2017-08-25 14:56:03 -07:00