Commit Graph
16 Commits
Author SHA1 Message Date
Erica Fischer 2695db5ebf Also use pread instead of mapping to read in the data to sort 2023-01-12 18:21:46 -08:00
Erica Fischer 966fdb63d4 Sort in allocated rather than mapped memory too 2023-01-12 18:19:43 -08:00
Erica Fischer da2b5bd9fb Move the string pool and search tree from mmap to allocated memory 2023-01-12 11:19:44 -08:00
Erica Fischer 42e2434cca Add missing #include 2023-01-03 11:13:59 -08:00
Erica Fischer 69bf5b324b Limit the size of the string pools and trees to fit in memory 2023-01-03 11:10:52 -08:00
Erica Fischer f59f8588eb Revert "Shave a few bytes off temporary files by using more unsigned integers"
This reverts commit eef29084ec.
2023-01-03 10:55:42 -08:00
Erica Fischer fc8007a2ab Flush stderr after logging progress 2022-12-30 12:38:37 -08:00
Erica Fischer eef29084ec Shave a few bytes off temporary files by using more unsigned integers 2022-12-30 11:46:03 -08:00
Erica Fischer d05f8f8e4c Remove the concept of "separate metadata"
This was an extra level of attribute indirection (features point
to metadata records which point to key and value strings) which was
intended to reduce the size of temporary storage for features with
large numbers of attributes that were also spread across large numbers
of tiles at maxzoom.

For other kinds of features, the extra indirection slowed things down
instead, and, especially when maxzoom guessing was being used, many more
features were having their metadata externalized than could actually
benefit from it.
2022-12-29 14:47:27 -08:00
Erica Fischer 4ea8a37611 Accept tilestats limiting options in tile-join, not just tippecanoe (#8) 2022-08-12 13:57:04 -07:00
Erica Fischer fec5e8354c Improve maxzoom guessing for tightly-clustered point data sources (#4)
* Improve maxzoom guessing for tightly-clustered point data sources

* Go back to the old distance estimate, since it is less mysterious

* Update changelog and version
2022-08-11 10:13:54 -07:00
Erica Fischer a9bf6cacd4 Add --smallest-maximum-zoom-guess to specify a minimum maxzoom to guess (#2)
* Add --maximum-zoom-at-least to specify a minimum maxzoom to guess

* Check whether the specified minimum maxzoom is out of range

* Rename the option to --smallest-maximum-zoom-guess

* Update tests, changelog, and version
2022-08-11 10:04:03 -07:00
Erica Fischer 86f3e98c02 Track the desired tile size for each zoom level in tileset metadata (#7) 2022-08-11 09:58:00 -07:00
Erica Fischer c65d6938a4 Drop or coalesce points too when dropping or coalescing smallest (#6)
* Handle points too when dropping or coalescing the "smallest" features.

* Add statistics of tile size reduction strategies to tileset metadata

* Update changelog and version

* Update documentation
2022-08-09 15:29:03 -07:00
Erica Fischer 67cd9d8d85 Reduce tippecanoe memory usage (#5)
* Change JSON objects to a union type to use less memory

* Stop storing the string representation of JSON numbers

* Restore the ability to create features with large integer attributes

* Make sure large-integer feature IDs still behave as before

* Add missing #include

* Don't preallocate as much space for arrays and objects

* Treat inability to check free disk space as a warning, not an error

* Update changelog and version
2022-08-09 15:18:51 -07:00
Erica Fischer 6978a0e102 Reindent with clang-format (#3) 2022-07-28 11:19:30 -07:00