Commit Graph
1898 Commits
Author SHA1 Message Date
Erica Fischer d271cf233d Handle nonzero initial zooms 2023-02-01 13:32:58 -08:00
Erica Fischer 1b176e55c3 It made a tileset! 2023-02-01 13:11:06 -08:00
Erica Fischer 8713b3a003 One bug down 2023-02-01 12:35:36 -08:00
Erica Fischer 893f257f8f More attempts that don't work 2023-01-31 13:45:51 -08:00
Erica Fischer f5366eb767 Works until we get to concatenated tiles 2023-01-30 14:45:48 -08:00
Erica Fischer 4caa1d2758 Sort of works 2023-01-30 14:18:30 -08:00
Erica Fischer 1b1597f44b Closer, but still doesn't work 2023-01-30 13:53:07 -08:00
Erica Fischer 26d93a1a8d Work on alternating compressed and uncompressed in decompression 2023-01-30 13:32:24 -08:00
Erica Fischer 34926cf1dc Tile number is uncompressed, tile content is compressed 2023-01-30 13:21:32 -08:00
Erica Fischer c5f18671d8 Start on decompression 2023-01-27 15:19:32 -08:00
Erica Fischer b89cc9f425 Make enough deflate calls to flush out all the buffered data 2023-01-27 13:43:25 -08:00
Erica Fischer 69ebf3e988 Merge remote-tracking branch 'origin/main' into compression 2023-01-27 13:24:40 -08:00
Erica Fischer 5f5ee62a98 Merge branch 'compression' of github.com:felt/tippecanoe into compression 2023-01-27 13:23:53 -08:00
Erica Fischer b155b4671b It only needs to look for other small features when coalescing, not dropping (#64)
* Back out the slow search for other small features when dropping features

* Actually, *do* look for small features still when coalescing

* Update changelog
2.22.0
2023-01-27 12:04:36 -08:00
Erica Fischer e615668475 Avoid placing polygon labels in holes (#62)
* Avoid placing polygon labels in holes

* Traverse vertices in hilbert order to find potential label points

* Update test to reflect new label placements

* Sorting by Y coordinate is better and easier than Hilbert order

* Give a bonus for being near the center of mass

* Make label points a little more border-shy

* Tall places, not just wide places; border lines, not just border points

* Clean up

* Try diagonals through the features too

* Limit the search for label points to prevent slowing down too much

* At this point the center of mass bonus is doing more harm than good

* Forgot to update test
2.21.0
2023-01-27 11:31:57 -08:00
Erica Fischer c58a8e3b97 Round coordinates instead of truncating them (#60)
* Round coordinates instead of truncating them

* Update all the tests for coordinate rounding changes

* Curses, integer division still truncates

* Fix tests

* Also round instead of shifting when looking for no-op linetos

* Should I worry that the same change for moveto doesn't change any tests?

* Also round instead of shifting when scaling down to maxzoom resolution

* Replace another explicit shift, for origin point

* Round instead of shift when writing clipped geometries to the next zoom

* Fix low-zoom gridding and smaller-than-a-pixel checks

* Don't guess an excessively large maxzoom when there is only one feature

* Add a test for guessing the maxzoom of a single point

* Explicitly sort by index if no other order distinguishes features

* Another affected test
2.20.0
2023-01-27 10:57:22 -08:00
Brandon Liu 1c576c2dc1 Add a few PMTiles details to README; regenerate man page (#58)
* Add details on pmtiles input/output to a few parts of the README. [#57]

* regenerate man page [#57]
2023-01-27 10:44:54 -08:00
Erica Fischer fb7d87145c Remove unused code from the other day 2023-01-26 22:08:29 -08:00
Erica Fischer 8b0490ba35 Hook up more compression functions 2023-01-26 22:07:44 -08:00
Erica Fischer a7bc8856fe Starting toward compression again from a different point 2023-01-26 21:33:52 -08:00
Erica Fischer 98f6be7ebd Merge remote-tracking branch 'origin/main' into compression 2023-01-26 16:44:21 -08:00
Erica Fischer 3cb6e4dd5e Set base zoom if requested as part of maxzoom guessing (#63)
* Set base zoom if requested as part of maxzoom guessing

* Add a test for guessing the maxzoom of a single point

* Don't guess an excessively large maxzoom when there is only one feature

* Update changelog
2.19.0
2023-01-26 15:27:06 -08:00
Erica Fischer 12e3af018b Pull the writing of the serialized feature out to the callers 2023-01-25 14:41:22 -08:00
Erica Fischer 5806be8a5b Add compressed stream writer functions 2023-01-25 13:45:38 -08:00
Erica Fischer a409792fb5 Track file position within fwrite_check() 2023-01-25 13:09:34 -08:00
Erica Fischer da81ad3fed Start adding a stream compressor object 2023-01-25 11:19:04 -08:00
Erica Fischer a035e4338a Remove per-feature compression; move minzoom back into the object 2023-01-25 10:07:22 -08:00
Erica Fischer c7393d8e2e Don't need to store the length of the geometry 2023-01-24 13:26:12 -08:00
Erica Fischer 484dc27c5e Compress individual features in the temporary files 2023-01-24 13:24:36 -08:00
Erica Fischer 21c946dd49 Serialize features into memory rather than straight to disk 2023-01-24 12:19:58 -08:00
Erica Fischer 4c52aeacf1 Fix the pointer bug that was causing the crash 2023-01-24 10:17:25 -08:00
Erica Fischer f1bffd9de0 Crashy work in progress toward compression 2023-01-24 09:38:07 -08:00
Erica Fischer 3addc57bfc Add to-memory versions of the serialization functions 2023-01-23 15:47:38 -08:00
Erica Fischer fade093279 Merge remote-tracking branch 'origin/main' into no-separate-metadata 2023-01-23 10:53:27 -08:00
Erica Fischer 01aa8358f0 Switch string pool from memory to disk when memory is 10% full 2023-01-18 13:45:16 -08:00
Erica Fischer 54b47a6757 Fix crash when using tile-join to copy an empty pmtiles tileset (#61)
* Fix crash when using tile-join to copy an empty pmtiles tileset

* Update changelog
2.18.0
2023-01-17 15:16:22 -08:00
Erica Fischer 1ef0f260f1 When the pool gets too big, switch to just the file, not memory 2023-01-17 11:04:26 -08:00
Erica Fischer 2695db5ebf Also use pread instead of mapping to read in the data to sort 2023-01-12 18:21:46 -08:00
Erica Fischer 966fdb63d4 Sort in allocated rather than mapped memory too 2023-01-12 18:19:43 -08:00
Erica Fischer da2b5bd9fb Move the string pool and search tree from mmap to allocated memory 2023-01-12 11:19:44 -08:00
Erica Fischer 42e2434cca Add missing #include 2023-01-03 11:13:59 -08:00
Erica Fischer 69bf5b324b Limit the size of the string pools and trees to fit in memory 2023-01-03 11:10:52 -08:00
Erica Fischer f59f8588eb Revert "Shave a few bytes off temporary files by using more unsigned integers"
This reverts commit eef29084ec.
2023-01-03 10:55:42 -08:00
Erica Fischer fc8007a2ab Flush stderr after logging progress 2022-12-30 12:38:37 -08:00
Erica Fischer eef29084ec Shave a few bytes off temporary files by using more unsigned integers 2022-12-30 11:46:03 -08:00
Erica Fischer d05f8f8e4c Remove the concept of "separate metadata"
This was an extra level of attribute indirection (features point
to metadata records which point to key and value strings) which was
intended to reduce the size of temporary storage for features with
large numbers of attributes that were also spread across large numbers
of tiles at maxzoom.

For other kinds of features, the extra indirection slowed things down
instead, and, especially when maxzoom guessing was being used, many more
features were having their metadata externalized than could actually
benefit from it.
2022-12-29 14:47:27 -08:00
Erica FischerandBrandon Liu 3b2599f587 Add support for pmtiles output format (#55)
* add pmtiles.hpp from github.com/protomaps/PMTiles [#10]

* tippecanoe main writes pmtiles output. [#10]

* detect output format using suffix
* after mbtiles is done writing, replace with pmtiles based on map/image tables.
* add method to write_json for writing json sub-object.

* tippecanoe-decode reads pmtiles input. [#10]

* tile-join reads and writes pmtiles. [#10]

* pmtiles test suite for decode and tile-join [#10]

* add base GitHub CI action for compiling and test suite.

* update pmtiles.hpp with z>15 fix

* Fix some ordering problems with pmtiles decode

* Pmtiles should also pass the raw tiles tests

* Eradicate spaces from tileset metadata JSON fields

* Eradicate spaces from more test fixtures

* Update more tests

* Pmtiles tests pass now too

* Remove unnecessary sort (and make indent)

* Update changelog

* The allow-existing test for pmtiles needs -o, not -e

* Declare --allow-existing to be unsupported for pmtiles.

It was always a bad idea even for mbtiles.

Co-authored-by: Brandon Liu <bdon@bdon.org>
2.17.0
2022-12-29 12:00:13 -08:00
Sam Hashemi a374263466 Update README (#54) 2022-12-28 14:06:42 -08:00
Erica Fischer 079929717e Avoid spending gigabytes of memory on statistics for -as-needed dropping (#50)
* Downsample indices and areas during tiling if they get too big

* Cap the indices rather than downsampling them
2.16.0
2022-12-20 12:54:21 -08:00
Erica Fischer 2b7bbbdbca Report JSON object context on the same line as the error message (#49)
* Report JSON object context on the same line as the error message

* Update changelog
2022-12-14 12:34:21 -08:00