Erica Fischer
40075c843a
Remove debug spew
2023-02-01 14:09:21 -08:00
Erica Fischer
40a1022cc3
Tests pass!
2023-02-01 14:01:46 -08:00
Erica Fischer
373227e422
Fix seeking within compressed feature streams
2023-02-01 13:58:12 -08:00
Erica Fischer
d271cf233d
Handle nonzero initial zooms
2023-02-01 13:32:58 -08:00
Erica Fischer
1b176e55c3
It made a tileset!
2023-02-01 13:11:06 -08:00
Erica Fischer
8713b3a003
One bug down
2023-02-01 12:35:36 -08:00
Erica Fischer
893f257f8f
More attempts that don't work
2023-01-31 13:45:51 -08:00
Erica Fischer
f5366eb767
Works until we get to concatenated tiles
2023-01-30 14:45:48 -08:00
Erica Fischer
4caa1d2758
Sort of works
2023-01-30 14:18:30 -08:00
Erica Fischer
1b1597f44b
Closer, but still doesn't work
2023-01-30 13:53:07 -08:00
Erica Fischer
26d93a1a8d
Work on alternating compressed and uncompressed in decompression
2023-01-30 13:32:24 -08:00
Erica Fischer
34926cf1dc
Tile number is uncompressed, tile content is compressed
2023-01-30 13:21:32 -08:00
Erica Fischer
c5f18671d8
Start on decompression
2023-01-27 15:19:32 -08:00
Erica Fischer
b89cc9f425
Make enough deflate calls to flush out all the buffered data
2023-01-27 13:43:25 -08:00
Erica Fischer
69ebf3e988
Merge remote-tracking branch 'origin/main' into compression
2023-01-27 13:24:40 -08:00
Erica Fischer
5f5ee62a98
Merge branch 'compression' of github.com:felt/tippecanoe into compression
2023-01-27 13:23:53 -08:00
Erica Fischer
98f6be7ebd
Merge remote-tracking branch 'origin/main' into compression
2023-01-26 16:44:21 -08:00
Erica Fischer
12e3af018b
Pull the writing of the serialized feature out to the callers
2023-01-25 14:41:22 -08:00
Erica Fischer
5806be8a5b
Add compressed stream writer functions
2023-01-25 13:45:38 -08:00
Erica Fischer
a409792fb5
Track file position within fwrite_check()
2023-01-25 13:09:34 -08:00
Erica Fischer
da81ad3fed
Start adding a stream compressor object
2023-01-25 11:19:04 -08:00
Erica Fischer
a035e4338a
Remove per-feature compression; move minzoom back into the object
2023-01-25 10:07:22 -08:00
Erica Fischer
c7393d8e2e
Don't need to store the length of the geometry
2023-01-24 13:26:12 -08:00
Erica Fischer
484dc27c5e
Compress individual features in the temporary files
2023-01-24 13:24:36 -08:00
Erica Fischer
21c946dd49
Serialize features into memory rather than straight to disk
2023-01-24 12:19:58 -08:00
Erica Fischer
4c52aeacf1
Fix the pointer bug that was causing the crash
2023-01-24 10:17:25 -08:00
Erica Fischer
f1bffd9de0
Crashy work in progress toward compression
2023-01-24 09:38:07 -08:00
Erica Fischer
3addc57bfc
Add to-memory versions of the serialization functions
2023-01-23 15:47:38 -08:00
Erica Fischer
fade093279
Merge remote-tracking branch 'origin/main' into no-separate-metadata
2023-01-23 10:53:27 -08:00
Erica Fischer
01aa8358f0
Switch string pool from memory to disk when memory is 10% full
2023-01-18 13:45:16 -08:00
Erica Fischer
1ef0f260f1
When the pool gets too big, switch to just the file, not memory
2023-01-17 11:04:26 -08:00
Erica Fischer
2695db5ebf
Also use pread instead of mapping to read in the data to sort
2023-01-12 18:21:46 -08:00
Erica Fischer
966fdb63d4
Sort in allocated rather than mapped memory too
2023-01-12 18:19:43 -08:00
Erica Fischer
da2b5bd9fb
Move the string pool and search tree from mmap to allocated memory
2023-01-12 11:19:44 -08:00
Erica Fischer
42e2434cca
Add missing #include
2023-01-03 11:13:59 -08:00
Erica Fischer
69bf5b324b
Limit the size of the string pools and trees to fit in memory
2023-01-03 11:10:52 -08:00
Erica Fischer
f59f8588eb
Revert "Shave a few bytes off temporary files by using more unsigned integers"
...
This reverts commit eef29084ec
2023-01-03 10:55:42 -08:00
Erica Fischer
fc8007a2ab
Flush stderr after logging progress
2022-12-30 12:38:37 -08:00
Erica Fischer
eef29084ec
Shave a few bytes off temporary files by using more unsigned integers
2022-12-30 11:46:03 -08:00
Erica Fischer
d05f8f8e4c
Remove the concept of "separate metadata"
...
This was an extra level of attribute indirection (features point
to metadata records which point to key and value strings) which was
intended to reduce the size of temporary storage for features with
large numbers of attributes that were also spread across large numbers
of tiles at maxzoom.
For other kinds of features, the extra indirection slowed things down
instead, and, especially when maxzoom guessing was being used, many more
features were having their metadata externalized than could actually
benefit from it.
2022-12-29 14:47:27 -08:00
Erica Fischer
4ea8a37611
Accept tilestats limiting options in tile-join, not just tippecanoe ( #8 )
2022-08-12 13:57:04 -07:00
Erica Fischer
fec5e8354c
Improve maxzoom guessing for tightly-clustered point data sources ( #4 )
...
* Improve maxzoom guessing for tightly-clustered point data sources
* Go back to the old distance estimate, since it is less mysterious
* Update changelog and version
2022-08-11 10:13:54 -07:00
Erica Fischer
a9bf6cacd4
Add --smallest-maximum-zoom-guess to specify a minimum maxzoom to guess ( #2 )
...
* Add --maximum-zoom-at-least to specify a minimum maxzoom to guess
* Check whether the specified minimum maxzoom is out of range
* Rename the option to --smallest-maximum-zoom-guess
* Update tests, changelog, and version
2022-08-11 10:04:03 -07:00
Erica Fischer
86f3e98c02
Track the desired tile size for each zoom level in tileset metadata ( #7 )
2022-08-11 09:58:00 -07:00
Erica Fischer
c65d6938a4
Drop or coalesce points too when dropping or coalescing smallest ( #6 )
...
* Handle points too when dropping or coalescing the "smallest" features.
* Add statistics of tile size reduction strategies to tileset metadata
* Update changelog and version
* Update documentation
2022-08-09 15:29:03 -07:00
Erica Fischer
67cd9d8d85
Reduce tippecanoe memory usage ( #5 )
...
* Change JSON objects to a union type to use less memory
* Stop storing the string representation of JSON numbers
* Restore the ability to create features with large integer attributes
* Make sure large-integer feature IDs still behave as before
* Add missing #include
* Don't preallocate as much space for arrays and objects
* Treat inability to check free disk space as a warning, not an error
* Update changelog and version
2022-08-09 15:18:51 -07:00
Erica Fischer
6978a0e102
Reindent with clang-format ( #3 )
2022-07-28 11:19:30 -07:00
Erica Fischer