Commit Graph
200 Commits
Author SHA1 Message Date
James 434817bde9 Update makefile to use c++17 standard (#94)
The C++ standard didn't match the utilization inside the code and would fail to compile using the original makefile
2023-05-04 10:49:00 -07:00
Erica FischerandAurele Nitoref 329dd86bfd Add --cluster-maxzoom option (#75)
* Add `--cluster-maxzoom` option

* Add tests, update changelog

* Update manpage

* Update tests

---------

Co-authored-by: Aurele Nitoref <aurele.nitoref@icloud.com>
2023-02-24 16:05:45 -08:00
DarcyB 7371072e45 Make Tippecanoe build on FreeBSD in response to issue 18 (#72)
* Make tippecanoe build on FreeBSD in responce to issue 18 (https://github.com/felt/tippecanoe/issues/18)

* Make conditional FreeBSD specific, instead of otherway around.
2023-02-24 11:32:56 -08:00
DarcyB 0fa758c1e3 Allow passing a file with a list of input files to tile-join in response to issue 69 (#73)
* Allow passing a file with a list of input files to tile-join in response to Issue 69 (https://github.com/felt/tippecanoe/issues/69)

* Do not print usage if -r is specified without a commandline readfrom file

* Add test for -r option to tile-join
2023-02-24 11:31:46 -08:00
Erica Fischer f2127fec97 Reduce thrashing during feature ingestion and tiling (#56)
* Remove the concept of "separate metadata"

This was an extra level of attribute indirection (features point
to metadata records which point to key and value strings) which was
intended to reduce the size of temporary storage for features with
large numbers of attributes that were also spread across large numbers
of tiles at maxzoom.

For other kinds of features, the extra indirection slowed things down
instead, and, especially when maxzoom guessing was being used, many more
features were having their metadata externalized than could actually
benefit from it.

* Shave a few bytes off temporary files by using more unsigned integers

* Flush stderr after logging progress

* Revert "Shave a few bytes off temporary files by using more unsigned integers"

This reverts commit eef29084ec.

* Limit the size of the string pools and trees to fit in memory

* Add missing #include

* Move the string pool and search tree from mmap to allocated memory

* Sort in allocated rather than mapped memory too

* Also use pread instead of mapping to read in the data to sort

* When the pool gets too big, switch to just the file, not memory

* Switch string pool from memory to disk when memory is 10% full

* Add to-memory versions of the serialization functions

* Crashy work in progress toward compression

* Fix the pointer bug that was causing the crash

* Serialize features into memory rather than straight to disk

* Compress individual features in the temporary files

* Don't need to store the length of the geometry

* Remove per-feature compression; move minzoom back into the object

* Start adding a stream compressor object

* Track file position within fwrite_check()

* Add compressed stream writer functions

* Pull the writing of the serialized feature out to the callers

* Starting toward compression again from a different point

* Hook up more compression functions

* Remove unused code from the other day

* Make enough deflate calls to flush out all the buffered data

* Start on decompression

* Tile number is uncompressed, tile content is compressed

* Work on alternating compressed and uncompressed in decompression

* Closer, but still doesn't work

* Sort of works

* Works until we get to concatenated tiles

* More attempts that don't work

* One bug down

* It made a tileset!

* Handle nonzero initial zooms

* Fix seeking within compressed feature streams

* Tests pass!

* Remove debug spew

* Oops: remember to delete the temporary files so they don't hang around

* Test that fails with the current compression code

* Properly account for bytes read while closing the compressed stream

* Limit the number of warnings about bad label points

* A little more armor when closing decompression

* This time for sure

* A different, less fragile, test that failed previously with compression

* Move feature stream compression to its own file

* Remove now-unused code to deserialize from a file

* Forgot to add the new files

* Remove a little debugging logging

* Add a couple of comments on what it means to be within decompression

* Fix indentation

* Update changelog. Remove stray debugging comment.
2023-02-14 12:47:40 -08:00
Erica Fischer c58a8e3b97 Round coordinates instead of truncating them (#60)
* Round coordinates instead of truncating them

* Update all the tests for coordinate rounding changes

* Curses, integer division still truncates

* Fix tests

* Also round instead of shifting when looking for no-op linetos

* Should I worry that the same change for moveto doesn't change any tests?

* Also round instead of shifting when scaling down to maxzoom resolution

* Replace another explicit shift, for origin point

* Round instead of shift when writing clipped geometries to the next zoom

* Fix low-zoom gridding and smaller-than-a-pixel checks

* Don't guess an excessively large maxzoom when there is only one feature

* Add a test for guessing the maxzoom of a single point

* Explicitly sort by index if no other order distinguishes features

* Another affected test
2023-01-27 10:57:22 -08:00
Erica Fischer 54b47a6757 Fix crash when using tile-join to copy an empty pmtiles tileset (#61)
* Fix crash when using tile-join to copy an empty pmtiles tileset

* Update changelog
2023-01-17 15:16:22 -08:00
Erica FischerandBrandon Liu 3b2599f587 Add support for pmtiles output format (#55)
* add pmtiles.hpp from github.com/protomaps/PMTiles [#10]

* tippecanoe main writes pmtiles output. [#10]

* detect output format using suffix
* after mbtiles is done writing, replace with pmtiles based on map/image tables.
* add method to write_json for writing json sub-object.

* tippecanoe-decode reads pmtiles input. [#10]

* tile-join reads and writes pmtiles. [#10]

* pmtiles test suite for decode and tile-join [#10]

* add base GitHub CI action for compiling and test suite.

* update pmtiles.hpp with z>15 fix

* Fix some ordering problems with pmtiles decode

* Pmtiles should also pass the raw tiles tests

* Eradicate spaces from tileset metadata JSON fields

* Eradicate spaces from more test fixtures

* Update more tests

* Pmtiles tests pass now too

* Remove unnecessary sort (and make indent)

* Update changelog

* The allow-existing test for pmtiles needs -o, not -e

* Declare --allow-existing to be unsupported for pmtiles.

It was always a bad idea even for mbtiles.

Co-authored-by: Brandon Liu <bdon@bdon.org>
2022-12-29 12:00:13 -08:00
Erica Fischer 4ef430b913 Add an option for the Visvalingam simplification algorithm (#16)
* First attempt at porting Paul Mach's Visvalingam implementation

* Make indent

* Mostly working

* Approximate equivalence from Douglas-Peucker to Visvalingam levels

* Don't simplify away tile boundary crossing points

* Add a command-line option and test for Visvalingam simplification

* Update changelog

* Cleanup in response to review feedback

* Include <stdio.h> to fix compiler warnings
2022-09-30 12:00:07 -07:00
Erica Fischer af1a7ed7ae Once features can't possibly fit in a tile, stop trying (#9)
* Stop adding features to a tile if it can't possibly work

* Add --integer and --fraction options to tippecanoe-decode

* Carry the strategies field from tileset metadata through tile-join

* Update changelog

* Assign different codes to different kinds of error exits
2022-09-23 20:01:13 -04:00
Erica Fischer 4ea8a37611 Accept tilestats limiting options in tile-join, not just tippecanoe (#8) 2022-08-12 13:57:04 -07:00
Brandon Liu f442e403c5 add logging struct with -u json options flag, log tiling progress [#13] 2022-07-05 15:32:11 +08:00
Brandon Liu 8a5a2496c3 Implement FlatGeobuf Polygon type parsing [#2] 2022-03-27 19:13:03 +08:00
Eric Fischer e89bdecb6d Merge branch 'master' into raw-tiles-zoom-enumerate 2019-04-08 16:28:12 -07:00
Eric Fischer 382563a5b1 Sort wildcards in Makefile to provide test stability 2019-04-05 13:15:46 -07:00
Eric Fischer f95b9bcc65 Use single-quote quoting instead of backslash quoting 2019-04-05 11:30:57 -07:00
Eric Fischer 4f7459c797 Record the command line options in the tileset metadata 2019-04-04 17:03:36 -07:00
Eric Fischer 25b9c51ac8 Fix -Z and -z for tile directories in tile-join and tippecanoe-decode 2019-03-18 18:01:32 -07:00
Eric Fischer cc7c0cc6c9 Don't test string ID conversion with geobuf, because it fails 2018-11-02 15:28:51 -07:00
Eric Fischer b2999fe9a8 Create the output tile directory even if there are no valid features 2018-09-24 11:16:13 -07:00
jingsam 5d6e6e07d6 chore: ignore generated check and mbtiles files in tests 2018-08-23 14:16:50 +08:00
Eric Fischer addbd0a854 Keep tile-join from generating a tileset name longer than 255 chars 2018-08-20 17:45:18 -07:00
Eric Fischer 79363e397b Factor out the GeoJSON parsing loop from feature serialization 2018-08-08 13:11:10 -07:00
Eric Fischer 8d67e460d4 Add --exclude-all to tile-join 2018-08-07 10:27:38 -07:00
Eric Fischer c9cc12e7ff Add "csv" to the list of suffixes to trim off of layer names 2018-08-01 17:49:54 -07:00
Eric Fischer 87a1bb7851 Add an option to treat empty CSV columns as nulls, not empty strings 2018-07-19 14:33:33 -07:00
Eric Fischer 3cb09d0e2a Add the ability to specify the file format with -L{"format":"…"} 2018-07-17 14:57:56 -07:00
Eric Fischer d1a0e495ce Require that per-feature maxzoom and minzoom be small integers
Also fixes a place in the Makefile that accidentally still used
CFLAGS instead of CXXFLAGS
2018-07-12 11:24:30 -07:00
Eric Fischer 4da6b932b3 Use "generator" instead of "creator". Silence another status message. 2018-06-07 15:17:58 -07:00
Eric Fischer b0229530ac Turn off progress indicators during tests
(For the sake of Travis's 4MB log limit)

Also fix a few places that logged progress even when told not to.
2018-06-07 13:27:49 -07:00
Eric Fischer 11c6c869aa Include a creator field in tileset metadata with the Tippecanoe version
Also finally eradicates binary diffs from the tests
2018-06-07 11:20:12 -07:00
Eric Fischer 48f7f1dd38 Preserve layer and attribute descriptions across tile-join 2018-05-24 16:01:57 -07:00
Eric Fischer 32b3a15406 Add test for JSON specification of layer file, name, and description 2018-05-24 12:41:14 -07:00
Eric Fischer d0d953f4a2 Use a loop instead of spelling out .json and .json.gz separately 2018-05-14 10:46:45 -07:00
Eric Fischer 450a8a741f Fix reference to uncompressed version of test file that I compressed 2018-05-11 16:49:09 -07:00
Eric Fischer dbd69142b0 Test and documentation 2018-05-11 16:34:18 -07:00
Eric Fischer d01d8177a0 Allow filtering features by zoom level in conditional expressions
This moves filtering from the serialization stage to the
tiling stage so that the zoom level can be known to the filter.

The side effect is to carry null attributes much further through
the pipeline than previously.
2018-04-05 14:15:19 -07:00
Eric Fischer ebb0334ef0 Use the standard JSON writer for tilestats
With a lot of weird special cases to keep the formatting
exactly like it accidentally was before
2018-03-16 15:19:23 -07:00
Eric Fischer 35aa291aac Restore accidentally deleted test 2018-03-15 12:35:59 -07:00
Eric Fischer 71df4d3e81 Always include tile and layer in tippecanoe-decode, fixing corrupt JSON. 2018-03-14 16:35:59 -07:00
Eric Fischer e4e39289a2 Fix the raw-tiles test and make it a little less fragile 2018-02-26 11:32:18 -08:00
Eric Fischer 2e5b597ca1 Add --coalesce-fraction-as-needed and --coalesce-densest-as-needed 2018-02-06 14:39:05 -08:00
Eric Fischer 82a2b5dfdc Add CSV test and documentation 2017-12-13 11:32:19 -08:00
Eric Fischer afb5cece96 Verify that CSV input is encoded as UTF-8 2017-12-06 13:32:44 -08:00
Eric Fischer f3e6afa4e9 Basic geographic CSV parsing 2017-12-05 17:18:19 -08:00
Eric Fischer e4743d9527 Test successful and failed attempts to write to existing tilesets 2017-11-30 15:56:35 -08:00
Eric Fischer 1acd771743 Factor out code for enumerating the tiles in a directory 2017-11-30 15:37:46 -08:00
Eric Fischer ba1c343204 Make --allow-existing work on dirs; trim .geojson from layer names 2017-11-29 16:24:48 -08:00
Eric Fischer ab86846d4b Merge branch 'master' into json-join 2017-11-17 15:30:14 -08:00
Eric Fischer 3f54a70459 Add a test for joining with tippecanoe-json-tool 2017-11-17 14:05:37 -08:00