Commit Graph
32 Commits
Author SHA1 Message Date
Erica Fischer 430d8edd17 Tool for overzooming individual tiles (#121)
* Starting work on overzooming

* Factor clipping out of geometry.cpp to simplify linkage

* Pull out more geometry functions into now-badly-named clip.cpp

* Not surprisingly, there is a bug

* Found the bug

* Make indent

* Pass attributes through

* Make formatting more consistent

* Fix typos in comments

* Fix the typos better

* Add tippecanoe-overzoom to the install list

* Give overzoom a predictable exit status

* Forgot to translate to and from polygon ring closepaths

* Fix geometry collapse at z21

* Add attribute stripping; don't generate layers if they have no features

* Add docs for tippecanoe-overzoom (as it will be, not as it is)

* Change overzoom to accept input and output files as arguments

* Working on overzooming tests

* Oops

* Hook up and test the detail and buffer options, and the empty-tile case

* Also test attribute stripping

* Fix error message

* Update changelog and version
2023-08-09 11:48:16 -07:00
Erica Fischer f2127fec97 Reduce thrashing during feature ingestion and tiling (#56)
* Remove the concept of "separate metadata"

This was an extra level of attribute indirection (features point
to metadata records which point to key and value strings) which was
intended to reduce the size of temporary storage for features with
large numbers of attributes that were also spread across large numbers
of tiles at maxzoom.

For other kinds of features, the extra indirection slowed things down
instead, and, especially when maxzoom guessing was being used, many more
features were having their metadata externalized than could actually
benefit from it.

* Shave a few bytes off temporary files by using more unsigned integers

* Flush stderr after logging progress

* Revert "Shave a few bytes off temporary files by using more unsigned integers"

This reverts commit eef29084ec.

* Limit the size of the string pools and trees to fit in memory

* Add missing #include

* Move the string pool and search tree from mmap to allocated memory

* Sort in allocated rather than mapped memory too

* Also use pread instead of mapping to read in the data to sort

* When the pool gets too big, switch to just the file, not memory

* Switch string pool from memory to disk when memory is 10% full

* Add to-memory versions of the serialization functions

* Crashy work in progress toward compression

* Fix the pointer bug that was causing the crash

* Serialize features into memory rather than straight to disk

* Compress individual features in the temporary files

* Don't need to store the length of the geometry

* Remove per-feature compression; move minzoom back into the object

* Start adding a stream compressor object

* Track file position within fwrite_check()

* Add compressed stream writer functions

* Pull the writing of the serialized feature out to the callers

* Starting toward compression again from a different point

* Hook up more compression functions

* Remove unused code from the other day

* Make enough deflate calls to flush out all the buffered data

* Start on decompression

* Tile number is uncompressed, tile content is compressed

* Work on alternating compressed and uncompressed in decompression

* Closer, but still doesn't work

* Sort of works

* Works until we get to concatenated tiles

* More attempts that don't work

* One bug down

* It made a tileset!

* Handle nonzero initial zooms

* Fix seeking within compressed feature streams

* Tests pass!

* Remove debug spew

* Oops: remember to delete the temporary files so they don't hang around

* Test that fails with the current compression code

* Properly account for bytes read while closing the compressed stream

* Limit the number of warnings about bad label points

* A little more armor when closing decompression

* This time for sure

* A different, less fragile, test that failed previously with compression

* Move feature stream compression to its own file

* Remove now-unused code to deserialize from a file

* Forgot to add the new files

* Remove a little debugging logging

* Add a couple of comments on what it means to be within decompression

* Fix indentation

* Update changelog. Remove stray debugging comment.
2023-02-14 12:47:40 -08:00
Erica Fischer f1df09f147 Generate fewer duplicate label points at high zoom levels (#42) 2022-12-02 11:02:33 -08:00
Erica Fischer 5b03b18f31 Generate label points after simplification, not before. (#27)
* Generate labels points after simplification, not before.

Previously there were some cases where dropping the smallest
polygons would never reduce the number of labels.

* Also wait until after polygon cleaning to make labels

* Remove the extra newlines after the TILES ONLY COMPLETE message
2022-10-21 13:59:36 -07:00
Erica Fischer b763625862 Add an option to generate label points in place of polygons (#20)
* Add an option to generate label points in place of polygons

* Change all these places where I said "extent" but really meant "area"

* Revert "Change all these places where I said "extent" but really meant "area""

This reverts commit 403828d2f7.

* Add --order-smallest-first and --order-largest-first options

* Use Turf's center-of-mass algorithm for polygon label points

* If the label point isn't within the polygon, find one that is

* Don't choose a label point that is too close to a border

* Try a little harder to find an optimal label point

* Checkerboard which tiles labels are generated in, to reduce adjacency

* Use a label point for the general representative point for polygons

(Skipping the iteration to find one that is as far as possible from
the borders)

This makes the labels look better in many cases (like France at z1)
but unfortunately ripples into changing the sequence of polygons in
many tests, so the diff is big.

* Revert "Use a label point for the general representative point for polygons"

This reverts commit 2261adf05e.

* Checkpoint work on spiral labels

* Clip label spirals to the feature bounds

* Fix label test

* Be careful not to place spiral labels too close to borders either

* For spiral anchors, only check tile scale, not feature size

* Update test

* Only try to find a central label point for the largest ring

* In tiny polygon dust, keep the attributes of the largest feature
2022-10-13 14:21:36 -07:00
Erica FischerandRoman Karavia 182093bdc7 Improve precision of polygon area calculations (#19)
* Improve precision of get_area by using long double

* Trying to get consistent polygon area results between ARM and x86

* Calculate polygon area closer to the origin for better precision

* Update changelog

* Also exercise tiny polygon dust in the ring area test

They previously behaved differently here between x86 and ARM

* On M1 Macs, long double is just double anyway, so don't use it

* Be more careful about overflow: scale the polygon ring down into range

* Fix the bug I just introduced in the scaled area calculation

* Use only the sign from the scaled-down area calculation

Co-authored-by: Roman Karavia <47303530+romankaravia@users.noreply.github.com>
2022-10-04 10:04:34 -07:00
Erica Fischer 4ef430b913 Add an option for the Visvalingam simplification algorithm (#16)
* First attempt at porting Paul Mach's Visvalingam implementation

* Make indent

* Mostly working

* Approximate equivalence from Douglas-Peucker to Visvalingam levels

* Don't simplify away tile boundary crossing points

* Add a command-line option and test for Visvalingam simplification

* Update changelog

* Cleanup in response to review feedback

* Include <stdio.h> to fix compiler warnings
2022-09-30 12:00:07 -07:00
Eric Fischer 42ee195bc3 Add an option to keep intersection nodes from being simplified away 2019-05-13 17:39:07 -07:00
Eric Fischer 5dfbc4a281 Add --clip-bounding-box option to clip input geometry 2018-10-22 16:49:33 -07:00
Eric Fischer 59dd095607 Make file positions and lengths thread-safe 2018-05-07 14:42:49 -07:00
Eric Fischer ba62ab8596 More structure initializers 2017-11-07 15:20:17 -08:00
Eric Fischer d89d46da65 Remove unused parameters 2017-11-07 10:55:34 -08:00
Eric Fischer c79f19e3ca Merge branch 'master' into plugins 2017-08-08 11:08:10 -07:00
Eric Fischer 65c095cc2b Clean up #includes and add fields for counting attributes 2017-07-14 16:56:23 -07:00
Eric Fischer 7d47226444 Merge branch 'master' into plugins 2017-03-28 16:42:22 -07:00
Eric Fischer fa56adc530 Clean up utility functions that are also used in other projects 2017-03-16 15:06:58 -07:00
Eric Fischer 9cbbc4c68d Merge branch 'master' into plugins-merge 2017-03-02 10:45:25 -08:00
Eric Fischer 62e1018c87 Fix area of placeholders for degenerate multipolygons 2017-02-06 14:14:34 -08:00
Eric Fischer 5194a39c16 Factor out clipping to tile boundaries; test random attributes & layers 2016-12-09 10:47:03 -08:00
Eric Fischer 569825324a Factor out feature deserialization 2016-12-08 17:11:37 -08:00
Eric Fischer 0db8d9ed8b Option to snap low zooms to a stairstep grid 2016-11-28 15:45:38 -08:00
Eric Fischer 9ea29601ab Don't let line simplification reduce a polygon ring to below 3 points 2016-11-14 11:22:21 -08:00
Eric Fischer 8296190487 Remove leftovers from early versions of --detect-shared-borders 2016-10-26 15:47:26 -07:00
Eric Fischer 9160e6add5 Guard against uninitialized variables 2016-10-14 12:19:56 -07:00
Eric Fischer f7daa05515 Add a critical point where the set of rings using a polygon edge changes 2016-10-05 17:16:18 -07:00
Eric Fischer 9908db5e56 Add an option to vary the level of line and polygon simplification 2016-07-12 16:51:56 -07:00
Eric Fischer 331deca4b4 Use bitfields to further reduce data structure size 2016-05-13 15:45:33 -07:00
Eric Fischer c0edefa721 Pack structures tighter to use a little less memory 2016-05-11 14:23:39 -07:00
Eric Fischer 5cc6d97d9f Don't let polygons with nonzero area disappear during cleaning
If they collapse, turn them into placeholder squares with the
appropriate area so that there aren't visible coverage gaps.
2016-05-05 13:42:32 -07:00
Eric Fischer f1b3f6d231 Fix warnings about shadowed variables 2016-05-03 15:48:42 -07:00
Eric Fischer ee97e6c307 Whittle down tile.hpp to the things that actually related to tile.cpp 2016-04-27 15:09:06 -07:00
Eric Fischer b12413eddb Rename everything from .cc to .cpp to match other projects 2016-04-27 12:22:47 -07:00