Commit Graph
100 Commits
Author SHA1 Message Date
Eric Fischer aa7191b1ee Also test large integers. Work around an apparent bug in strtoull. 2017-11-09 12:52:38 -08:00
Eric Fischer 87ce5b9310 Be more careful about checking for overflow when parsing numbers 2017-11-09 12:11:07 -08:00
Eric Fischer 5a68886b4d Merge pull request #486 from mapbox/surrogate-pairs
Support UTF-16 surrogate pairs in JSON and arbitrarily long lines in CSVs
2017-11-06 10:45:53 -08:00
Eric Fischer 12a1258797 Also remove the point styling script since CartoCSS is long deprecated 2017-11-03 16:59:45 -07:00
Eric Fischer 0585742a20 Remove unused constant and out-of-date comment 2017-11-03 16:51:45 -07:00
Eric Fischer 0801a9324b Silence the projection warning if you asked for --quiet 2017-11-03 15:25:14 -07:00
Eric Fischer 8ae9a8b665 Merge pull request #483 from mapbox/docker-test
Test inside docker on travis
2017-11-03 15:22:04 -07:00
Eric Fischer e39db074fe Bump version number 2017-11-03 15:19:43 -07:00
Eric Fischer 1960eb8dae Follow JSON rules for what looks like a number in a CSV 2017-11-03 15:17:09 -07:00
Eric Fischer 6467a5b70d Fix the arbitrary line length limit 2017-11-03 15:15:46 -07:00
Eric Fischer 21042a7308 Move CSV code into its own file 2017-11-03 15:15:28 -07:00
Eric Fischer c217a77b0a Add UTF-8 and UTF-16 emoji to a test 2017-11-03 15:13:08 -07:00
Eric Fischer 621cf72e5a Handle surrogate pairs in JSON strings 2017-11-03 15:09:00 -07:00
Eric Fischer cc734c8709 Merge pull request #485 from mapbox/milo-bounds
Port fix for array index out of bounds in decimal conversion
2017-11-03 10:10:12 -07:00
Eric Fischer 41c026796d Port fix for array index out of bounds in decimal conversion
https://github.com/miloyip/dtoa-benchmark/issues/7

commit fe550f38669fe0f488926c1ef0feb6c101f586d6
Author: Eli Fidler <efidler@topologyinc.com>
Date:   Tue May 31 11:51:37 2016 -0400

    avoid array index out-of-bounds

    UBSAN gave "runtime error: index 13 out of bounds for type 'const uint32_t [10]'"
2017-11-02 16:42:29 -07:00
Eric Fischer dc42c7ccc2 Merge pull request #481 from mapbox/crashers
Be more careful about corrupt mbtiles files
2017-10-30 15:16:07 -07:00
Eric Fischer faf40658a6 Bump version number 2017-10-30 13:53:54 -07:00
Eric Fischer 93a325605c Guard against impossible tile coordinates when decoding 2017-10-30 13:24:31 -07:00
Eric Fischer e2b9b96ba8 Detect impossible zoom levels in mbtiles when decoding 2017-10-30 12:55:22 -07:00
Eric Fischer a3e95db0c3 Guard against decoding tiles with an impossible extent 2017-10-30 12:48:55 -07:00
Eric Fischer dfbb13e7db Guard against impossible zoom level 2017-10-27 17:40:39 -07:00
Eric Fischer d13e08c9b5 Guard against null keys and values in tileset metadata 2017-10-27 17:38:07 -07:00
Eric Fischer 4c7de922ce Merge pull request #480 from mapbox/join-ints
Make sure to encode tile-joined integers as ints, not doubles
2017-10-27 17:31:00 -07:00
Eric Fischer 197d36bdc3 Make sure to encode tile-joined integers as ints, not doubles 2017-10-27 12:43:23 -07:00
Eric Fischer 68e4f312f8 Merge pull request #479 from mapbox/rename-layer
Add tile-join option to rename layers
2017-10-27 12:21:52 -07:00
Eric Fischer 18a5300e87 Add tile-join option to rename layers 2017-10-27 11:04:06 -07:00
Eric Fischer cd47a398fa Merge pull request #477 from mapbox/empty-attribute-keys
Fix error when parsing attributes with empty-string keys
2017-10-13 12:32:32 -07:00
Eric Fischer ee6da93494 Fix error when parsing attributes with empty-string keys 2017-10-13 11:22:15 -07:00
Eric Fischer 42a56b1ad3 Merge pull request #471 from mapbox/coalesce-smallest
Experiment with coalescing features to reduce tile size
2017-10-10 11:33:29 -07:00
Eric Fischer 91e469aac2 Merge any leftover feature remnants onto some feature.
Add a test for coalescing small features onto large ones.
2017-10-09 17:37:21 -07:00
Eric Fischer 748ef3b1d5 Add a tippecanoe-decode option to report tile size and feature count 2017-10-09 17:05:29 -07:00
Eric Fischer ecadd779c9 Don't try to coalesce features of different types 2017-10-09 16:26:55 -07:00
Eric Fischer 8ae1ec5379 Distribute coalescing error more evenly 2017-10-06 18:06:37 -07:00
Eric Fischer be922702ce Experiment with coalescing features to reduce tile size 2017-10-06 18:01:08 -07:00
Eric Fischer e000bcc261 Merge pull request #463 from mapbox/stringpool
Limit the depth of the search in the string pool.
2017-09-08 10:16:19 -07:00
Eric Fischer a2d12f178f Reorder and reword documentation 2017-09-07 15:34:55 -07:00
Eric Fischer 84a6aa6d73 Use std::isinf() and std::isnan() instead of the C versions
According to https://github.com/mapbox/tippecanoe/issues/464 this
is necessary for g++-5 on Linux
2017-09-07 14:40:50 -07:00
Eric Fischer c3d23675d1 Also fix arithmetic overflow when reading a Geobuf GeometryCollection 2017-09-07 13:48:09 -07:00
Eric Fischer c77c2a2b1e Fix compiler complaints about arithmetic overflow in hashing 2017-09-07 12:04:32 -07:00
Eric Fischer 93920d06e1 Slightly lower average traversal using a hash for comparison 2017-09-06 17:10:10 -07:00
Eric Fischer 764a8ac17c Use 32-bit offsets in the string pool search tree to reduce its size 2017-09-06 15:26:19 -07:00
Eric Fischer 0f8b32c69f Limit the depth of the search in the string pool.
Use a better comparison function. (Attributes often have a common
prefix but rarely have a common suffix.)
2017-09-06 13:06:08 -07:00
Eric Fischer 2518f238d4 Use tippecanoe instead of tile-join in the filter example 2017-09-06 11:49:55 -07:00
Eric Fischer 7ce882c035 Merge pull request #462 from mapbox/gl-filters
Add feature attribute filters (as in the GL style spec) to tile-join
2017-09-06 11:20:32 -07:00
Eric Fischer c9e4275217 Do fewer tests in the address sanitizer build 2017-09-05 14:02:43 -07:00
Eric Fischer bdd95af8b2 Be careful not to generate empty layers in tile-join.
Add a test of filtering during tile-join.
Bump version number.
2017-09-05 13:33:10 -07:00
Eric Fischer d349b7700d Add the tests for filtering for feature type 2017-09-05 12:17:23 -07:00
Eric Fischer 7b71f41e43 Fix compiler warnings 2017-09-01 17:36:43 -07:00
Eric Fischer 86ff221663 I think this should fix the access to already-freed memory
Also drop the reference to the JSON parser when disconnecting
a JSON object from the parse tree.
2017-09-01 17:22:05 -07:00
Eric Fischer 681907e88d Add the feature ID filter tests 2017-09-01 16:53:19 -07:00
Eric Fischer 8d502eb805 Fix memory leak 2017-09-01 16:41:01 -07:00
Eric Fischer c232e565db Working on testing against Mapbox GL JS filter tests 2017-09-01 16:27:27 -07:00
Eric Fischer 0fd4454129 Allow filter expressions during tippecanoe as well as during tile-join 2017-09-01 11:51:12 -07:00
Eric Fischer dbb789dadc Merge branch 'master' into gl-filters 2017-09-01 10:29:08 -07:00
Eric Fischer dd07511193 Merge pull request #460 from mapbox/geobuf
Add input support for Geobuf format
2017-09-01 10:28:22 -07:00
Eric Fischer 9a21c04f06 Bump version number; fix node version that I accidentally reverted 2017-08-31 14:17:18 -07:00
Eric Fischer 223d837736 Multithreaded geobuf feature parsing 2017-08-31 13:54:04 -07:00
Eric Fischer 12d744e961 Try specifying a newer version of Node 2017-08-30 16:03:00 -07:00
Eric Fischer 6384b6a49a Fix #include path and json2geobuf path 2017-08-30 15:45:37 -07:00
Eric Fischer a5b1378d1a Minimize external dependencies for Geobuf testing 2017-08-30 15:32:36 -07:00
Eric Fischer 076dfcdfeb Work around differences of opinion about the range of feature IDs
Geobuf uses signed 64-bit ints
Vector tile spec uses unsigned 64-bit ints
Geobuf silently corrupts IDs through use of floating point
2017-08-30 14:34:59 -07:00
Eric Fischer f65faaf3da Support GeometryCollections in geobuf 2017-08-30 14:10:35 -07:00
Eric Fischer 5c5fbe2617 Support top-level geometries 2017-08-30 14:01:09 -07:00
Eric Fischer 5665d08745 Support per-feature minzoom and maxzoom in geobuf 2017-08-30 11:44:57 -07:00
Eric Fischer 607ea6c643 Lower the precision of all test input coordinates to 6 decimal digits 2017-08-30 10:31:51 -07:00
Eric Fischer 15a48e67d6 Canonicalize numbers in stringified attributes so geobuf can match 2017-08-29 17:40:57 -07:00
Eric Fischer caac717b17 Add another missing #include 2017-08-29 17:25:36 -07:00
Eric Fischer 8b6a51e20b Fix wildcarding for geobuf test invocation 2017-08-29 16:15:22 -07:00
Eric Fischer 4d6ebaa088 Fix nulls and negative integers in geobuf parsing 2017-08-29 16:04:50 -07:00
Eric Fischer c65a388597 Add missing #include 2017-08-29 15:58:02 -07:00
Eric Fischer 12784de211 Produce "nan" and "inf" strings instead of aborting 2017-08-29 15:54:25 -07:00
Eric Fischer 228567364f Remove the memmove I accidentally left in, corrupting small numbers 2017-08-29 15:43:16 -07:00
Eric Fischer cef6b022dd Fix the typo I made in the conversion to std::string 2017-08-29 14:58:16 -07:00
Eric Fischer 0d56d1bf38 Add missing #include 2017-08-29 14:49:24 -07:00
Eric Fischer f9a007e8c3 Use milo dtoa for consistent string representation of numbers 2017-08-29 14:44:34 -07:00
Eric Fischer 3840176d5c Convert to output to std::string 2017-08-29 14:26:20 -07:00
Eric Fischer 5f09ccae88 Add namespace; fix warning 2017-08-29 13:31:59 -07:00
Eric Fischer 18b2a2c39c Milo dtoa from https://github.com/miloyip/dtoa-benchmark 2017-08-29 12:51:59 -07:00
Eric Fischer 4b66aa828d Geobuf testing exposes a bug in type coercion of 0.0 to boolean 2017-08-29 11:43:39 -07:00
Eric Fischer 1581b79a3e Forgot to implement "in" and "!in" 2017-08-28 14:35:07 -07:00
Eric Fischer 754cbdc634 Add short circuits; warnings for mismatched types, wrong array lengths 2017-08-28 14:26:36 -07:00
Eric Fischer 4e1611eec9 Fix boolean comparisions 2017-08-28 14:08:05 -07:00
Eric Fischer 8d7f8af1d9 Make indent 2017-08-28 14:01:57 -07:00
Eric Fischer b7df68c164 Remember how to keep a parse tree around after the parser is closed 2017-08-28 13:56:52 -07:00
Eric Fischer 5878213516 Simplify the filter format, and actually run the filter. 2017-08-28 13:42:14 -07:00
Eric Fischer 7be21f6046 First (untested) pass at handling GL Style Spec filters 2017-08-28 13:26:11 -07:00
Eric Fischer 7c031a9796 Memory-map geobuf input instead of reading it into a temporary string 2017-08-28 11:30:30 -07:00
Eric Fischer 5943c82457 Move file-format-neutral code out of JSON-specific source file 2017-08-28 11:10:57 -07:00
Eric Fischer 8d09f0769e Fix multipolygon and feature ID parsing 2017-08-25 17:40:08 -07:00
Eric Fischer ad89fb88fb Most of the way through geometry decoding 2017-08-25 16:36:53 -07:00
Eric Fischer b98bf6e8c7 Get attribute value decoding working 2017-08-25 15:46:32 -07:00
Eric Fischer 75be013059 Start on geobuf parsing 2017-08-25 14:56:03 -07:00
Eric Fischer e7ee83f27b Move attribute type coercion out of parsing and into serialization 2017-08-24 17:27:30 -07:00
Eric Fischer f4818ffb07 Move attribute include/exclude logic into serialization 2017-08-24 17:10:15 -07:00
Eric Fischer 34b1b215f4 Move tilestats management out of parsing and into serialization 2017-08-24 16:30:01 -07:00
Eric Fischer ed8fbd0236 Split more serialization details out from being parsing parameters 2017-08-24 15:57:33 -07:00
Eric Fischer b114e22d39 Factor out geometry fixups 2017-08-23 12:48:45 -07:00
Eric Fischer 6caf20b9c8 Put the pieces back together 2017-08-23 11:43:48 -07:00
Eric Fischer 6cea2d5db6 Progress on factoring out serialization state into a single object 2017-08-22 18:10:52 -07:00
Eric Fischer 235dbf57af More progress on splitting apart parsing and serialization 2017-08-22 16:51:11 -07:00