Commit Graph
100 Commits
Author SHA1 Message Date
Eric Fischer c604a51039 Don't coalesce features whose non-string-pool attributes don't match
Fortunately most attributes are in the string pool, but ones that
have passed through the prefilter are not. (Nor are attributes that
are generated by clustering or by attribute accumulation.)
2018-02-27 13:38:37 -08:00
Eric Fischer e939a85251 Make sure numeric command line arguments are actually numbers 2018-02-26 17:35:39 -08:00
Eric Fischer 45b1a1d9e0 Sum, product, concat, and comma operations work now 2018-02-26 16:47:15 -08:00
Eric Fischer 06ed9d14d8 Add an attribute for square root of cluster count 2018-02-26 12:53:43 -08:00
Eric Fischer e4e39289a2 Fix the raw-tiles test and make it a little less fragile 2018-02-26 11:32:18 -08:00
Eric Fischer a8a342f701 Send dot-dropping through the same pipeline.
The first feature in a tile can never be dropped, since there is
no previous feature to attach its properties to.

Remove the previous special case that reset the dropping counter
at the first feature within each tile proper (as opposed to the
first feature in each tile, including its buffer, which is now
the one that is guaranteed to be preserved).
2018-02-23 17:19:54 -08:00
Eric Fischer 7fe0c87806 Get attribute accmulation by concatenation working 2018-02-23 16:16:11 -08:00
Eric Fischer 27154c8f06 Cluster onto the previous feature instead of onto the next
This also allows doing the attribute and tilestats all at once
at the end of the tile instead of having to update a feature
to handle the last cluster.
2018-02-23 12:26:58 -08:00
Eric Fischer 30a277a43e Always keep the first feature so there is something to coalesce onto 2018-02-22 17:25:54 -08:00
Eric Fischer bc0edec753 Call a function that will eventually preserve the attributes 2018-02-20 17:24:15 -08:00
Eric Fischer b73e710855 Add --accumulate-attribute flag and pass the result to tiling 2018-02-20 16:19:49 -08:00
Eric Fischer 03577cf967 Merge pull request #529 from mapbox/files-left-open
Close input files that were being left open after parallel reading
2018-02-12 15:53:32 -05:00
Eric Fischer 2d625d5807 Pass through any non-ASCII characters from file names into layer names
The Docker build doesn't include proper locale support
2018-02-07 13:15:36 -08:00
Eric Fischer 2b7d64736f Set locale before running tests in Docker 2018-02-06 17:58:24 -08:00
Eric Fischer 17942b3a89 Add an option to produce only a single tile 2018-02-06 17:25:58 -08:00
Eric Fischer 7ab94d43c6 Preserve non-ASCII characters when making layernames from filenames 2018-02-06 17:22:26 -08:00
Eric Fischer 8726366a4c Remember to close the input file after CSV parsing too 2018-02-06 16:23:44 -08:00
Eric Fischer 2e5b597ca1 Add --coalesce-fraction-as-needed and --coalesce-densest-as-needed 2018-02-06 14:39:05 -08:00
Eric Fischer 8fa5a5ff87 Report distances in both feet and meters 2018-02-06 12:15:24 -08:00
Eric Fischer 8e7b22cf1b Close input files that were being left open after parallel reading 2018-02-06 12:02:23 -08:00
Eric Fischer e2a34929d1 Merge pull request #522 from carimatics/master
Fix Release file not found error when execute docker build command.
2018-01-31 07:13:57 -08:00
Eric Fischer 21b9b1993f Merge pull request #515 from mapbox/vector-bounds
Fix out-of-bounds error when no threads were needed for a zoom level
2018-01-19 10:27:20 -08:00
Eric Fischer d49606babf Fix opportunities for integer overflow 2018-01-18 12:37:15 -08:00
Eric Fischer e07a4dd51b Fix out-of-bounds error when no threads were needed for a zoom level 2018-01-18 12:12:35 -08:00
Eric Fischer eb966ae398 Merge pull request #511 from mapbox/cluster
Add an option for clustering features
2017-12-21 09:55:20 -08:00
Eric Fischer aaf08a6c55 Add an option to specify the clustering distance directly 2017-12-20 17:33:30 -08:00
Eric Fischer fc32a0e897 Add a test for the case that was getting stuck in a loop 2017-12-19 12:46:13 -08:00
Eric Fischer 01884a3bf6 Make sure the feature gap always increases when trying to reduce density 2017-12-19 12:09:30 -08:00
Eric Fischer 6524b6510e Credit dangling clustered count at the end of a tile to the last feature 2017-12-18 12:55:22 -08:00
Eric Fischer b03dabe098 Add an option for clustering features 2017-12-14 14:30:08 -08:00
Eric Fischer 5687ee041f Merge pull request #503 from mapbox/geocsv
Add input support for CSV file format
2017-12-14 10:11:07 -08:00
Eric Fischer be79d13cbc Add missing #include 2017-12-13 14:52:46 -08:00
Eric Fischer 2e32004589 Don't coalesce features that have different IDs.
Remove the 700-point limit on coalesced features, since
polygon merging is no longer a performance problem.
2017-12-13 12:05:53 -08:00
Eric Fischer 82a2b5dfdc Add CSV test and documentation 2017-12-13 11:32:19 -08:00
Eric Fischer ed0b69a3c8 Fix check for non-numeric CSV data. Use same column name code as mapnik 2017-12-07 11:28:39 -08:00
Eric Fischer ec475b3beb Check for strings vs numbers in CSV attributes 2017-12-06 15:20:01 -08:00
Eric Fischer afb5cece96 Verify that CSV input is encoded as UTF-8 2017-12-06 13:32:44 -08:00
Eric Fischer f3e6afa4e9 Basic geographic CSV parsing 2017-12-05 17:18:19 -08:00
Eric Fischer f2f76082df Merge pull request #501 from mapbox/refactor-dirtiles
Clean up duplicated code for reading tiles from a directory
2017-12-01 14:35:03 -08:00
Eric Fischer e4743d9527 Test successful and failed attempts to write to existing tilesets 2017-11-30 15:56:35 -08:00
Eric Fischer d7f44ab082 Factor out code for reading metadata.json 2017-11-30 15:49:38 -08:00
Eric Fischer 1acd771743 Factor out code for enumerating the tiles in a directory 2017-11-30 15:37:46 -08:00
Eric Fischer ad86d06cce Merge pull request #500 from mapbox/allow-existing-test
Make --allow-existing work on dirs; trim .geojson from layer names
2017-11-30 11:15:38 -08:00
Eric Fischer 1b1be9f872 Also trim .geobuf from layer names 2017-11-29 16:44:55 -08:00
Eric Fischer ba1c343204 Make --allow-existing work on dirs; trim .geojson from layer names 2017-11-29 16:24:48 -08:00
Eric Fischer ca028dd1cc Merge pull request #495 from mapbox/bare-geom-crash
Add null pointer check when checking for bare geometry. Fix coalescing of wrong features.
2017-11-22 17:53:25 -08:00
Eric Fischer 8ac7c46788 Make the same null pointer fix in jsontool.cpp as in geojson.cpp 2017-11-22 13:06:39 -08:00
Eric Fischer 2234e37883 Quadkey-order index is a better secondary key for --reorder anyway 2017-11-22 12:49:40 -08:00
Eric Fischer 8230987405 Fix a bug that could cause the wrong features to be coalesced 2017-11-22 11:54:25 -08:00
Eric Fischer 9964784e77 Add null pointer check when testing for bare geometry in GeoJSON 2017-11-21 15:01:33 -08:00
Eric Fischer ac67013930 Merge pull request #473 from mapbox/json-join
Working on tools for joining CSV data to GeoJSON
2017-11-21 10:50:01 -08:00
Eric Fischer 9a12a76855 Fix overflow when iterating through 0-length lists backwards 2017-11-17 15:47:10 -08:00
Eric Fischer 6fd4918c4d Attach coalesced-as-needed leftovers to the last feature, not the first 2017-11-17 15:43:02 -08:00
Eric Fischer ecbfad2576 Fix problem where --detect-shared-borders could simplify polygons away 2017-11-17 15:35:19 -08:00
Eric Fischer ab86846d4b Merge branch 'master' into json-join 2017-11-17 15:30:14 -08:00
Eric Fischer 4754084130 Merge pull request #492 from mapbox/rate-clarify
Clarify documentation; add --no-progress-indicator option; fix many compiler warnings
2017-11-17 14:11:22 -08:00
Eric Fischer 3f54a70459 Add a test for joining with tippecanoe-json-tool 2017-11-17 14:05:37 -08:00
Eric Fischer 2b1cba0b53 Warn during json-tool extraction if the extracted field isn't found 2017-11-17 13:52:45 -08:00
Eric Fischer f318fcc657 Turn off very verbose initializer warnings 2017-11-17 11:21:59 -08:00
Eric Fischer 621e0afc3c Merge branch 'more-warnings' into rate-clarify 2017-11-17 11:21:01 -08:00
Eric Fischer b6163dd435 Merge branch 'master' into json-join 2017-11-17 11:11:44 -08:00
Eric Fischer 76d86f7d45 Add an option to quiet the progress indicator but not warnings 2017-11-16 17:08:59 -08:00
Eric Fischer 4dec9b43ae Try to clarify documentation for point dropping 2017-11-15 13:19:42 -08:00
Eric Fischer 2d87059a04 Some more constructors 2017-11-10 13:59:57 -08:00
Eric Fischer f1c3811d97 Merge pull request #488 from mapbox/test-centos7
Test on centos7 via docker
2017-11-10 13:49:07 -08:00
Eric Fischer 069807caf4 Use a proper constructor, discover two unused fields 2017-11-10 13:06:47 -08:00
Eric Fischer dca1ebbbfb Get rid of some C99-style variable-length arrays 2017-11-09 16:54:06 -08:00
Eric Fischer 23004808e4 Merge branch 'master' into more-warnings 2017-11-09 15:13:07 -08:00
Eric Fischer 450032a62e Merge pull request #490 from mapbox/float-bounds
Be more careful about checking for overflow when parsing numbers
2017-11-09 15:01:56 -08:00
Eric Fischer 948680fbeb Exclude failing overflow test from geobuf tests 2017-11-09 14:10:29 -08:00
Eric Fischer fda0e1f28a Fix more cases of loss of precision for large magnitude integers 2017-11-09 13:56:07 -08:00
Eric Fischer aa7191b1ee Also test large integers. Work around an apparent bug in strtoull. 2017-11-09 12:52:38 -08:00
Eric Fischer 87ce5b9310 Be more careful about checking for overflow when parsing numbers 2017-11-09 12:11:07 -08:00
Eric Fischer 0152db4a20 More initializers 2017-11-07 15:57:56 -08:00
Eric Fischer 30aa422483 Missed one 2017-11-07 15:27:47 -08:00
Eric Fischer 4f974b3dc6 Less verbose initializer syntax 2017-11-07 15:25:54 -08:00
Eric Fischer ba62ab8596 More structure initializers 2017-11-07 15:20:17 -08:00
Eric Fischer 891f99f24c More structure initializers 2017-11-07 13:32:03 -08:00
Eric Fischer 1b60fdbaf0 Some structure initializers 2017-11-07 12:52:43 -08:00
Eric Fischer 20e6e80cb1 Fix signed comparison warning from g++ 2017-11-07 11:38:38 -08:00
Eric Fischer c18f9a1572 Enable more warnings 2017-11-07 11:15:04 -08:00
Eric Fischer 50910b211b Remove more unused parameters 2017-11-07 11:09:22 -08:00
Eric Fischer d89d46da65 Remove unused parameters 2017-11-07 10:55:34 -08:00
Eric Fischer 5a68886b4d Merge pull request #486 from mapbox/surrogate-pairs
Support UTF-16 surrogate pairs in JSON and arbitrarily long lines in CSVs
2017-11-06 10:45:53 -08:00
Eric Fischer 12a1258797 Also remove the point styling script since CartoCSS is long deprecated 2017-11-03 16:59:45 -07:00
Eric Fischer 0585742a20 Remove unused constant and out-of-date comment 2017-11-03 16:51:45 -07:00
Eric Fischer 0801a9324b Silence the projection warning if you asked for --quiet 2017-11-03 15:25:14 -07:00
Eric Fischer 8ae9a8b665 Merge pull request #483 from mapbox/docker-test
Test inside docker on travis
2017-11-03 15:22:04 -07:00
Eric Fischer e39db074fe Bump version number 2017-11-03 15:19:43 -07:00
Eric Fischer 1960eb8dae Follow JSON rules for what looks like a number in a CSV 2017-11-03 15:17:09 -07:00
Eric Fischer 6467a5b70d Fix the arbitrary line length limit 2017-11-03 15:15:46 -07:00
Eric Fischer 21042a7308 Move CSV code into its own file 2017-11-03 15:15:28 -07:00
Eric Fischer c217a77b0a Add UTF-8 and UTF-16 emoji to a test 2017-11-03 15:13:08 -07:00
Eric Fischer 621cf72e5a Handle surrogate pairs in JSON strings 2017-11-03 15:09:00 -07:00
Eric Fischer cc734c8709 Merge pull request #485 from mapbox/milo-bounds
Port fix for array index out of bounds in decimal conversion
2017-11-03 10:10:12 -07:00
Eric Fischer 41c026796d Port fix for array index out of bounds in decimal conversion
https://github.com/miloyip/dtoa-benchmark/issues/7

commit fe550f38669fe0f488926c1ef0feb6c101f586d6
Author: Eli Fidler <efidler@topologyinc.com>
Date:   Tue May 31 11:51:37 2016 -0400

    avoid array index out-of-bounds

    UBSAN gave "runtime error: index 13 out of bounds for type 'const uint32_t [10]'"
2017-11-02 16:42:29 -07:00
Eric Fischer dc42c7ccc2 Merge pull request #481 from mapbox/crashers
Be more careful about corrupt mbtiles files
2017-10-30 15:16:07 -07:00
Eric Fischer faf40658a6 Bump version number 2017-10-30 13:53:54 -07:00
Eric Fischer 93a325605c Guard against impossible tile coordinates when decoding 2017-10-30 13:24:31 -07:00
Eric Fischer e2b9b96ba8 Detect impossible zoom levels in mbtiles when decoding 2017-10-30 12:55:22 -07:00