Commit Graph
558 Commits
Author SHA1 Message Date
Eric Fischer 32111af79d Merge pull request #136 from mapbox/multithread-input2
Multithreaded input
2016-01-04 17:29:25 -08:00
Eric Fischer 1d77261509 Bump version number 2016-01-04 17:20:38 -08:00
Eric Fischer 2b378ceb9f Use multiple threads to sort the features 2016-01-04 14:45:34 -08:00
Eric Fischer b39a1714c0 Merge remote-tracking branch 'origin/master' into multithread-input2 2015-12-22 17:04:32 -08:00
Eric Fischer 228a4d6bb9 Make the geometric origin a per-reader property for thread safety 2015-12-22 16:58:27 -08:00
Dane Springmeyer 76cf1f8951 Merge pull request #135 from mapbox/build-inherit
Inherit CXX,CC,CFLAGS,CXXFLAGS,LDFLAGS from env
2015-12-22 16:33:22 -08:00
Eric Fischer 2fd00413cf Clean up progress indicator and feature sequence 2015-12-22 16:29:45 -08:00
Dane Springmeyer 8e3f8b8dc3 inherit CXX,CC,CFLAGS,CXXFLAGS,LDFLAGS from env 2015-12-22 16:26:24 -08:00
Eric Fischer 02cf4d46ad Close files that were being left open 2015-12-22 15:42:51 -08:00
Eric Fischer fde3aa45de Make parallel reading a command-line option 2015-12-22 14:58:19 -08:00
Eric Fischer ca4d1beb7c Guard against trying to map an empty string pool into memory 2015-12-22 14:27:38 -08:00
Eric Fischer a8b39aa2ff Merge branch 'master' into multithread-input2 2015-12-22 14:10:21 -08:00
Eric Fischer 40ec317c36 Launch a separate thread to read each segment 2015-12-22 14:02:31 -08:00
Eric Fischer f290794f22 Bump version number 2015-12-22 10:40:13 -08:00
Eric Fischer a9ad1d99df Merge pull request #134 from mapbox/fixshards
Round the number of threads to use down to a power of 2
2015-12-22 10:35:22 -08:00
Eric Fischer edd325e338 Round the number of threads to use down to a power of 2
Since other calculations depend on this being true
2015-12-22 10:33:08 -08:00
Eric Fischer bd081c864e It tiles again! 2015-12-21 18:00:07 -08:00
Eric Fischer aaf9e87bce Reassemble the combined string pool and metadata files 2015-12-21 17:46:04 -08:00
Eric Fischer de60545da8 Include the segment number in the serialized geometry 2015-12-21 17:21:18 -08:00
Eric Fischer 1f335eec44 Get rid of a global I had forgotten about 2015-12-21 17:00:07 -08:00
Eric Fischer 2dc99698d2 Stub out enough to get it to compile 2015-12-21 16:17:47 -08:00
Dane Springmeyer fee18cc31e Add errno.h - refs #131 2015-12-20 15:13:33 -08:00
Dane Springmeyer ee35b9f7c9 fix matrix 2015-12-20 15:11:15 -08:00
Dane Springmeyer 255960b48d Add travis 2015-12-20 15:09:35 -08:00
Eric Fischer f6dfe0ace0 WIP on splitting geometry-reading state into per-thread structures 2015-12-18 17:10:20 -08:00
Eric Fischer 2159d464d0 Segment the file into input chunks. Allow commas at the top level. 2015-12-18 15:59:07 -08:00
Eric Fischer ed90c7b53a Use memory-mapped I/O for GeoJSON reading if possible
Conflicts:
	geojson.c
2015-12-18 15:20:43 -08:00
Eric Fischer dd2a4b0fc8 Pass the input line number around instead of the JSON parser itself 2015-12-18 15:12:27 -08:00
Eric Fischer c6dfae26cb Add a function to disconnect a JSON object from the parse tree 2015-12-18 12:05:22 -08:00
Eric Fischer dfcdd03b19 Factor out the JSON token reading loop
Conflicts:
	geojson.c
2015-12-18 12:04:52 -08:00
Eric Fischer 08a6607b0b Merge pull request #127 from mapbox/maxzoom
Separate the concept of basezoom for dot-dropping from maxzoom for tiling
2015-12-18 11:29:00 -08:00
Eric Fischer fd682b66a8 Mention base zoom and drop rate calculation in changelog 2015-12-18 11:28:00 -08:00
Eric Fischer 4fe1a950f5 Calculate effective basezoom and droprate, taking gamma into account 2015-12-17 17:16:04 -08:00
Eric Fischer acd4444439 Allow specifying a marker width for automatic basezoom guessing 2015-12-17 12:33:52 -08:00
Eric Fischer f9d1e87360 Heuristics for basezoom and droprate if no tiles are naturally small enough 2015-12-16 16:18:16 -08:00
Eric Fischer 38fe00c7ab Replicate the dot-gamma logic in the base/rate guessing code 2015-12-16 13:09:31 -08:00
Eric Fischer b7a411769a Don't allow <0 gamma (randomized) if basezoom or droprate not known 2015-12-16 12:33:07 -08:00
Eric Fischer 6681817243 Add an option to guess the drop rate 2015-12-16 12:13:24 -08:00
Eric Fischer eb906dfc72 Write spaces over the reading progress message at the end 2015-12-15 16:21:07 -08:00
Eric Fischer 966bee8919 Fix bounding box calculation 2015-12-15 15:55:54 -08:00
Eric Fischer 7f8eb4de83 Add an option to guess an appropriately dense base zoom level 2015-12-15 15:52:08 -08:00
Eric Fischer 8e2c20265f Try to clarify documentation 2015-12-15 12:22:19 -08:00
Eric Fischer fee8b33bc8 Fix a place where it was still passing around maxzoom as basezoom 2015-12-15 12:08:07 -08:00
Eric Fischer 76de0c82d2 Further cleanup of minzoom- and maxzoom-related variable names 2015-12-15 12:00:05 -08:00
Eric Fischer 7372a2c4bc Add basezoom parameter for dot-dropping, independent of maxzoom 2015-12-15 11:56:49 -08:00
Eric Fischer 3b8a5d42dd Merge pull request #125 from mapbox/integers
Encode numeric attributes as integers instead of floating point if possible
2015-12-14 16:17:21 -08:00
Eric Fischer 9391c82216 Print floating point attributes as integers if they are exact integers 2015-12-14 16:10:34 -08:00
Eric Fischer a60a9238e5 Use integers instead of floating point for numeric attributes if possible 2015-12-14 15:45:57 -08:00
Eric Fischer 9de2ddd23c Merge pull request #124 from mapbox/pointclip
Don't simplify "unused" movetos from points. Don't write empty geometries.
2015-12-11 12:39:33 -08:00
Eric Fischer cf279940da Don't simplify "unused" movetos from points. Don't write empty geometries. 2015-12-11 12:34:55 -08:00