Improve tiling speed (#195)

* Add a way to run tippecanoe single-threaded for profiling

* Do less work when the tilestats sample values list is already full

* Save a copy when retrieving the attribute key

* Fewer atomic operations

* Move string hashing from mbtiles to text

* Only do approximate attribute deduplication when writing tiles

* Feature dropping tests are sensitive to exact tile size

* All tile creators now create a string pool for the tile

* Features clipped away to nothing should not participate in that tile

* Revert "Only do approximate attribute deduplication when writing tiles"

This reverts commit c42b34b498.

* Also revert the related test changes

* Revert "Revert "Only do approximate attribute deduplication when writing tiles""

This reverts commit 18509876c3.

* Be more specific about the string hash function

* Use fnv1a instead of std::hash for everything

* Reduce the chance of hash collisions

* Stick a hash search on the front of the tree search in addpool

* Eliminate repeated hashing of the same string

* Switch instead of ifs in json parsing

* A few more cases to populate the hash in addpool

* Store the hash in the tree instead of recalculating

* Add explanatory comment for mysterious argument

* Fewer copies in attribute stringification

* Clean up ancient weirdness in JSON attribute stringification

* More serial_val cleanup

* Pass a serial_feature to rewrite instead of many broken-down arguments

* Get rid of the multiple geometries within `partial`

* Revert "Pass a serial_feature to rewrite instead of many broken-down arguments"

This reverts commit 6f4ab9b725.

* Goodbye, struct coalesce

* Revert "Features clipped away to nothing should not participate in that tile"

This reverts commit 124462fbdc.

* Migrating fields from partial to serial_feature

* Name reconciliation between serial_feature and partial

* Replace struct partial with an augmented serial_feature

* Fix some overzealous search-and-replace renaming

* Don't say struct so often

* Remove more of the former partial construction

* Commenting and cleaning up

* Trying again to avoid all these arguments to rewrite

* I swear I did this same thing before and it didn't work.

* More rewrite cleanup

* Exile --detect-shared-borders to its own file

* Add missing headers

* More commenting and cleanup

* More comments

* Sprinkle consts around

* Emplacing and std::moving

* More cleanup

* That shouldn't have worked after a std::move

* Don't need to allocate memory to compare keys

* Reduce use of the global string pool in tiling

* Another avoidable mvt_value construction

* Further reduction to explicit string pool passing

* These reverses are no longer optimizations

* These layernames can all be references

* Don't drag an unused layername string around with every feature

* Heed a compiler warning about potential buffer overflow

* Fix my confusion about which feature's string pool is relevant

* Avoid some unnecessary allocations in attribute accumulation

* Maybe faster serialization?

* Eliminate a comparison

* Do the same here

* Save a couple of allocations when parsing numbers in JSON

* Immediately assign features to layers instead of subdividing later

* Maintain tilestats for tippecanoe:retain_points_multiplier_sequence

* Crunch out more duplicate attribute values when writing out the tile

* Do tilestats for tippecanoe:retain_points_multiplier_first too

* Shell filters need to be real threads, even if nothing else does

* Simplify tippecanoe_minzoom/maxzoom representation

* Update version and changelog

mvt.hpp

@@ -11,6 +11,7 @@
 #include <memory>
 
 #include "errors.hpp"
+#include "text.hpp"
 
 struct mvt_value;
 struct mvt_layer;
@@ -132,28 +133,28 @@ struct std::hash<mvt_value> {
 	std::size_t operator()(const mvt_value &k) const {
 		switch (k.type) {
 		case mvt_string:
-			return std::hash<std::string_view>()(k.get_string_view());
+			return fnv1a(k.c_str(), 0);
 
 		case mvt_float:
-			return std::hash<float>()(k.numeric_value.float_value);
+			return fnv1a(sizeof(float), (void *) &k.numeric_value.float_value);
 
 		case mvt_double:
-			return std::hash<double>()(k.numeric_value.double_value);
+			return fnv1a(sizeof(double), (void *) &k.numeric_value.double_value);
 
 		case mvt_int:
-			return std::hash<long long>()(k.numeric_value.int_value);
+			return fnv1a(sizeof(long long), (void *) &k.numeric_value.int_value);
 
 		case mvt_uint:
-			return std::hash<unsigned long long>()(k.numeric_value.uint_value);
+			return fnv1a(sizeof(unsigned long long), (void *) &k.numeric_value.uint_value);
 
 		case mvt_sint:
-			return std::hash<long long>()(k.numeric_value.sint_value);
+			return fnv1a(sizeof(long long), (void *) &k.numeric_value.sint_value);
 
 		case mvt_bool:
-			return std::hash<bool>()(k.numeric_value.bool_value);
+			return fnv1a(sizeof(bool), (void *) &k.numeric_value.bool_value);
 
 		case mvt_null:
-			return std::hash<int>()(k.numeric_value.null_value);
+			return fnv1a(sizeof(int), (void *) &k.numeric_value.null_value);
 
 		default:
 			fprintf(stderr, "mvt_value hash can't happen\n");
@@ -174,8 +175,8 @@ struct mvt_layer {
 	void tag(mvt_feature &feature, std::string const &key, mvt_value const &value);
 
 	// For tracking the key-value constants already used in this layer
-	std::unordered_map<std::string, size_t> key_map{};
-	std::unordered_map<mvt_value, size_t> value_map{};
+	std::vector<ssize_t> key_dedup = std::vector<ssize_t>(65536, -1);
+	std::vector<ssize_t> value_dedup = std::vector<ssize_t>(65536, -1);
 };
 
 struct mvt_tile {
@@ -190,7 +191,7 @@ int decompress(std::string const &input, std::string &output);
 int compress(std::string const &input, std::string &output, bool gz);
 int dezig(unsigned n);
 
-mvt_value stringified_to_mvt_value(int type, const char *s);
+mvt_value stringified_to_mvt_value(int type, const char *s, std::shared_ptr<std::string> const &tile_stringpool);
 long long mvt_value_to_long_long(mvt_value const &v);
 
 bool is_integer(const char *s, long long *v);