Improve tiling speed (#195)

* Add a way to run tippecanoe single-threaded for profiling

* Do less work when the tilestats sample values list is already full

* Save a copy when retrieving the attribute key

* Fewer atomic operations

* Move string hashing from mbtiles to text

* Only do approximate attribute deduplication when writing tiles

* Feature dropping tests are sensitive to exact tile size

* All tile creators now create a string pool for the tile

* Features clipped away to nothing should not participate in that tile

* Revert "Only do approximate attribute deduplication when writing tiles"

This reverts commit c42b34b498.

* Also revert the related test changes

* Revert "Revert "Only do approximate attribute deduplication when writing tiles""

This reverts commit 18509876c3.

* Be more specific about the string hash function

* Use fnv1a instead of std::hash for everything

* Reduce the chance of hash collisions

* Stick a hash search on the front of the tree search in addpool

* Eliminate repeated hashing of the same string

* Switch instead of ifs in json parsing

* A few more cases to populate the hash in addpool

* Store the hash in the tree instead of recalculating

* Add explanatory comment for mysterious argument

* Fewer copies in attribute stringification

* Clean up ancient weirdness in JSON attribute stringification

* More serial_val cleanup

* Pass a serial_feature to rewrite instead of many broken-down arguments

* Get rid of the multiple geometries within `partial`

* Revert "Pass a serial_feature to rewrite instead of many broken-down arguments"

This reverts commit 6f4ab9b725.

* Goodbye, struct coalesce

* Revert "Features clipped away to nothing should not participate in that tile"

This reverts commit 124462fbdc.

* Migrating fields from partial to serial_feature

* Name reconciliation between serial_feature and partial

* Replace struct partial with an augmented serial_feature

* Fix some overzealous search-and-replace renaming

* Don't say struct so often

* Remove more of the former partial construction

* Commenting and cleaning up

* Trying again to avoid all these arguments to rewrite

* I swear I did this same thing before and it didn't work.

* More rewrite cleanup

* Exile --detect-shared-borders to its own file

* Add missing headers

* More commenting and cleanup

* More comments

* Sprinkle consts around

* Emplacing and std::moving

* More cleanup

* That shouldn't have worked after a std::move

* Don't need to allocate memory to compare keys

* Reduce use of the global string pool in tiling

* Another avoidable mvt_value construction

* Further reduction to explicit string pool passing

* These reverses are no longer optimizations

* These layernames can all be references

* Don't drag an unused layername string around with every feature

* Heed a compiler warning about potential buffer overflow

* Fix my confusion about which feature's string pool is relevant

* Avoid some unnecessary allocations in attribute accumulation

* Maybe faster serialization?

* Eliminate a comparison

* Do the same here

* Save a couple of allocations when parsing numbers in JSON

* Immediately assign features to layers instead of subdividing later

* Maintain tilestats for tippecanoe:retain_points_multiplier_sequence

* Crunch out more duplicate attribute values when writing out the tile

* Do tilestats for tippecanoe:retain_points_multiplier_first too

* Shell filters need to be real threads, even if nothing else does

* Simplify tippecanoe_minzoom/maxzoom representation

* Update version and changelog

mbtiles.cpp

@@ -101,19 +101,6 @@ sqlite3 *mbtiles_open(char *dbname, char **argv, int forcetable) {
 	return outdb;
 }
 
-unsigned long long fnv1a(std::string const &s) {
-	// Store tiles by a hash of their contents (fnv1a 64-bit)
-	// http://www.isthe.com/chongo/tech/comp/fnv/
-	const unsigned long long fnv_offset_basis = 14695981039346656037u;
-	const unsigned long long fnv_prime = 1099511628211u;
-	unsigned long long h = fnv_offset_basis;
-	for (size_t i = 0; i < s.size(); i++) {
-		h ^= (unsigned char) s[i];
-		h *= fnv_prime;
-	}
-	return h;
-}
-
 void mbtiles_write_tile(sqlite3 *outdb, int z, int tx, int ty, const char *data, int size) {
 	std::string hash = std::to_string(fnv1a(std::string(data, size)));
 
@@ -930,10 +917,17 @@ void add_to_tilestats(std::map<std::string, tilestat> &tilestats, std::string co
 
 	auto pt = std::lower_bound(tsa->second.sample_values.begin(), tsa->second.sample_values.end(), val);
 	if (pt == tsa->second.sample_values.end() || *pt != val) {  // not found
-		tsa->second.sample_values.insert(pt, val);
-
-		if (tsa->second.sample_values.size() > max_tilestats_sample_values) {
-			tsa->second.sample_values.pop_back();
+		if (tsa->second.sample_values.size() >= max_tilestats_sample_values) {
+			if (pt == tsa->second.sample_values.end()) {
+				// insertion point would be at the end,
+				// and the list is already full, so do nothing
+			} else {
+				// bump the former last value, insert this one
+				tsa->second.sample_values.insert(pt, val);
+				tsa->second.sample_values.pop_back();
+			}
+		} else {
+			tsa->second.sample_values.insert(pt, val);
 		}
 	}