Add point retention multiplier (#179)

* Add an option to retain N times as many points as usual at each zoom

* Tests for point multipler with specified and guessed maxzooms

* Work in progress on inverse spatial ordering

* Fix inverse spatial feature order

* --reorder was depending on a feature index that wasn't being preserved

* Separate ordering by feature_minzoom from ordering inverse-spatially

* Add a test for the inverse spatial ordering

* Store the basezoom/droprate/multiplier decisions in tileset metadata

* Progress on adding filters to tippecanoe-overzoom

* Type promotion for comparison

* Look up the attribute value for ordering

* Add test of thinning and ordering features

* Plumb tippecanoe_decisions metadata through pmtiles

* Be careful not to put infinities in JSON

* Fix accidental dropping in what is meant to preserve sparse points

* Start distinguishing true, false, and null in expressions

* Most of the type conversions

* Add boolean conversions

* Literals and conjunctions

* Add filtering to tippecanoe-overzoom

* Add a test of filtering in overzoom

* Fix boolean conjunctions

* Handle the combination of cluster size and filtering

* Rework dot dropping to reconcile density threshold and multiplier

* Revert "Rework dot dropping to reconcile density threshold and multiplier"

This reverts commit f253a66382.

* Retain points by multiplier within each tile, not in global probability

* Test that intends to verify that the multiplier is reversible

* Get the test to detect the discrepancy

* Mark the start of multiplier clusters with a magic attribute

* Add string-contains

* Add in and ni operators

* Revert "Look up the attribute value for ordering"

This reverts commit 56bc73e49a.

* Revert "Type promotion for comparison"

This reverts commit 6f3256f5af.

* Make number formatting in tippecanoe_decisions consistent

* Revert "Add a test for the inverse spatial ordering"

This reverts commit c8047de9ab.

* Revert "Separate ordering by feature_minzoom from ordering inverse-spatially"

This reverts commit 35b19a223c.

* Revert "Fix inverse spatial feature order"

This reverts commit 5978ecdb44.

* Revert "Work in progress on inverse spatial ordering"

This reverts commit fdf230f632.

* Somehow missed the tests associated with that last revert

* Round-robin assign attributes to partials from across the multiplier

* Count the multiplier separately in each layer

* Fix distribution of accumulated attribute across multiplier features

* Update changelog, version, and docs

* Add "is null" and "isnt null" expressions

* Update interpretation of FSL expressions to pass the tests

* Test to assert that polygons are unaffected by the multiplier

* Clean up and comment

* Remove accidental unused case
This commit is contained in:
Erica Fischer
2024-01-18 15:56:09 -08:00
committed by GitHub
parent e8ca6c6de3
commit 5d92a17193
56 changed files with 83462 additions and 113 deletions
+25 -1
View File
@@ -7,6 +7,8 @@
#include <stdlib.h>
#include <string.h>
#include <sqlite3.h>
#include <cmath>
#include <climits>
#include <vector>
#include <string>
#include <set>
@@ -601,6 +603,17 @@ void mbtiles_write_metadata(sqlite3 *db, const metadata &m, bool forcetable) {
sqlite3_free(sql);
}
if (m.decisions_json.size() > 0) {
sql = sqlite3_mprintf("INSERT INTO metadata (name, value) VALUES ('tippecanoe_decisions', %Q);", m.decisions_json.c_str());
if (sqlite3_exec(db, sql, NULL, NULL, &err) != SQLITE_OK) {
fprintf(stderr, "set decisions: %s\n", err);
if (!forcetable) {
exit(EXIT_SQLITE);
}
}
sqlite3_free(sql);
}
if (m.vector_layers_json.size() > 0 || m.tilestats_json.size() > 0) {
std::string json;
json_writer state(&json);
@@ -642,7 +655,7 @@ void mbtiles_write_metadata(sqlite3 *db, const metadata &m, bool forcetable) {
}
}
metadata make_metadata(const char *fname, int minzoom, int maxzoom, double minlat, double minlon, double maxlat, double maxlon, double minlat2, double minlon2, double maxlat2, double maxlon2, double midlat, double midlon, const char *attribution, std::map<std::string, layermap_entry> const &layermap, bool vector, const char *description, bool do_tilestats, std::map<std::string, std::string> const &attribute_descriptions, std::string const &program, std::string const &commandline, std::vector<strategy> const &strategies) {
metadata make_metadata(const char *fname, int minzoom, int maxzoom, double minlat, double minlon, double maxlat, double maxlon, double minlat2, double minlon2, double maxlat2, double maxlon2, double midlat, double midlon, const char *attribution, std::map<std::string, layermap_entry> const &layermap, bool vector, const char *description, bool do_tilestats, std::map<std::string, std::string> const &attribute_descriptions, std::string const &program, std::string const &commandline, std::vector<strategy> const &strategies, int basezoom, double droprate, int retain_points_multiplier) {
metadata m;
m.name = fname;
@@ -677,6 +690,17 @@ metadata make_metadata(const char *fname, int minzoom, int maxzoom, double minla
m.strategies_json = stringify_strategies(strategies);
if (isinf(droprate)) {
droprate = LLONG_MAX;
}
if (basezoom != maxzoom || droprate != 2.5 || retain_points_multiplier != 1) {
m.decisions_json = std::string("{") +
"\"basezoom\":" + milo::dtoa_milo(basezoom) + "," +
"\"droprate\":" + milo::dtoa_milo(droprate) + "," +
"\"retain_points_multiplier\":" + std::to_string(retain_points_multiplier) +
std::string("}");
}
if (vector) {
{
json_writer state(&m.vector_layers_json);