diegoripley/tippecanoe
1
0
Fork 0
mirror of https://github.com/felt/tippecanoe.git synced 2026-10-03 17:05:41 +02:00
Code Issues Packages Projects Releases Wiki Activity

Run prefilter/postfilter shells via single-threaded worker processes

Browse Source 
Forking and waitpid()-ing for the -C/-c shell filters used to happen
directly in the multi-threaded tiling threads, which made fork() expensive
(full COW of the parent's address space) and unsafe (other threads could
hold libc/malloc locks at fork time). Now we pre-fork a small pool of
single-threaded worker processes once, before any tiling or reader threads
are spawned, and the tiling threads ask a worker to fork+exec the shell
child on their behalf, receiving the input/output pipe fds back via
SCM_RIGHTS. The fork() and waitpid() happen entirely inside the
single-threaded worker, where they are cheap and only block that worker.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Erica Fischer
2026-05-30 14:50:37 -07:00
co-authored by Cursor
parent cb6cacef15
commit 4f4214ee23
4 changed files with 443 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plugin.hpp
+14 -1
View File
@@ -1,4 +1,17 @@
struct key_pool;
// Identifies which worker process is currently running a filter on this
// thread's behalf. Returned by setup_filter and consumed by wait_filter.
struct filter_handle {
ssize_t worker_idx = -1;
};
std::vector<mvt_layer> filter_layers(const char *filter, std::vector<mvt_layer> &layer, unsigned z, unsigned x, unsigned y, std::vector<std::map<std::string, layermap_entry>> *layermaps, size_t tiling_seg, std::vector<std::vector<std::string>> *layer_unmaps, int extent);
void setup_filter(const char *filter, int *write_to, int *read_from, pid_t *pid, unsigned z, unsigned x, unsigned y);
void setup_filter(const char *filter, int *write_to, int *read_from, filter_handle *handle, unsigned z, unsigned x, unsigned y);
void wait_filter(filter_handle handle);
serial_feature parse_feature(json_pull *jp, int z, unsigned x, unsigned y, std::vector<std::map<std::string, layermap_entry>> *layermaps, size_t tiling_seg, std::vector<std::vector<std::string>> *layer_unmaps, bool filters, key_pool &key_pool);
// Pre-fork the pool of single-threaded filter worker processes used by
// setup_filter / wait_filter. Must be called from a single-threaded context
// (i.e. before any reader or tiling threads are spawned).
void filter_workers_init(size_t n);