Commit Graph
12 Commits
Author SHA1 Message Date
Erica FischerandCursor 4f4214ee23 Run prefilter/postfilter shells via single-threaded worker processes
Forking and waitpid()-ing for the -C/-c shell filters used to happen
directly in the multi-threaded tiling threads, which made fork() expensive
(full COW of the parent's address space) and unsafe (other threads could
hold libc/malloc locks at fork time). Now we pre-fork a small pool of
single-threaded worker processes once, before any tiling or reader threads
are spawned, and the tiling threads ask a worker to fork+exec the shell
child on their behalf, receiving the input/output pipe fds back via
SCM_RIGHTS. The fork() and waitpid() happen entirely inside the
single-threaded worker, where they are cheap and only block that worker.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-30 14:50:37 -07:00
Erica Fischer 23667bb8eb Reduce memory consumption from attribute accumulation (#290)
* Progress on plumbing a string pool for full_keys through

* More plumbing for key_pool

* Don't keep features with identical locations as multiplier features

* Revert "Don't keep features with identical locations as multiplier features"

This reverts commit 413f0c8024.

* Adjust calculated maxzoom to account for duplicate feature locations

* Update changelog and version

* Add a test affected by the maxzoom change with duplicate locations

* Round the drop rate a little for cross-platform test consistency
2024-11-05 14:39:41 -08:00
Eric Fischer 635429cd87 Fix dangling pointer. Defer tilestats generation until tiling if filtering. 2017-08-08 13:38:48 -07:00
Eric Fischer cf6a2d3a67 Provide layer names to postfilter and read layer names back in 2016-12-21 11:47:49 -08:00
Eric Fischer c2fa8e3633 Switch the segment and layer ID based on prefilter output 2016-12-12 17:00:45 -08:00
Eric Fischer 9c0e2cdfa7 Fill out layermaps when reading the output of the prefilter 2016-12-12 16:08:08 -08:00
Eric Fischer 5e7f718afc Fill out layermaps when reading the output of the postfilter 2016-12-12 15:21:05 -08:00
Eric Fischer 0e5b513637 Start getting features (just geometry so far) back from the prefilter 2016-12-09 15:35:57 -08:00
Eric Fischer d940eb1cef Factor out filter setup from the reading and writing loops 2016-12-08 15:46:12 -08:00
Eric Fischer 112f451c66 Move JSON-writing again to keep it from requiring all plugin code 2016-12-06 17:19:22 -08:00
Eric Fischer 94bebbd276 Write GeoJSON to the filter and read (but don't parse) what comes back 2016-12-06 17:19:22 -08:00
Eric Fischer adfceed554 Factor out conversion from vector tile to GeoJSON 2016-12-06 17:19:22 -08:00