mirror of
https://github.com/felt/tippecanoe.git
synced 2026-10-02 16:35:40 +02:00
Consolidate the jsonpull comments
Comments were 25% of the added lines, and the ownership model was spelled out in five places. Collect it into one block at the top of jsonpull.h and point at it from the rest, cutting the ratio to 14% and the total by about 200 lines. Removed the duplicate explanations of the deleter dispatch, of what detach does to the back-pointers, and of "json_read returns intermediate containers, do not free them". Trimmed the comments that argued for a choice rather than described the code -- the reserve(2) / reserve(4) rationales, the string-buffer copy, the pmtiles check ordering -- to a line each, and shortened the test preambles, keeping the parts that say why a test is shaped the way it is. No code changes; the test suite is unchanged in both configurations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KNxyHKasyWrWcvre2yK4r
This commit is contained in:
Claude
+48
-129
@@ -31,70 +31,42 @@ typedef enum json_type {
|
||||
struct json_object;
|
||||
struct json_pull;
|
||||
|
||||
// json_object is non-virtual so that JSON_TRUE / JSON_FALSE / JSON_NULL
|
||||
// nodes don't have to pay for a vptr, but the typed subclasses
|
||||
// (json_number, json_string, json_array, json_hash) have non-trivial
|
||||
// destructors that need to run to free their std::vector / std::string
|
||||
// members. So json_object_ptr is given a custom empty deleter that
|
||||
// dispatches on `type` and static_casts to the right subclass before
|
||||
// `delete`. The deleter is stateless, so the unique_ptr stays one
|
||||
// pointer wide.
|
||||
// Ownership model
|
||||
//
|
||||
// Every node has exactly one owner: its parent (a json_object_ptr in the
|
||||
// parent's vector or hash entry), the parser (jp->root) for a top-level value,
|
||||
// or the caller once json_read_tree / json_disconnect hands the tree over.
|
||||
// json_read and json_hash_get return borrowed pointers, valid while the owning
|
||||
// container is intact.
|
||||
//
|
||||
// `parent` and `parser` are non-owning, so they cannot form a cycle. Detaching
|
||||
// clears every `parser` in the subtree, since the json_pull may die first, and
|
||||
// clears `parent` only on the detached root, which pointed out of the subtree.
|
||||
// Interior `parent` links stay, so a detached tree is still walkable upwards
|
||||
// and json_free / json_disconnect still work inside it -- both find a node's
|
||||
// owner through `parent`.
|
||||
//
|
||||
// json_object has no vptr; json_object_deleter switches on `type` and
|
||||
// static_casts so the right subclass destructor runs. Payloads live in those
|
||||
// subclasses rather than one wide struct, and the accessors assert on `type`
|
||||
// before downcasting.
|
||||
//
|
||||
// json_pull_ptr is a shared_ptr: a parser is created and freed once.
|
||||
|
||||
// Stateless, so json_object_ptr stays one pointer wide. See Ownership model.
|
||||
struct json_object_deleter {
|
||||
void operator()(json_object *p) const noexcept;
|
||||
};
|
||||
|
||||
// Ownership of a JSON subtree is unique: every node has a single owner,
|
||||
// which is either its parent (via a json_object_ptr in the parent's
|
||||
// vector or hash entry) or, for the root, the parser (via jp->root) or
|
||||
// the caller (after json_read_tree / json_disconnect).
|
||||
//
|
||||
// Callers receive borrowed `json_object *` views from json_read,
|
||||
// json_hash_get, etc.; those pointers stay valid as long as the owning
|
||||
// container is intact (which, for json_read results, means "until the
|
||||
// next json_read, json_free, or json_disconnect call on that subtree").
|
||||
//
|
||||
// json_pull_ptr stays a shared_ptr because the parser is created once
|
||||
// and freed once and the cost of shared_ptr there is irrelevant.
|
||||
typedef std::unique_ptr<json_object, json_object_deleter> json_object_ptr;
|
||||
typedef std::shared_ptr<json_pull> json_pull_ptr;
|
||||
|
||||
// A single key/value pair inside a JSON_HASH. The pairs are stored in
|
||||
// insertion order in a single std::vector<json_entry> on json_hash, so
|
||||
// callers can range-for over `o->entries()` with structured bindings
|
||||
// (`for (auto &[k, v] : o->entries()) ...`) while still preserving the
|
||||
// order keys appeared in the source document.
|
||||
// One key/value pair in a JSON_HASH, held in source order.
|
||||
struct json_entry {
|
||||
json_object_ptr key;
|
||||
json_object_ptr value;
|
||||
};
|
||||
|
||||
// json_object is a small base type that just records the JSON type and
|
||||
// the back-pointers to its parent and parser. The actual value payload
|
||||
// lives in a type-specific subclass (json_number, json_string, json_array,
|
||||
// json_hash), so that JSON_TRUE / JSON_FALSE / JSON_NULL nodes pay only
|
||||
// the base-class cost and a JSON_HASH does not also drag along a string
|
||||
// or a number field. Type-tagged accessor methods on the base class
|
||||
// downcast and return references to the underlying subclass storage.
|
||||
//
|
||||
// Children are owned by their parent (via std::vector<json_object_ptr>
|
||||
// inside json_array / json_hash); the raw `parent` and `parser`
|
||||
// back-pointers own nothing. json_disconnect() splices a node out of its
|
||||
// parent and walks the detached subtree clearing every `parser` pointer,
|
||||
// so the subtree can outlive the original parser. The `parent` pointers
|
||||
// within the subtree survive -- they refer to nodes the caller now owns
|
||||
// as one unit -- so a detached tree can still be walked upwards, and
|
||||
// json_free() / json_disconnect() still work on its interior nodes. Only
|
||||
// the detached root's `parent`, which pointed out of the subtree, is
|
||||
// cleared.
|
||||
//
|
||||
// json_object intentionally has no virtual functions and no virtual
|
||||
// destructor; the json_object_ptr deleter (see below in this header)
|
||||
// switches on `type` and static_casts to the correct subclass before
|
||||
// `delete`, so each subclass's destructor still runs without costing
|
||||
// a vptr per node. Dispatch on `type` is what the rest of the code
|
||||
// already does. The accessor methods assert at debug time that the
|
||||
// type matches before downcasting.
|
||||
|
||||
struct json_object {
|
||||
json_object *parent = nullptr;
|
||||
json_pull *parser = nullptr;
|
||||
@@ -108,17 +80,11 @@ struct json_object {
|
||||
: parent(p), parser(pl), type(t) {
|
||||
}
|
||||
|
||||
// Type-tagged accessors. Each one asserts that the receiver is of
|
||||
// the right kind, then downcasts to the storage in the appropriate
|
||||
// subclass. Inline so the assert and cast disappear at -O.
|
||||
inline std::string &string();
|
||||
inline const std::string &string() const;
|
||||
|
||||
// Numbers are stored in a discriminated union (double / unsigned /
|
||||
// signed) so a json_number is only 32 bytes instead of 48. The
|
||||
// large_*() accessors return 0 when the number is not currently
|
||||
// stored in that representation, matching the prior convention
|
||||
// where "0" meant "not set, fall through to the next slot".
|
||||
// large_unsigned() / large_signed() return 0 when the number is not
|
||||
// held in that representation, the convention callers already expect.
|
||||
inline double number() const;
|
||||
inline unsigned long long large_unsigned() const;
|
||||
inline long long large_signed() const;
|
||||
@@ -170,12 +136,7 @@ struct json_string : json_object {
|
||||
struct json_array : json_object {
|
||||
std::vector<json_object_ptr> array_value;
|
||||
|
||||
// Coordinate-heavy GeoJSON dominates the parse workload, and every
|
||||
// `[x, y]` (or `[x, y, z]`) pair would otherwise force the inner
|
||||
// vector through 0 -> 1 -> 2 -> 4 growths plus the matching
|
||||
// shared_ptr copies. Reserving 2 slots up front eliminates those
|
||||
// reallocations for the common case and adds only a single small
|
||||
// allocation for larger rings (which still grow geometrically).
|
||||
// 2 slots: coordinate pairs dominate the parse workload.
|
||||
json_array()
|
||||
: json_object(JSON_ARRAY) {
|
||||
array_value.reserve(2);
|
||||
@@ -189,10 +150,7 @@ struct json_array : json_object {
|
||||
struct json_hash : json_object {
|
||||
std::vector<json_entry> entries_value;
|
||||
|
||||
// Most GeoJSON property hashes have a handful of keys (type, id,
|
||||
// properties, geometry, plus a few attribute fields). Reserving 4
|
||||
// slots avoids the 0 -> 1 -> 2 -> 4 growth chain for the typical
|
||||
// case while only modestly over-allocating for one-key hashes.
|
||||
// 4 slots: the typical GeoJSON property hash.
|
||||
json_hash()
|
||||
: json_object(JSON_HASH) {
|
||||
entries_value.reserve(4);
|
||||
@@ -276,9 +234,6 @@ inline void json_object_deleter::operator()(json_object *p) const noexcept {
|
||||
if (p == nullptr) {
|
||||
return;
|
||||
}
|
||||
// Dispatch on the discriminator so the correct subclass destructor
|
||||
// runs. json_object has no virtual destructor, so a bare `delete p`
|
||||
// would skip the std::vector / std::string members of the subclass.
|
||||
switch (p->type) {
|
||||
case JSON_NUMBER:
|
||||
delete static_cast<json_number *>(p);
|
||||
@@ -293,9 +248,7 @@ inline void json_object_deleter::operator()(json_object *p) const noexcept {
|
||||
delete static_cast<json_hash *>(p);
|
||||
break;
|
||||
default:
|
||||
// JSON_TRUE / JSON_FALSE / JSON_NULL (and the parse-token
|
||||
// types, which never appear as owned nodes) are bare
|
||||
// json_objects with no extra fields.
|
||||
// JSON_TRUE / JSON_FALSE / JSON_NULL: no extra fields.
|
||||
delete p;
|
||||
break;
|
||||
}
|
||||
@@ -311,33 +264,21 @@ struct json_pull {
|
||||
ssize_t buffer_tail = 0;
|
||||
ssize_t buffer_head = 0;
|
||||
|
||||
// Stack of currently-open containers; the top is the innermost
|
||||
// container being parsed. Each frame also remembers what token is
|
||||
// expected next (an item, a comma, a key, a colon, or a value).
|
||||
// The frame's `container` is a borrowed raw pointer; actual
|
||||
// ownership of the in-progress container lives in either the
|
||||
// surrounding container's vector (for nested containers) or
|
||||
// `root` (for the outermost container).
|
||||
// Currently-open containers, innermost last, each with the token it
|
||||
// expects next. `container` is borrowed; the owner is the surrounding
|
||||
// container, or `root` for the outermost.
|
||||
struct parse_frame {
|
||||
json_object *container;
|
||||
json_type expect;
|
||||
};
|
||||
std::vector<parse_frame> container_stack;
|
||||
|
||||
// The most recently completed top-level value. The parser owns
|
||||
// it (as a unique_ptr) until either: the next top-level value
|
||||
// starts parsing (the old root is destroyed), the caller calls
|
||||
// json_read_tree (ownership is transferred out), or the caller
|
||||
// calls json_free / json_disconnect (the parser's reference is
|
||||
// dropped explicitly).
|
||||
// Most recently completed top-level value, owned until the next one
|
||||
// starts parsing or the caller takes it.
|
||||
json_object_ptr root;
|
||||
|
||||
// Scratch buffers reused across tokens so we don't reallocate per
|
||||
// number/string. number_buffer accumulates raw digits before atof();
|
||||
// string_buffer accumulates decoded bytes before being copied into
|
||||
// the final json_string. Both are cleared (capacity preserved) at
|
||||
// the start of each token, so once they grow to the largest seen
|
||||
// size they stop reallocating entirely.
|
||||
// Reused across tokens, cleared but not shrunk, so they stop
|
||||
// reallocating once grown to the largest token seen.
|
||||
std::string number_buffer;
|
||||
std::string string_buffer;
|
||||
};
|
||||
@@ -347,32 +288,20 @@ json_pull_ptr json_begin_string(const char *s);
|
||||
|
||||
json_pull_ptr json_begin(ssize_t (*read)(struct json_pull *, char *buffer, size_t n), void *source);
|
||||
|
||||
// json_end is a thin convenience that resets the caller's json_pull_ptr.
|
||||
// The parser (and any tree it still owns) is freed when the last
|
||||
// shared_ptr to it is dropped, so calling json_end is optional if the
|
||||
// json_pull_ptr will go out of scope on its own.
|
||||
// Resets the caller's pointer. Optional: the parser frees itself when the
|
||||
// last json_pull_ptr to it goes away.
|
||||
void json_end(json_pull_ptr &p);
|
||||
|
||||
typedef void (*json_separator_callback)(json_type type, json_pull *j, void *state);
|
||||
|
||||
// json_read returns a borrowed pointer to the next completed JSON node
|
||||
// in the stream. The returned pointer is valid until the next call that
|
||||
// extends or trims the parser's tree (the next json_read on the same
|
||||
// parser, a json_free on the same node, or a json_disconnect that
|
||||
// extracts the node). Returns nullptr at end of input or on error.
|
||||
//
|
||||
// For top-level values, ownership stays with the parser (via jp->root);
|
||||
// for nested values, ownership stays with the enclosing container.
|
||||
// The next completed node, borrowed. Valid until the next call that extends
|
||||
// or trims the parser's tree. nullptr at end of input or on error.
|
||||
json_object *json_read(json_pull_ptr &j);
|
||||
json_object *json_read_separators(json_pull_ptr &j, json_separator_callback cb, void *state);
|
||||
|
||||
// json_read_tree drains the next top-level value out of the parser
|
||||
// and hands ownership to the caller. After it returns, jp->root is
|
||||
// empty, every `parser` back-pointer in the subtree has been cleared
|
||||
// (as has the root's `parent`), and the caller's json_object_ptr is the
|
||||
// only thing keeping the tree alive. The returned tree can outlive the
|
||||
// json_pull it was parsed from, and stays internally navigable: the
|
||||
// `parent` pointers between its nodes are left intact.
|
||||
// Drains the next top-level value out of the parser and hands it over. The
|
||||
// returned tree can outlive the json_pull. See Ownership model for what the
|
||||
// detach does and does not clear.
|
||||
json_object_ptr json_read_tree(json_pull_ptr &j);
|
||||
|
||||
// json_free splices `o` out of its parent (if any), or clears the
|
||||
@@ -381,23 +310,13 @@ json_object_ptr json_read_tree(json_pull_ptr &j);
|
||||
// that must not be used. Safe to call with nullptr.
|
||||
void json_free(json_object *o);
|
||||
|
||||
// Splice `o` out of its parent's array/object (or out of the parser's
|
||||
// root), walk the detached subtree clearing every `parser` back-pointer
|
||||
// (and the root's `parent`, which pointed out of the subtree), and return
|
||||
// ownership of the subtree to the caller as a json_object_ptr. After this
|
||||
// returns, the parser no longer references any node in the subtree, and
|
||||
// the subtree can outlive the original parser. The `parent` pointers
|
||||
// among the subtree's own nodes are preserved, so the detached tree can
|
||||
// still be walked upwards and json_free() / json_disconnect() still work
|
||||
// on its interior nodes.
|
||||
// Splices `o` out of whatever owns it and hands it over, same detach as
|
||||
// json_read_tree. See Ownership model.
|
||||
json_object_ptr json_disconnect(json_object *o);
|
||||
|
||||
// Look up `s` in the hash `o`. Returns a borrowed pointer; ownership
|
||||
// stays with the hash. nullptr if `o` is not a hash, or `s` is absent,
|
||||
// or the hash is still being parsed and the value slot for `s` is not
|
||||
// yet filled. A JSON `null` value is *not* one of those cases: it comes
|
||||
// back as a JSON_NULL node. Accepts a json_object_ptr by reference as a
|
||||
// convenience so callers don't have to write `.get()`.
|
||||
// Borrowed value for `s`. nullptr if `o` is not a hash, `s` is absent, or the
|
||||
// hash is mid-parse with that value slot unfilled -- a JSON `null` is none of
|
||||
// those, and comes back as a JSON_NULL node.
|
||||
json_object *json_hash_get(const json_object_ptr &o, const char *s);
|
||||
json_object *json_hash_get(json_object *o, const char *s);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user