perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes) by gsoldevila · Pull Request #5870 · colinhacks/zod

gsoldevila · 2026-04-20T15:47:12Z

Summary

Own-property count before/after this PR:

Schema	Before	After
`z.string()`	~91	~18
`z.string().min(1).max(10).email()`	~91	~18
`z.number()`	~91	~17
`z.object({})`	~91	~18

Problem

Two sources contributed to the ~91 own properties per schema instance:

The copy-loop in $constructor copies every method from _.prototype to each new instance, adding ~60 own properties.
Classic ZodType.init assigns ~30 builder methods as per-instance own properties.

V8 switches from fast-property (inline-cache) to dictionary mode at ~20–30 own properties, degrading all property access by up to 24×.

Solution: shadow-proto

Introduce a hidden internal prototype layer between the user-visible _.prototype and the parent prototype. Library methods are placed on internalProto; the copy-loop continues to target the empty _.prototype.

inst
 └── _.prototype           ← user-visible (empty by default; copy-loop targets this)
      └── internalProto    ← library space (_initProto writes here)
           └── Parent / Object.prototype

core.ts

Export $internalProto (a unique Symbol) so schemas.ts can locate each constructor's internal prototype.
Inside $constructor, create internalProto = Object.create(Parent.prototype), wire _.prototype → internalProto via Object.setPrototypeOf, and expose it as _[$internalProto].
The copy-loop is unchanged — Object.keys(_.prototype) now returns [] by default, making it a no-op unless a user explicitly adds methods to _.prototype.

classic/schemas.ts

Add _protoInitMap (WeakMap) and _initProto helper that targets inst._zod.constr[$internalProto].
All 133 per-instance builder method assignments replaced with _initProto calls.
Parse-family closures (parse, safeParse, parseAsync, safeParseAsync, encode/decode variants) remain per-instance — they capture inst and must work when detached.

mini/schemas.ts

Same _initProto infrastructure added.
The 6 builder methods in ZodMiniType (check, with, clone, brand, register, apply) moved to the internal prototype.

Key advantage: no breaking changes

The original prototype-extension contract is fully preserved:

// Still works exactly as before
z.ZodType.prototype.myHelper = function() { return "custom"; };
z.string().myHelper(); // ✓  "custom"

Because _.prototype is now empty by default, the copy-loop activates only when users add methods there — propagating them to new instances exactly as the original code did.

Both prototypes.test.ts suites (classic and mini) pass without modification.

New test: `fast-properties.test.ts`

Asserts for common schemas that:

Own-property count is < 25
Builder methods (optional, nullable, etc.) are NOT own properties
Parse methods ARE own properties (detached usage must work: const { parse } = schema; parse("x"))

All tests passed ✅

…aking changes) ## Problem Zod v4 schema instances accumulate too many own properties, pushing them past V8's fast-property threshold (~20–30) and into dictionary mode. Dictionary mode degrades property access by up to 24× (see issue colinhacks#5760). Two sources contribute: 1. The `// support prototype modifications` copy-loop in `$constructor` copies every method from `_.prototype` to each new instance — adding ~60 own props. 2. Classic `ZodType` init assigns ~30 builder methods as per-instance own props. ## Solution: shadow-proto design Introduce a hidden *internal* prototype layer between the user-visible `_.prototype` and the parent prototype chain. Library methods are placed there; the copy-loop continues to target only the user-visible `_.prototype`. ``` inst └── _.prototype ← user space (empty by default; copy-loop targets this) └── internalProto ← library space (_initProto writes here) └── Parent / Object.prototype ``` ### core.ts - Export `$internalProto` (a unique symbol) to identify each constructor's internal prototype from `schemas.ts`. - In `$constructor`, create `internalProto` for each constructor, wire `_.prototype → internalProto`, and expose it as `_[$internalProto]`. - The copy-loop is **unchanged** — but since `_.prototype` is now empty by default, it is a no-op unless a user explicitly adds methods there. ### classic/schemas.ts - Add `_protoInitMap` (WeakMap) and `_initProto` helper. - `_initProto` targets `inst._zod.constr[$internalProto]` (the internal proto), not `Object.getPrototypeOf(inst)` (the user-visible proto). - All 133 per-instance builder method assignments replaced with `_initProto` calls; parse-family closures stay per-instance for detached-usage safety. ### mini/schemas.ts - Same `_initProto` infrastructure added. - The 6 builder methods in `ZodMiniType` (`check`, `with`, `clone`, `brand`, `register`, `apply`) moved to the internal prototype. ## Key advantage over a simpler approach Unlike removing the copy-loop entirely (which breaks prototype augmentation), this design preserves the original contract: ```js // Still works — copy-loop propagates user extensions from _.prototype to instances z.ZodType.prototype.myHelper = function() { ... }; z.string().myHelper(); // ✓ ``` The copy-loop is only a no-op for *library* methods (they live on internalProto, which `Object.keys(_.prototype)` never enumerates). ## Results - `z.string()` own-property count: ~91 → ~18 (well below V8's threshold) - No breaking changes; all 3,589 tests pass including both `prototypes.test.ts` suites without modification - `z.ZodType.prototype` augmentation works exactly as before ## New test `fast-properties.test.ts` asserts: - Own-property count < 25 for common schemas - Builder methods are NOT own properties - Parse methods ARE own properties (detached usage must work) Closes: colinhacks#5760 Made-with: Cursor

…el savings) Replaces the shared-method-references patch with the more complete shadow-proto strategy: - Introduces a hidden `internalProto` layer between `_.prototype` (user space) and the parent prototype. All library methods are placed on `internalProto` via `_initProto`, making them prototype-inherited rather than own properties. - Own-property count per schema instance drops from ~91 → ~22, keeping all instances firmly in V8's fast-property mode (threshold: ~27). - Methods are shared across instances (prototype-inherited), eliminating the ~50-closure per-instance allocation from the original design. - Backward-compatible: user-added prototype extensions continue to work because `_.prototype` (user space) shadows `internalProto`. - Covers classic and mini Zod API (6 files: schemas.cjs/js + core.cjs/js + mini/schemas.cjs/js). Upstream PR: colinhacks/zod#5870 Made-with: Cursor

…263121) ## Summary Applies a memory optimization patch to Zod v4 to address significant heap cost regression introduced with the upgrade to Zod 4.x (see upstream issue [#5760](colinhacks/zod#5760)). ### Problem Zod v4 assigns every method (`parse`, `check`, `optional`, `email`, etc.) as an **own-property arrow function** on each schema instance. Because each arrow function closes over `inst`, each instance allocates ~91 unique function objects. V8 switches from fast-property mode to slow "dictionary mode" when an object accumulates more than ~27 own properties, causing an extra heap tax on top of the closure overhead. In practice this costs **~12.8 KB of heap per schema** (vs ~1.5 KB with Zod 3). ### Fix — Shadow-Proto Architecture Introduces a hidden intermediate prototype layer (`internalProto`) between `_.prototype` (the user-visible class prototype) and the parent. Library methods are placed on `internalProto` via `_initProto` (once per concrete type, lazily), so they become **inherited** rather than own properties. ``` instance → _.prototype (user space, empty by default) → internalProto (library space: 69 shared methods) → Parent ``` Key properties: - **Own-property count**: ~91 → **~22** (well below V8's dictionary-mode threshold of ~27) - **Shared methods**: all 69 builder/parse methods are prototype-inherited; instances share the same function objects (`s1.optional === s2.optional`) - **Backward-compatible**: user-added prototype extensions on `_.prototype` still shadow `internalProto` transparently - **Covers both variants**: classic and mini Zod APIs (6 files patched) Six Zod files are patched (both CJS and ESM variants): - `zod/v4/classic/schemas.cjs` / `schemas.js` — `_initProto` helper + moves all builder methods to `internalProto` - `zod/v4/mini/schemas.cjs` / `schemas.js` — same for the mini API surface - `zod/v4/core/core.cjs` / `core.js` — wires `_.prototype → internalProto`, exposes `$internalProto` symbol; copy-loop left as no-op for library methods The patch is managed by `patch-package` and lives in `patches/zod+4.3.6.patch`. An upstream PR is being prepared: [colinhacks/zod#5870](colinhacks/zod#5870). ### Result | Metric | Before (Zod v4, unpatched) | After (shadow-proto patch) | | ------------------------------- | -------------------------- | -------------------------- | | Own properties per instance | ~91 | **~22** | | V8 property storage mode | Dictionary (slow) | **Fast** | | Heap cost per `z.string()` | ~12.8 KB | **~2.5 KB** | | Shared method references | ✗ (per-instance closures) | **✓** (prototype-inherited)| | **Memory reduction** | — | **~80%** | | `z.iso.datetime().optional()` | ✅ | ✅ | | Prototype augmentation | ✅ | ✅ | | All parse/validate/chain APIs | ✅ | ✅ | > Validated by full Kibana heap snapshot comparison: ~113 MB reduction at startup. ### Patch persistence The patch is applied automatically during `yarn kbn bootstrap` (after `yarn install`) via `patch-package --error-on-fail`. The `patches/` directory is committed and version-controlled. **Removal criteria**: remove `patches/zod+4.3.6.patch` (and the `patch-package` devDependency) once the upstream Zod issue is resolved and Kibana upgrades to the patched version. ### New dependency: `patch-package` | | | |---|---| | **Purpose** | Applies and maintains the shadow-proto patch to `node_modules/zod` across `yarn install` runs. Invoked as `patch-package --error-on-fail` in the bootstrap step. | | **Justification** | The optimization requires modifying Zod's compiled JavaScript internals (`$constructor` wiring, `_initProto` helper, builder method placement). These changes cannot be applied at the TypeScript/import level, so a post-install patch is the correct mechanism. `patch-package` is the industry-standard tool for this pattern, with a simple invocation model and deterministic patch application. | | **Alternatives explored** | (1) **Custom patching script** — Kibana used a bespoke `src/dev/node_modules_patches/` mechanism in an earlier iteration of this PR; `patch-package` is strictly simpler and more maintainable. (2) **`@kbn/zod` wrapper** — the wrapper re-exports `zod/v4` but cannot intercept the compiled `$constructor` and prototype wiring needed for this optimization. (3) **Private Elastic fork/registry** — viable but significantly heavier: requires maintaining a fork, publishing to a registry, and updating consumers; disproportionate effort for a temporary vendor patch. | | **Existing dependencies** | Kibana has no existing dependency providing `patch-package`-equivalent functionality. `yarn patch` (a built-in Yarn Berry feature) is not available since Kibana uses Yarn Classic. | ### Test plan - `npx patch-package --error-on-fail` applies cleanly from scratch - `z.iso.datetime().optional()` works - `z.string().email().optional()`, `obj.pick()`, `enum.extract()` all work - `z.httpUrl()` correctly rejects `ftp://`, `file:///` URLs (only `http`/`https` accepted) - Shared references confirmed: `z.string().optional === z.string().optional` → `true` - Own-property count confirmed: `Object.getOwnPropertyNames(z.string()).length` → `22` - Memory benchmark: ~2.5 KB per `z.string()` (down from ~12.8 KB) - `node scripts/check_changes.ts` passes - Heap snapshot comparison: ~113 MB reduction at Kibana startup --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>

…ces (#263121) (#265044) # Backport This will backport the following commits from `main` to `9.4`: - [fix(zod): reduce per-schema heap cost via shared method references (#263121)](#263121)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Gerard Soldevila <[email protected]> Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>

…lastic#263121) ## Summary Applies a memory optimization patch to Zod v4 to address significant heap cost regression introduced with the upgrade to Zod 4.x (see upstream issue [elastic#5760](colinhacks/zod#5760)). ### Problem Zod v4 assigns every method (`parse`, `check`, `optional`, `email`, etc.) as an **own-property arrow function** on each schema instance. Because each arrow function closes over `inst`, each instance allocates ~91 unique function objects. V8 switches from fast-property mode to slow "dictionary mode" when an object accumulates more than ~27 own properties, causing an extra heap tax on top of the closure overhead. In practice this costs **~12.8 KB of heap per schema** (vs ~1.5 KB with Zod 3). ### Fix — Shadow-Proto Architecture Introduces a hidden intermediate prototype layer (`internalProto`) between `_.prototype` (the user-visible class prototype) and the parent. Library methods are placed on `internalProto` via `_initProto` (once per concrete type, lazily), so they become **inherited** rather than own properties. ``` instance → _.prototype (user space, empty by default) → internalProto (library space: 69 shared methods) → Parent ``` Key properties: - **Own-property count**: ~91 → **~22** (well below V8's dictionary-mode threshold of ~27) - **Shared methods**: all 69 builder/parse methods are prototype-inherited; instances share the same function objects (`s1.optional === s2.optional`) - **Backward-compatible**: user-added prototype extensions on `_.prototype` still shadow `internalProto` transparently - **Covers both variants**: classic and mini Zod APIs (6 files patched) Six Zod files are patched (both CJS and ESM variants): - `zod/v4/classic/schemas.cjs` / `schemas.js` — `_initProto` helper + moves all builder methods to `internalProto` - `zod/v4/mini/schemas.cjs` / `schemas.js` — same for the mini API surface - `zod/v4/core/core.cjs` / `core.js` — wires `_.prototype → internalProto`, exposes `$internalProto` symbol; copy-loop left as no-op for library methods The patch is managed by `patch-package` and lives in `patches/zod+4.3.6.patch`. An upstream PR is being prepared: [colinhacks/zod#5870](colinhacks/zod#5870). ### Result | Metric | Before (Zod v4, unpatched) | After (shadow-proto patch) | | ------------------------------- | -------------------------- | -------------------------- | | Own properties per instance | ~91 | **~22** | | V8 property storage mode | Dictionary (slow) | **Fast** | | Heap cost per `z.string()` | ~12.8 KB | **~2.5 KB** | | Shared method references | ✗ (per-instance closures) | **✓** (prototype-inherited)| | **Memory reduction** | — | **~80%** | | `z.iso.datetime().optional()` | ✅ | ✅ | | Prototype augmentation | ✅ | ✅ | | All parse/validate/chain APIs | ✅ | ✅ | > Validated by full Kibana heap snapshot comparison: ~113 MB reduction at startup. ### Patch persistence The patch is applied automatically during `yarn kbn bootstrap` (after `yarn install`) via `patch-package --error-on-fail`. The `patches/` directory is committed and version-controlled. **Removal criteria**: remove `patches/zod+4.3.6.patch` (and the `patch-package` devDependency) once the upstream Zod issue is resolved and Kibana upgrades to the patched version. ### New dependency: `patch-package` | | | |---|---| | **Purpose** | Applies and maintains the shadow-proto patch to `node_modules/zod` across `yarn install` runs. Invoked as `patch-package --error-on-fail` in the bootstrap step. | | **Justification** | The optimization requires modifying Zod's compiled JavaScript internals (`$constructor` wiring, `_initProto` helper, builder method placement). These changes cannot be applied at the TypeScript/import level, so a post-install patch is the correct mechanism. `patch-package` is the industry-standard tool for this pattern, with a simple invocation model and deterministic patch application. | | **Alternatives explored** | (1) **Custom patching script** — Kibana used a bespoke `src/dev/node_modules_patches/` mechanism in an earlier iteration of this PR; `patch-package` is strictly simpler and more maintainable. (2) **`@kbn/zod` wrapper** — the wrapper re-exports `zod/v4` but cannot intercept the compiled `$constructor` and prototype wiring needed for this optimization. (3) **Private Elastic fork/registry** — viable but significantly heavier: requires maintaining a fork, publishing to a registry, and updating consumers; disproportionate effort for a temporary vendor patch. | | **Existing dependencies** | Kibana has no existing dependency providing `patch-package`-equivalent functionality. `yarn patch` (a built-in Yarn Berry feature) is not available since Kibana uses Yarn Classic. | ### Test plan - `npx patch-package --error-on-fail` applies cleanly from scratch - `z.iso.datetime().optional()` works - `z.string().email().optional()`, `obj.pick()`, `enum.extract()` all work - `z.httpUrl()` correctly rejects `ftp://`, `file:///` URLs (only `http`/`https` accepted) - Shared references confirmed: `z.string().optional === z.string().optional` → `true` - Own-property count confirmed: `Object.getOwnPropertyNames(z.string()).length` → `22` - Memory benchmark: ~2.5 KB per `z.string()` (down from ~12.8 KB) - `node scripts/check_changes.ts` passes - Heap snapshot comparison: ~113 MB reduction at Kibana startup --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>

vercel Bot deployed to Preview – zod-v4 April 20, 2026 15:48 View deployment

This was referenced Apr 20, 2026

Schema instances have ~91 own properties → dictionary mode → ~24x slower property access vs Zod 3 #5760

Open

fix(zod): reduce per-schema heap cost via shared method references elastic/kibana#263121

Merged

gsoldevila mentioned this pull request Apr 22, 2026

perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes) gsoldevila/zod#2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes)#5870

perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes)#5870
gsoldevila wants to merge 1 commit intocolinhacks:mainfrom
gsoldevila:zod-memory-optimization-v2

gsoldevila commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gsoldevila commented Apr 20, 2026

Summary

Problem

Solution: shadow-proto

core.ts

classic/schemas.ts

mini/schemas.ts

Key advantage: no breaking changes

New test: fast-properties.test.ts

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

New test: `fast-properties.test.ts`