Skip to content

fix: make input validation method-aware for extension models (#626)#638

Merged
stack72 merged 1 commit intomainfrom
fix/method-aware-input-validation-626
Mar 6, 2026
Merged

fix: make input validation method-aware for extension models (#626)#638
stack72 merged 1 commit intomainfrom
fix/method-aware-input-validation-626

Conversation

@stack72
Copy link
Copy Markdown
Contributor

@stack72 stack72 commented Mar 6, 2026

Summary

Fixes #626. Input validation previously enforced all required inputs unconditionally before method execution. This caused methods like delete to fail on extension models with CRUD patterns because create-time inputs (e.g., dropletName, region) weren't provided — even though delete doesn't use them.

Problem

Extension models define inputs in globalArguments with ${{ inputs.* }} expressions. The InputValidationService checked ALL required inputs before any method could run, making it impossible to run methods that don't need those inputs.

For example, a @test/droplet extension model with:

  • inputs.required: [dropletName, region, tagName, vpcName]
  • globalArguments referencing all four inputs
  • create method using context.globalArgs (needs all inputs)
  • delete method not accessing context.globalArgs (needs no inputs)

Running swamp model method run web-droplet delete would fail with input validation errors for dropletName, region, etc.

Approach: Three-Layer Fix

The fix required changes at three different layers because inputs flow through multiple stages before reaching method code:

Layer 1: CLI Validation (src/cli/commands/model_method_run.ts)

  • Scan the method's YAML arguments for ${{ inputs.* }} expression references
  • Filter the required array to only include inputs the method references
  • Type/enum validation still runs for any inputs the user provides
  • Defaults are still applied from the original schema

Layer 2: CEL Expression Evaluation (src/domain/expressions/expression_evaluation_service.ts)

  • Skip evaluating CEL expressions that reference inputs not provided by the user
  • Without this, the CEL evaluator would crash with No such key: apiToken when trying to resolve ${{ inputs.apiToken }} for a method that doesn't need it
  • Follows the same pattern as the existing vault/env runtime expression skipping

Layer 3: Runtime Guard (src/domain/models/method_execution_service.ts)

  • Wrap context.globalArgs in a Proxy that throws a clear error if a method accesses a field containing an unresolved ${{ inputs.* }} expression
  • Strip unresolved fields from globalArguments before Zod schema validation (otherwise validation fails on raw expression strings)
  • This catches the case where extension model TypeScript code tries to use context.globalArgs.dropletName but the user didn't provide that input

Supporting Code (src/domain/expressions/expression_parser.ts)

  • New extractInputReferencesFromCel() — extracts input field names from a single CEL expression string
  • New extractInputReferences() — scans a data structure for ${{ }} expressions and returns the set of inputs.* fields referenced
  • Handles both dot notation (inputs.fieldName) and bracket notation (inputs["field-name"])
  • Uses negative lookbehind to exclude cross-model references (model.foo.input.bar)

Why all three layers?

A single-layer fix was insufficient:

  1. Layer 1 alone — Doesn't help extension models where inputs flow through globalArguments → TypeScript code, not YAML method arguments
  2. Layer 1 + Layer 2 alone — Validation passes and CEL evaluation succeeds, but Zod schema validation in executeWorkflow() fails on raw ${{ inputs.* }} strings, and if that's bypassed, create silently produces data with unresolved expression strings instead of failing
  3. All three layers — Validation is method-aware, CEL gracefully skips missing inputs, and the runtime Proxy catches actual access to unresolved fields with a clear error message

Binary Testing

After compiling, tested the binary against a reproduction repo (/tmp/swamp-626-test/) with a @test/droplet extension model matching the exact scenario from issue #626:

Scenario Expected Result
swamp model method run web-droplet delete (no inputs) Exit 0 — delete doesn't use globalArgs Exit 0
swamp model method run web-droplet create (no inputs) Exit 1 — clear error about missing inputs Exit 1: Missing required input(s): dropletName (needed by globalArguments.dropletName)
swamp model method run web-droplet create --input dropletName=test --input region=nyc3 --input tagName=demo --input vpcName=my-vpc Exit 0 — data persisted correctly Exit 0, data written to .swamp/data/

Test plan

  • 7 new unit tests for extractInputReferences() in expression_parser_test.ts — dot/bracket notation, deduplication, cross-model exclusion, nested structures
  • 3 new integration tests in inputs_test.ts — unreferenced required inputs succeed, referenced required inputs still validated, globalArguments inputs don't block unrelated methods
  • Full test suite: 2733/2733 passed
  • deno check — passed
  • deno lint — passed
  • deno fmt — passed
  • deno run compile — passed
  • Compiled binary tested against reproduction repo (3 scenarios above)

🤖 Generated with Claude Code

Input validation previously enforced all `required` inputs unconditionally
before method execution. This caused methods like `delete` to fail on
extension models with CRUD patterns because create-time inputs (e.g.,
`dropletName`, `region`) weren't provided, even though `delete` doesn't
use them.

The fix uses a three-layer approach:

1. CLI validation (model_method_run.ts): scan method YAML arguments for
   `${{ inputs.* }}` references and only require inputs the method
   actually uses
2. CEL evaluation (expression_evaluation_service.ts): skip evaluating
   expressions that reference inputs not provided by the user, leaving
   raw `${{ inputs.* }}` strings for methods that don't need them
3. Runtime guard (method_execution_service.ts): wrap `context.globalArgs`
   in a Proxy that throws a clear error if a method accesses a field
   containing an unresolved input expression; also strip unresolved
   fields before Zod schema validation

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Verdict: Approved

This PR implements a well-designed three-layer fix for #626, making input validation method-aware for extension models.

Code Quality

Criteria Status
TypeScript strict mode ✅ No new any types
Named exports ✅ All new exports are named
License headers ✅ All files have AGPLv3 header
Unit tests ✅ 7 new tests in expression_parser_test.ts
Integration tests ✅ 3 new tests in inputs_test.ts
CI checks ✅ Lint, Test, Format passed
Security ✅ No vulnerabilities

Domain-Driven Design

  • Domain services (ExpressionEvaluationService, MethodExecutionService) are stateless and appropriately placed
  • Pure functions (extractInputReferences, extractInputReferencesFromCel) are well-isolated in the expression parser module
  • Changes follow existing architectural patterns

Suggestions (non-blocking)

  1. The third integration test validates success when globalArgs have unresolved inputs, but doesn't explicitly test the error case when a method accesses an unresolved globalArg. An automated test for this edge case could be added for regression protection (manual testing confirmed it works per the PR description).

  2. Consider adding a unit test for the regex handling of model.*.inputs.* patterns to ensure negative lookbehind handles both input and inputs in cross-model paths.

Highlights

  • The three-layer approach is thorough: CLI validation, CEL evaluation, and runtime Proxy guard
  • Well-documented comments explaining the "why" behind each layer
  • Clear error messages via the Proxy pattern for missing inputs
  • Comprehensive test coverage including edge cases (dot/bracket notation, deduplication, cross-model exclusion)

Great work! 🎉

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial Review

Medium

  1. Nested globalArguments bypass unresolved expression detection (method_execution_service.ts:221-232)

    The loop that identifies unresolved input expressions only checks top-level string values:

    for (const [key, value] of Object.entries(rawGlobalArgs)) {
      if (typeof value === "string" && containsExpression(value) && ...) {

    Breaking example: A model with nested globalArguments:

    globalArguments:
      config:
        apiToken: "${{ inputs.apiToken }}"

    When running a method without providing apiToken:

    • config is an object, so typeof value === "string" fails
    • The nested expression isn't detected, hasUnresolved stays false
    • Full Zod validation runs and passes (the expression is a valid string)
    • The method receives context.globalArgs.config.apiToken as the raw string "${{ inputs.apiToken }}"

    The Proxy guard (lines 138-155) also only intercepts top-level property access, so context.globalArgs.config.apiToken bypasses the check entirely.

    Impact: Methods receive raw expression strings instead of values. Most likely causes runtime failures or incorrect behavior. Could leak expression syntax to external systems (API calls, logs).

    Suggested fix: Either recursively check nested structures, or use a recursive Proxy that guards all levels.

  2. Expressions referencing both provided and unprovided inputs skip entirely (expression_evaluation_service.ts:214-229)

    If an expression references multiple inputs and ANY is missing, the entire expression is skipped:

    if (hasMissing) {
      continue;  // Skip the entire expression
    }

    Breaking example:

    globalArguments:
      combined: "${{ inputs.provided + ' ' + inputs.unprovided }}"

    With only provided given:

    • The expression is skipped (not evaluated)
    • combined retains the raw expression string
    • Method may receive/use raw expression string

    Impact: Same as issue #1 - raw expressions may reach runtime code. This is intentional per the PR design (defer to runtime Proxy), but the Proxy doesn't catch all cases (nested objects, bracket access patterns like context.globalArgs["combined"]).

Low

  1. Single-quoted bracket notation not detected (expression_parser.ts:233)

    INPUTS_BRACKET_PATTERN only matches double quotes: inputs["field"]. Single quotes inputs['field'] won't be detected. CEL typically uses double quotes, so low risk, but inconsistent with CEL's actual syntax support.

  2. Escaped quotes in input names parsed incorrectly (expression_parser.ts:233)

    Pattern [^"]+ stops at first quote. inputs["field\"name"] extracts field\ not field"name. Edge case.

  3. Unicode input names not supported (expression_parser.ts:227)

    Pattern [a-zA-Z_][a-zA-Z0-9_]* is ASCII-only. If Unicode input names are valid elsewhere in the system, they won't be detected here.

Verdict

PASS — No critical or high severity issues.

The medium issues represent edge cases (nested globalArguments) that are unlikely in typical usage. The core fix for #626 is sound: method arguments are scanned correctly, and top-level globalArguments are guarded by the Proxy. The integration tests cover the intended scenarios.

The gaps in nested object handling could cause confusing failures in complex extension models, but won't cause data loss, security vulnerabilities, or crashes in production. The failure mode (method receives raw expression string) will typically manifest as an obvious runtime error when the method tries to use the invalid value.

Consider adding a follow-up task to recursively guard nested globalArguments structures if extension models with nested configs become common.

@stack72 stack72 merged commit 0f6f4d7 into main Mar 6, 2026
6 checks passed
@stack72 stack72 deleted the fix/method-aware-input-validation-626 branch March 6, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model inputs schema validated on all methods — delete fails requiring create-time inputs

1 participant