Skip to content

fix: unify workflow step execution with direct method run#964

Merged
stack72 merged 1 commit intomainfrom
fix/947-workflow-method-execution-parity
Mar 30, 2026
Merged

fix: unify workflow step execution with direct method run#964
stack72 merged 1 commit intomainfrom
fix/947-workflow-method-execution-parity

Conversation

@stack72
Copy link
Copy Markdown
Contributor

@stack72 stack72 commented Mar 30, 2026

Summary

Closes #947. Workflow step execution and direct swamp model method run used different code paths to invoke model methods. This caused features added to the CLI path (report context fields, failure reports, check control) to silently not apply to workflow-triggered runs.

This PR closes five specific divergences without introducing new abstractions — just targeted fixes to existing files:

  • Failed-execution reports: Workflow steps now produce @swamp/method-summary reports on failure (with executionStatus: "failed" and errorMessage), matching CLI behavior
  • swampSha in report context: Threaded through the full workflow chain so reports include the swamp git SHA
  • outputSpecs in report context: Extracted buildOutputSpecs to a shared domain module and call it in both paths
  • Check skip options: Added --skip-checks, --skip-check, --skip-check-label flags to workflow run, threaded through to MethodContext
  • Layer violation fix: Moved buildOutputSpecs from libswamp to domain layer (output_spec_builder.ts)

User Impact

Before After
Workflow reports missing swampSha and outputSpecs Identical report context in both CLI and workflow paths
No reports on workflow step failure Failed steps produce @swamp/method-summary with error details
No way to skip pre-flight checks in workflows --skip-checks, --skip-check <name>, --skip-check-label <label> flags on workflow run

Test Plan

Automated

  • Unit tests: 3768 passed, 0 failed
    • 4 new output_spec_builder_test.ts tests (empty/resource/file/both specs)
    • 1 new execution_service_test.ts test (check skip + swampSha context propagation)
    • All 13 existing run_test.ts tests pass (buildOutputSpecs extraction didn't break CLI path)
    • All 30 existing execution_service_test.ts tests pass
  • Formatting: deno fmt --check clean
  • Linting: deno lint clean
  • Type checking: deno check clean

Manual smoke tests (compiled binary, fresh repo)

  1. Direct CLI model method run sync — baseline works, reports include swampSha + outputSpecs
  2. Workflow sync step — reports now include swampSha + outputSpecs (previously missing) ✅
  3. Workflow with failing step (allowFailure: true) — @swamp/method-summary report runs with status: "failed" and error message (previously no reports on failure) ✅
  4. workflow run --skip-check-label live — correctly skips checks by label (previously not possible) ✅
  5. workflow run --skip-checks — correctly skips all checks (previously not possible) ✅
  6. workflow run --skip-check label-not-banned — correctly skips specific check by name (previously not possible) ✅

Out of scope

  • Env var detection (detectEnvVarUsageInDefinition) in workflow path — workflow authors control their expression context
  • Per-step check skip in workflow YAML — tracked as follow-up

🤖 Generated with Claude Code

Workflow step execution (`execution_service.ts:executeModelMethod`) and direct
CLI execution (`modelMethodRun`) both call `executionService.executeWorkflow()`
but diverged in context population, report handling, and check control. This
caused features added to the CLI path to silently not apply to workflow-triggered
method runs.

Closes the five specific divergences:

1. **Failed-execution reports**: Workflow steps now run `@swamp/method-summary`
   reports on failure (with `executionStatus: "failed"` and `errorMessage`),
   matching the CLI path behavior.

2. **`swampSha` in report context**: Threaded through `WorkflowRunInput` →
   `StepExecutionContext` → `MethodReportContext` so workflow-triggered reports
   include the swamp git SHA.

3. **`outputSpecs` in report context**: Extracted `buildOutputSpecs` to a shared
   domain module (`output_spec_builder.ts`) and call it in both paths, so
   workflow reports include model output spec schemas.

4. **Check skip options**: Added `--skip-checks`, `--skip-check <name>`, and
   `--skip-check-label <label>` flags to `workflow run`. Options are threaded
   through `WorkflowRunInput` → `StepExecutionContext` → `MethodContext`.

5. **`buildOutputSpecs` layer violation**: Extracted from the libswamp layer
   (`src/libswamp/models/run.ts`) to the domain layer
   (`src/domain/models/output_spec_builder.ts`) so both the CLI and workflow
   paths can use it without violating dependency direction.

Fixes #947

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLI UX Review

Blocking

None.

Suggestions

None.

Verdict

PASS — The three new flags (--skip-checks, --skip-check <name>, --skip-check-label <label>) added to workflow run are identical in name, description, and behavior to the existing flags on model method run, which is exactly what you want for parity. Help text is clear, flags use collect correctly, JSON output is unaffected (these are execution control flags, not output fields), and both log and JSON output modes are already handled by the existing renderer. No UX regressions visible.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial Review

Critical / High

None found.

Medium

  1. Degraded zodToJsonSchema fallback in src/domain/models/output_spec_builder.ts:28-33

    The new zodToJsonSchema function catches conversion failures and falls back to { type: "object" }. The original in src/libswamp/types/schema_helpers.ts:175-181 falls back to a comprehensive manualZodToJsonSchema that handles strings, numbers, booleans, arrays, records, enums, literals, and unknowns. If a model uses z.record(z.unknown()) or z.array(z.string()) as a resource schema — types known to crash z.toJSONSchema() per the comment on line 102 of schema_helpers.ts — the new code will silently produce { type: "object" } instead of an accurate schema representation. This affects report context data integrity for consumers that rely on outputSpecs[].schema.

    Breaking example: A model with resources: { data: { schema: z.record(z.unknown()), ... } } will produce { type: "object" } from the workflow path but { type: "object", additionalProperties: {} } from the CLI path (which still uses the old schema_helpers.ts zodToJsonSchema). Report consumers comparing CLI vs workflow output specs will see different schemas.

    Suggested fix: Import and reuse zodToJsonSchema from ../../libswamp/types/schema_helpers.ts instead of duplicating it with a weaker fallback — or move the comprehensive version to the domain layer alongside buildOutputSpecs.

  2. Ratchet bump to KNOWN_MUTUAL_DEPENDENCIES = 11 in integration/architecture_boundary_test.ts:134

    The PR description states it fixes a layer violation by moving buildOutputSpecs from libswamp to domain, but it introduces a new mutual dependency between bounded contexts in the process (bumping the ratchet from 10 to 11). The PR description mentions "Layer violation fix" but the net effect is trading one cross-boundary dependency for another. Worth confirming the new cycle is the intended domain/models → domain/reports (for OutputSpecInfo type import) and not something unintentional.

Low

  1. reportRegistry.getAll().length > 0 called twice in the error path (src/domain/workflows/execution_service.ts:756)getAll() builds a fresh array on each call. Not a correctness issue, but on the success path (line 598) it's also called this way. Minor inefficiency if the registry is large.

  2. No model-scope reports on failure (src/domain/workflows/execution_service.ts:826) — The success path runs both method-scope and model-scope reports (lines 700-725). The failure path only runs method-scope reports. This matches the CLI failure path in run.ts:488 which also only runs method-scope, so it's consistent — but worth noting that model-scope reports never see failures in either path.

Verdict

PASS — The changes are structurally sound, properly thread context through the full workflow chain, and maintain consistency with the existing CLI path. The zodToJsonSchema fallback divergence (Medium #1) is the most actionable finding — it creates a subtle behavioral difference between the two paths that this PR is specifically trying to unify. The architecture ratchet bump deserves a quick confirmation. No blocking issues found.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Blocking Issues

None — this PR is well-structured and addresses a real parity gap between the CLI and workflow execution paths.

Suggestions

  1. zodToJsonSchema fallback divergence (src/domain/models/output_spec_builder.ts:28-33): The new zodToJsonSchema falls back to { type: "object" } on failure, while the original in src/libswamp/types/schema_helpers.ts:175-181 falls back to a comprehensive manualZodToJsonSchema that handles records, arrays, enums, literals, etc. This means workflow-path reports will get degraded schema info when z.toJSONSchema() fails (e.g., z.record(z.unknown())). Consider importing and reusing the existing fallback, or extracting the manual converter to the domain layer alongside buildOutputSpecs.

  2. New models ↔ reports mutual dependency (ratchet 10→11): The extraction of buildOutputSpecs into src/domain/models/ creates a new circular dependency because it imports OutputSpecInfo from src/domain/reports/report_context.ts. One option to avoid this would be to define OutputSpecInfo in a shared types module or in the models bounded context (since it describes model output structure). Not blocking since the ratchet mechanism explicitly tracks this, but worth considering as a follow-up to keep the dependency count from growing.

  3. Large duplicated report execution block (src/domain/workflows/execution_service.ts:754-848): The ~95-line block for running reports on failed executions closely mirrors the success-path report block (~30 lines above). Consider extracting a shared helper to reduce duplication and ensure both paths stay in sync. Not blocking since the code is correct and tested.

@stack72 stack72 merged commit d90ab90 into main Mar 30, 2026
10 checks passed
@stack72 stack72 deleted the fix/947-workflow-method-execution-parity branch March 30, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workflow step execution bypasses modelMethodRun, diverging from direct method execution

1 participant