fix(extensions): complete legacy layout migration on repo upgrade (swamp-club#160)#1224
fix(extensions): complete legacy layout migration on repo upgrade (swamp-club#160)#1224
Conversation
…amp-club#160) `swamp repo upgrade` was silently deleting user working-tree files when it detected extensions tracked at a legacy on-disk layout. The two-phase migration added in PR #1186 renamed gen-1 paths (`extensions/<type>/…`) to gen-2 paths, then deleted the gen-2 files expecting the user to run `swamp extension install` afterward to restore them. No warning, no prompt, no automatic follow-up — just files gone and a terse info log. The fix collapses the split between the two primitives that owned "make extension state correct." Previously `migrateExtensionLayout` knew about layouts but not about pulling, and `extensionInstall` knew about pulling but not about layouts. This gap is where the user's files fell. Now `extensionInstall` widens its "needs install" predicate to also treat legacy-layout entries as needing re-pull, installs them into the per-extension subtree, and sweeps the original legacy paths only after a successful install. `repo upgrade` invokes `extensionInstall` directly as its final step. ## User-visible behavior - `swamp repo upgrade` now completes the migration end-to-end. No manual `swamp extension install` follow-up required for the common case. - On install failure (network down, extension yanked, auth missing), legacy files are **preserved** and the error names the failed extensions with a recovery command. - `swamp extension install` now also migrates legacy-layout repos (previously a silent no-op on legacy state). This is the positive side-effect of unifying the two primitives. ## Architecture - `migrateExtensionLayout`, `migrateExtensionLayoutPhase1`, `migrateExtensionLayoutPhase2`, `pruneEmptyDirsUp`, `ExtensionLayoutMigrationSummary`, and the `extensionMigration` result field are deleted (~280 LOC). Their responsibility moves into `extensionInstall` where it naturally belongs. - `RepoUpgradeEvent` gains an `{ kind: "extensions"; event: ... }` envelope; the upgrade renderer delegates that envelope to the existing install renderer so output stays consistent with `swamp extension install` directly. - New `InstallExtensionFn` test seam on `ExtensionInstallDeps` lets unit tests exercise the success path without a real tar archive. - New `createExtensionInstallDeps(repoDir, logger)` shared factory, used by both `repo upgrade` and `extension install`. - `needsInstallOrMigration` and `sweepLegacyPaths` are exported for direct unit testing. ## Latent bug caught during verification While running an end-to-end manual test against `@swamp/aws/s3`, empty `_lib/` directories were being stranded after the sweep. Root cause: `resolveRepoDir` returns `"."` by default, which propagated through `join(".", "path") → "path"` (relative), and `cleanupEmptyParentDirs`'s `startsWith(stopAtDir)` check failed against relative paths. The factory now absolutizes `repoDir` up front so downstream filesystem helpers see absolute paths. Added a regression test asserting nested parent dirs are pruned. ## Testing - 4728 unit/integration tests pass (up from 4727; added 8 new tests) - Type-check, lint, fmt all clean - Manual end-to-end verification against `@swamp/aws/s3`: all 11 model files migrated to per-extension subtree, `filesChecksum` anchor matches original pull byte-for-byte, legacy paths (including nested `_lib/`) fully swept. - Failure-mode verification: with a lockfile referencing an extension not in the registry, all legacy files are preserved and the user sees a clear error naming the extension + recovery command. ## Notes for reviewers - `swamp extension install` behavior change (no-op → auto-migrate on legacy repos) is intentional and noted in the `swamp-repo` skill. - There is no `CHANGELOG.md` in this repo — release notes belong in the PR description. - Plan step for an integration test against a real/fake registry was skipped as out of scope; unit coverage + manual verification cover the critical paths. Filing as a follow-up would be reasonable. Closes swamp-club#160 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
There was a problem hiding this comment.
CLI UX Review
Blocking
-
swamp repo upgrade --jsonemits two JSON objects —src/presentation/renderers/repo_init.tsJsonRepoUpgradeRendererdelegatesextensionsevents toinstallHandlers, which is the JSON extension-install renderer. That renderer unconditionally callsconsole.log(JSON.stringify(e.data, null, 2))when it seescompleted. Then the upgradecompletedhandler prints a second JSON object for the upgrade result.On every
swamp repo upgrade --jsonrun the output is now two separate objects:{ "entries": [...], "installed": 0, "migrated": 0, "upToDate": 2, "failed": 0 } { "path": "...", "previousVersion": "...", ... }This is invalid JSON and breaks any script that was parsing the output (e.g.
swamp repo upgrade --json | jq .pathnow errors). Before this PRswamp repo upgrade --jsonemitted exactly one object.Suggested fix: In
JsonRepoUpgradeRenderer, suppress the install-completed output by not forwarding the innercompletedevent to the install renderer — or forward it only when there were actual installs/migrations, and fold the counts into the upgrade JSON object instead. The simplest fix is to skip thecompleteddispatch indispatchInstallEventwhen called from the upgrade JSON renderer (e.g. pass a{ skip: ['completed'] }option, or just omit thecompletedcase from the JSON renderer'sdispatchInstallEventcall).
Suggestions
-
Log mode: "All extensions up to date." appears before the upgrade summary —
src/presentation/renderers/repo_init.tsOn a healthy repo with no migrations needed,
swamp repo upgradenow prints:All extensions up to date. Repository upgraded: /path/to/repo Version: 1.0.0 → 2.0.0 ...The extension-install summary fires before the upgrade summary and lacks any contextual framing (no "Extension layout:" prefix or similar). Users who don't know that upgrade now runs an install pass will be confused by the extra leading line. Similarly, repos with no extensions print "No extensions in lockfile." before the upgrade summary, which is surprising.
Consider suppressing the no-op install completion message (when
installed === 0 && migrated === 0 && failed === 0) within the upgrade log renderer, or prefixing it with something likeExtensions:so it reads as a sub-step of the upgrade.
Verdict
NEEDS CHANGES — swamp repo upgrade --json now emits two JSON objects, breaking machine-readable consumers. The log-mode output is cosmetically noisy but functional.
There was a problem hiding this comment.
Code Review
Blocking Issues
-
Libswamp import boundary violation in
src/cli/create_extension_install_deps.ts:27: importsExtensionInstallDepsfrom the internal path../libswamp/extensions/install.tsinstead of../libswamp/mod.ts. Per CLAUDE.md: "CLI commands and presentation renderers must import libswamp types and functions fromsrc/libswamp/mod.ts— never from internal module paths." The type is already re-exported frommod.ts(line 689), so this is a one-line fix:-import type { ExtensionInstallDeps } from "../libswamp/extensions/install.ts"; +import type { ExtensionInstallDeps } from "../libswamp/mod.ts";
Suggestions
-
JSON mode outputs two documents for
swamp repo upgrade --json: Theextensionsevent dispatches to the install JSON renderer, which callsconsole.log(JSON.stringify(e.data))on thecompletedevent. Then the upgrade's owncompletedhandler emits a second JSON document. Pre-PR, the extension migration data was embedded inRepoUpgradeData, so JSON consumers got one document. Now they get two (extension install data, then upgrade data). Consider either (a) suppressing the installcompletedoutput in the JSON upgrade renderer, or (b) documenting this as an intentional change. -
dispatchInstallEventlacks an exhaustiveness check (src/presentation/renderers/repo_init.ts:152): Theswitchonevent.kindhas nodefaultcase with aneverassertion. If a new event kind is added toExtensionInstallEvent, the compiler won't flag the missing branch. Addingdefault: { const _: never = event; }is a one-liner that prevents silent drops.
What looks good
- Collapsing the split between migration and install into a single
extensionInstallcall is a clean DDD improvement — "bring extension state into alignment with the lockfile" is one domain concept, not two. The removal of ~280 LOC fromrepo_service.tsand the corresponding test migration toinstall_test.tskeeps responsibility where it belongs (the application/libswamp layer orchestrates, not the domain service). - The
sweepLegacyPaths↔needsInstallOrMigrationseparation gives each function a single job with clean test coverage (8 new tests covering install, migration, failure-preserves-legacy, NotFound tolerance, mixed lockfiles, and empty-dir pruning). - The
resolve(repoDir)absolutization fix increateExtensionInstallDepsaddresses a real latent bug with relative-pathstartsWithcomparisons, and the regression test for nested parent dir pruning locks it down. - Test seam via
installExtensionFnis well-designed — avoids needing a real registry/tar in unit tests while still exercising the full migrate-then-sweep flow. - Renderer delegation via
dispatchInstallEventkeeps upgrade output consistent with standaloneswamp extension installoutput.
There was a problem hiding this comment.
Adversarial Review
Critical / High
-
src/presentation/renderers/repo_init.ts:131-136—repo upgrade --jsonemits two JSON objects, breaking the--jsoncontractThe
JsonRepoUpgradeRenderernow dispatches extension install events toJsonExtensionInstallRenderer, whosecompletedhandler doesconsole.log(JSON.stringify(e.data, null, 2)). Then the upgrade's owncompletedhandler also doesconsole.log(JSON.stringify(e.data, null, 2)). This produces two top-level JSON objects on stdout:{ "entries": [...], "installed": 0, "migrated": 0, ... } { "path": "...", "previousVersion": "...", "newVersion": "...", ... }Any downstream tool doing
swamp repo upgrade --json | jq .or parsing the output as a single JSON value will break. The old renderer only ever emitted one object (the upgrade data, withextensionMigrationembedded in it).Breaking example:
swamp repo upgrade --jsonon any repo with a lockfile (even if all extensions are up-to-date), then piping tojqor any single-document JSON parser.Suggested fix: Either (a) suppress the extension install JSON output during upgrade by passing a no-op renderer, or (b) accumulate the extension install data and embed it in the
RepoUpgradeDataobject so--jsonstill emits a single document, or (c) for the JSON upgrade renderer, replace theextensionshandler with a no-op (only emit the consolidated result incompleted).
Medium
-
src/libswamp/extensions/install.ts:152-175—sweepLegacyPathsfailure after successfulinstallExtensioncreates unrecoverable orphaned filesIf
installExtensionsucceeds (new files written, lockfile rewritten to current-layout paths) butsweepLegacyPathsthrows on a non-NotFound error (e.g., permission denied), the catch block marks the entry as"failed". InrepoUpgrade(init.ts:275-289), this triggers the error path with message "Legacy files have been preserved. Re-run 'swamp repo upgrade'."But the lockfile was already rewritten by
installExtension— itsfiles[]now points at the new per-extension subtree. On re-run,needsInstallOrMigrationreads the updated lockfile, finds all files present at current-layout paths, returns"up_to_date", and the orphaned legacy files are never swept. The error message's advice to re-run is a dead end.Breaking example: Extension
@org/exthas a legacy file atextensions/models/thing.tsowned by root.installExtensionsucceeds (writes.swamp/pulled-extensions/@org/ext/models/thing.ts, rewrites lockfile).Deno.remove("extensions/models/thing.ts")fails with permission denied. Entry is "failed", user told to re-run. Re-run skips it because lockfile says current layout, all files present.Suggested fix: Either (a) snapshot the lockfile before calling
installExtensionand restore it on sweep failure so the next run retries the migration, or (b) catch sweep failures separately from install failures and report them as a non-fatal warning (the migration succeeded, just the cleanup didn't). -
src/cli/commands/repo_init.ts:146-149— Extension install deps built from pre-upgrade marker, stale if--toolchangescreateExtensionInstallDeps(repoDir, ...)is called beforerepoUpgrade()runs. It reads the current marker to deriveskillsDirviaresolveSkillsDir(repoDir, marker.tool). If the user runsswamp repo upgrade --tool cursoron a repo that was previouslytool: "claude", the upgrade changes the marker tocursor, but the install deps still haveskillsDircomputed from"claude". Extension skills get installed to the wrong tool directory.Breaking example:
swamp repo upgrade --tool cursoron aclaude-initialized repo with extensions that provide skills.Suggested fix: Build
extensionInstallDepslazily insiderepoUpgradeafterdeps.upgrade()completes, or re-read the marker post-upgrade to compute the correctskillsDir.
Low
-
src/libswamp/extensions/install.ts:199-219—needsInstallOrMigrationdoes not short-circuit on first legacy classificationThe function continues iterating and stat-ing all remaining files after finding the first legacy file, when it could return
"migrate"early (similar to how it short-circuits on the first missing file). Not a correctness issue — just unnecessary I/O on large extension file lists with legacy paths.
Verdict
FAIL — The --json output regression (#1) is a contract-breaking change that will silently corrupt machine-readable output for all repo upgrade --json consumers. Fix that, and the remaining findings are addressable or acceptable.
…port path Two blocking issues from CI review on PR #1224: 1. `swamp repo upgrade --json` emitted two top-level JSON objects because `JsonRepoUpgradeRenderer` delegated the `extensions` envelope to `JsonExtensionInstallRenderer`, whose `completed` handler does its own `console.log(JSON.stringify(...))`. Downstream parsers that read stdout as a single document broke. Fix: JSON mode captures the install summary internally and folds it into the upgrade's final `completed` object as `extensionInstall`. The log renderer's delegation pattern is unchanged (it uses `logger.info`, not `console.log`, so no double-emit risk). 2. `create_extension_install_deps.ts` imported `ExtensionInstallDeps` from the internal path `../libswamp/extensions/install.ts` instead of `../libswamp/mod.ts`. CLAUDE.md explicitly requires CLI imports from `mod.ts`. The type was already re-exported there. Added a regression test that runs the JSON renderer through a full event sequence and asserts the stdout parses as exactly one JSON object, with the install summary present under `extensionInstall`. Verified end-to-end against `@swamp/aws/s3`: `swamp repo upgrade --json` emits a single parseable JSON object with `extensionInstall.migrated: 1`. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
There was a problem hiding this comment.
Code Review
Well-structured PR that correctly consolidates the two-phase legacy migration into the existing extensionInstall primitive. The approach — detect legacy layout, re-pull via the normal install path, then sweep orphaned files — is cleaner than the deleted migrateExtensionLayout approach and eliminates the "silent delete + manual follow-up" UX problem.
Blocking Issues
None.
Suggestions
-
Stale doc comment on
PULLED_TYPE_DIRS(src/libswamp/extensions/layout.ts:42-46): The JSDoc still references "the phase-two migration inRepoService" which was deleted in this PR. Worth updating the comment to referencesweepLegacyPaths/classifyExtensionFileas the remaining consumers. -
Shared
alreadyPulledset: EachcreateInstallContextcall increateExtensionInstallDepscreates a freshnew Set(). If two lockfile entries share a transitive dependency, it will be pulled twice. This is pre-existing behavior (the oldextension_install.tscommand had the same pattern), so not a regression — but worth noting as a potential optimization for a follow-up. -
dispatchInstallEventexhaustiveness: The switch statement handles all current event kinds but lacks adefault: { const _: never = event; }guard. TypeScript's narrowing would catch a new unhandled variant atdeno checktime anyway, so this is minor — an explicit exhaustiveness check would make the intent clearer.
What looks good
- DDD alignment: The composition root (
createExtensionInstallDeps) lives correctly in the CLI layer, wiring infrastructure (API client, marker repo) into a domain-facing deps interface. The domain service (RepoService) is properly slimmed — migration logic moved to the application service (extensionInstall) where it belongs as cross-aggregate orchestration. - Import boundary: All CLI and presentation imports go through
src/libswamp/mod.ts. Thelibswamp/repo/init.ts→../extensions/install.tsimport is internal to libswamp, which is correct. - Test coverage: 8 new tests covering
needsInstallOrMigration,sweepLegacyPaths, the fullextensionInstallmigration flow (happy path, mixed lockfile, failure preservation), and the JSON renderer's single-object invariant. The test seam (InstallExtensionFn) avoids coupling tests to a real registry. - Error handling: Legacy files are preserved on install failure, the error surfaces extension names and a recovery command, and
NotFoundis tolerated for interrupted-retry scenarios. - Relative path bug fix: Absolutizing
repoDirin the factory prevents thestartsWithcomparison failure incleanupEmptyParentDirs. The regression test for nested_lib/directories covers this. - JSON output invariant: The
JsonRepoUpgradeRenderercorrectly captures the install summary and folds it into the singlecompletedpayload instead of delegating toJsonExtensionInstallRenderer(which would emit a second JSON object mid-stream). - License headers: Present on both new files.
- Net -170 LOC with better behavior — good cleanup.
There was a problem hiding this comment.
CLI UX Review
Blocking
None.
Suggestions
-
Extra log noise on every
repo upgrade—src/presentation/renderers/repo_init.tsdelegates to the extension install renderer unconditionally, so every upgrade now printsReading lockfile...and eitherAll extensions up to date.orNo extensions in lockfile.even when no migration is needed. These lines are already appropriate forswamp extension install, but they read oddly as part ofswamp repo upgradeon a clean repo. Consider suppressingresolving/completedevents inLogRepoUpgradeRendererwhenmigrated === 0andfailed === 0, or adding a guard indispatchInstallEventto skip no-op results. -
Upgrade success is silent when extension migration fails — in
src/libswamp/repo/init.tsthecompletedevent (which printsUpgraded swamp repository: 0.1.0 → 0.1.1) is never yielded when any extension fails to migrate. The base upgrade (marker, skills, settings) DID succeed, but the user only sees the extension error. They have no log confirmation that the repo itself was upgraded. A line likeUpgraded swamp repository: X → Y; extension migration incomplete — re-run to finish.before the error would close that gap. -
swamp extension installfailure summary omits a recovery command —src/presentation/renderers/extension_install.ts:78 warns{count} extension(s) failed to install.with names and errors, but gives no next-step command.swamp repo upgradeerror message correctly saysRe-run 'swamp repo upgrade'; parity here would beRe-run 'swamp extension install' once the issue is resolved.(Pre-existing gap, surfaced by this PR's unification — low priority.)
Verdict
PASS — JSON output is correct (single top-level object, extensionInstall folded in cleanly), error messages are actionable and name the affected extensions with a recovery command, log-mode output is consistent with existing patterns. Suggestions above are all cosmetic.
There was a problem hiding this comment.
Adversarial Review
Critical / High
None found. The core install→sweep sequence is correct: legacy paths are snapshotted before installExtension rewrites the lockfile, the sweep only runs after a successful install (preserving legacy files on failure), and classifyExtensionFile prevents current-layout paths from being deleted. Error handling is clean — per-entry failures are captured without aborting the loop, and infrastructure errors bubble up through the generator's try/catch.
Medium
cleanupEmptyParentDirssilently fails on relativerepoDir— patched at callsite only (src/cli/create_extension_install_deps.ts:49,src/infrastructure/persistence/directory_cleanup.ts:37). The factory callsresolve(repoDir)to absolutize, which fixes the immediate bug (empty_lib/dirs stranded). ButcleanupEmptyParentDirs's guardcurrentDir.startsWith(stopAtDir)still silently no-ops for relative paths. Any future caller ofsweepLegacyPathsthat passes a non-absoluterepoDir(e.g."."fromresolveRepoDir's default) would reintroduce the stranded-directory bug. Consider addingresolve()insidesweepLegacyPathsorcleanupEmptyParentDirsitself as a follow-up.
Low
-
sweepLegacyPathscallscleanupEmptyParentDirsunconditionally afterDeno.removeNotFound (src/libswamp/extensions/install.ts:248). When the file was already gone, the cleanup walks parent dirs that may themselves not exist.cleanupEmptyParentDirshandles this gracefully (its own NotFound catch), so this is just unnecessary I/O, not a bug. -
dispatchInstallEventrouteserrorevents to install renderer which throwsUserError(src/presentation/renderers/repo_init.ts:183). Currently harmless becauseextensionInstallnever yieldserrorevents (per-entry failures are captured incompleted). But if a future refactor adds error events toextensionInstall, the throw inside theextensionshandler would bypass the upgrade renderer's ownerrorhandler, producing an uncontrolled exception path. -
createExtensionInstallDepsreads marker before upgrade runs (src/cli/commands/repo_init.ts:111-114). The factory snapshotsmodelsDirandtoolfrom the pre-upgrade marker. Ifupgrade()modifies these fields, the extension install pass uses stale values. In practicemodelsDirdoesn't change during upgrade, making this theoretical only.
Verdict
PASS. The migration logic is sound: install-before-sweep ordering prevents data loss, the failure path preserves legacy files, and the JSON single-object invariant is tested. The relative-path fragility in cleanupEmptyParentDirs is worth a defensive follow-up but doesn't block — the production callsite is correctly absolutized. Net code reduction with better behavior; well tested.
Summary
Closes swamp-club#160.
swamp repo upgradewas silently deleting user working-tree files when it detected extensions tracked at a legacy on-disk layout. The two-phase migration added in #1186 renamed gen-1 paths to gen-2, then deleted the gen-2 files expecting the user to runswamp extension installafterward to restore them. No warning, no prompt, no automatic follow-up.The fix collapses the split between the two primitives that owned "make extension state correct":
extensionInstallnow treats legacy-layout entries as needing re-pull (not just missing files), installs them into the per-extension subtree, and sweeps the original legacy paths only after a successful install.repo upgradeinvokesextensionInstalldirectly as its final step.migrateExtensionLayoutand its two phase methods are deleted (~280 LOC) along withExtensionLayoutMigrationSummary,pruneEmptyDirsUp, and theextensionMigrationresult field.Net LOC: −170 despite adding the install-pass wiring, a test seam, a shared factory, and several new tests.
User-visible behavior
swamp repo upgradenow completes the migration end-to-end. No manualswamp extension installfollow-up required for the common case.swamp extension installalso migrates legacy-layout repos. Previously a silent no-op on legacy state — positive side effect of unifying the two primitives.Latent bug caught during verification
Running an end-to-end manual test against
@swamp/aws/s3surfaced a second bug: empty_lib/directories were being stranded after the sweep. Root cause:resolveRepoDirreturns"."by default, which propagated throughjoin(".", "path") → "path"(relative), andcleanupEmptyParentDirs'sstartsWith(stopAtDir)check silently failed against relative paths. The factory now absolutizesrepoDirup front so downstream filesystem helpers always see absolute paths. Added a regression test asserting nested parent dirs are pruned.This one wouldn't have been caught by unit tests alone — my initial unit tests used absolute
tmpDirfromDeno.makeTempDir, which masked the relative-path issue. Running against a real extension with a nested_lib/found it immediately.Test plan
deno check— cleandeno lint— clean (1081 files)deno fmt— clean (1199 files)deno run test— 4728 pass, 0 fail, 1 ignored (added 8 new tests)deno run compile— binary produced@swamp/aws/s3seeded as gen-1 legacy,swamp repo upgraderun; all 11 model files migrated to.swamp/pulled-extensions/@swamp/aws/s3/models/…,filesChecksumanchor matches original pull byte-for-byte (a9378bf483…), legacy paths (including nested_lib/) fully swept.@me/launchd-daemon(not in registry);swamp repo upgradeexits non-zero with clear error naming the extension + recovery command; all 4 legacy files preserved on disk.extension install: same legacy seed;swamp extension installalso migrates successfully (positive side effect verified).Notes for reviewers
swamp extension installbehavior change (no-op → auto-migrate on legacy repos) is intentional; documented in theswamp-reposkill.CHANGELOG.mdin this repo, so the user-visible release notes live in this PR description.ExtensionApiClientasswamp extension install, so coupling should hold, but worth spot-checking if a reviewer has a private extension in a test lockfile.🤖 Generated with Claude Code