Skip to content

fix(extensions): refuse datastore auto-update on local edits (swamp-club#126)#1190

Merged
stack72 merged 1 commit intomainfrom
worktree-shimmying-mapping-quasar
Apr 18, 2026
Merged

fix(extensions): refuse datastore auto-update on local edits (swamp-club#126)#1190
stack72 merged 1 commit intomainfrom
worktree-shimmying-mapping-quasar

Conversation

@stack72
Copy link
Copy Markdown
Contributor

@stack72 stack72 commented Apr 18, 2026

Summary

Fixes swamp-club#126 — datastore auto-update uses force: true in the pullExtension callback and silently overwrites any local edits a user has made under .swamp/pulled-extensions/<name>/. Same data-loss family as swamp-club#121/#1187 (auto-resolver force-pull), different trigger (version-behind rather than type-not-found), different fix shape (per-file anchor rather than isInstalled skip — because auto-update, by definition, wants to replace the prior install).

Approach:

  • Record a rolled-up SHA-256 digest of the per-extension on-disk subtree in upstream_extensions.json at install time (filesChecksum), distinct from the existing archive checksum.
  • Before auto-update fires installExtension with force: true, the service calls a new detectLocalEdits dep that compares the stored anchor with a fresh disk digest. Tri-state outcome: match proceeds; mismatch returns a structured { skipped: "local_edits", previousVersion, newVersion } result and does NOT call pullExtension; no-anchor grandfathers pre-existing installs and proceeds.
  • The CLI caller (resolve_datastore.ts) inspects the skipped result and emits logger.warn directing the user to swamp extension pull <name> --force as the explicit opt-in. The refusal is a STRUCTURED RESULT rather than a thrown error because the outer try/catch intentionally swallows exceptions to honor "never block command execution for auto-update failures"; a structured result separates "error — swallow" from "refusal — surface" cleanly.

DDD split:

  • Pure combiner in src/domain/extensions/installed_extension_digest.ts takes pre-hashed entries and produces the rollup digest. No FS access.
  • FS-walking reader in src/infrastructure/persistence/installed_extension_digest_reader.ts walks the per-extension tree, hashes each file, and calls the pure combiner.
  • Mirrors the pattern already in use for src/domain/models/checksum.ts (pure compute) vs callers that read bytes from disk.

Grandfather / migration:

Pre-anchor lockfile entries (installs from before this change) lack filesChecksum. detectLocalEdits returns no-anchor for them and auto-update proceeds as it does today. On the next install, the anchor is written and all subsequent auto-updates are protected. A one-time silent-overwrite window remains for users with pending edits AND a stale lockfile entry AND an auto-update firing before they next pull — acceptable given the scope of the fix; mentioned here for release notes.

Follow-ups filed separately

  • swamp-club#129src/cli/commands/open.ts:185 has the same force: true pattern in the web UI's non-interactive install callback. Same data-loss family, different trigger; tracked separately.
  • swamp-club#130logger.warn is silent in --json mode because the logger's catch-all is lowestLevel: \"fatal\" in JSON output mode. The auto-update refusal is silent in JSON mode (data is still protected — refusal short-circuits before installExtension — but the user gets no signal why auto-update isn't happening). Pre-existing issue affecting ~50 logger.warn call sites; changing it is too much blast radius for a bug-fix PR.

Explicitly out of scope / confirmed safe:

  • src/cli/commands/extension_install.ts:114 uses force: true but the caller in src/libswamp/extensions/install.ts:88-97 short-circuits via hasAnyMissingFiles before calling installExtension if any tracked file exists. Local edits and force:true cannot co-exist on that path.
  • swamp extension pull / extension update — explicit user opt-in, not a silent path; unaffected.

Test plan

  • deno check clean
  • deno lint clean
  • deno fmt applied
  • deno run test — 4447 passed, 0 failed (includes 7 new tests covering: digest determinism, FS-reader invariants, auto-update tri-state decision tree, caller-layer warning message)
  • deno run compile succeeded
  • Manual JSON-mode verification deliberately skipped — this PR does not change JSON-mode behavior (the WARN is silent in JSON mode by pre-existing design; tracked as swamp-club#130)
  • UAT follow-up: a new tests/cli/extension/auto_update_preserves_local_edits_test.ts parallel to pull_wip_preservation_test.ts (which locks in the Issue 6: Migrate echo model to new architecture #121 fix) should be filed as a swamp-uat issue after merge.

🤖 Generated with Claude Code

…ed (#126)

Record a per-install SHA-256 digest of the on-disk per-extension subtree in
upstream_extensions.json and consult it before auto-update fires its force:true
pull. When the on-disk digest diverges from the anchor (user edits detected),
the auto-update service returns a structured { skipped: "local_edits" } result
instead of silently overwriting; the CLI caller emits a logger.warn directing
the user to `swamp extension pull <name> --force` to opt in.

Pre-anchor lockfile entries (installs from before this change) return
"no-anchor" and proceed as before, so existing repos upgrade cleanly — the
protection engages on the next install once an anchor is written.

The structured-result pattern (rather than throwing) threads through the outer
maybeAutoUpdateSwampDatastore try/catch that intentionally swallows exceptions
to honor the "never block command execution for auto-update failures" invariant.
A thrown error would get eaten and the user would never see the refusal;
a structured result separates "error — swallow" from "refusal — surface".

Follow-ups filed as separate issues:
- swamp-club#129: open.ts web UI has the same force:true pattern
- swamp-club#130: logger.warn is silent in --json mode (affects this and
  ~50 other call sites; out of scope for this fix)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLI UX Review

Blocking

None.

Suggestions

  1. Message format inconsistency vs. #121 precedent (src/cli/resolve_datastore.ts:208–212)
    The #121 auto-resolver refusal (renderAutoResolveAlreadyInstalled) renders as three separate logger.error lines and supports a JSON-mode branch. This PR emits a single long logger.warn string with no JSON branch. The PR acknowledges the JSON-mode silence as swamp-club#130 (pre-existing, ~50 sites), which is fair — data is still protected in JSON mode. But consider splitting the warn into two lines the way the #121 renderer does, so it's easier to scan:

    Not auto-updating @swamp/s3-datastore 2026.03.15.1 → 2026.03.30.1: local edits detected under .swamp/pulled-extensions/@swamp/s3-datastore/.
    Run `swamp extension pull @swamp/s3-datastore --force` to discard your edits and install the latest version.
    
  2. Optional version fields in buildLocalEditsWarning (src/cli/resolve_datastore.ts:207)
    The signature accepts { previousVersion?: string; newVersion?: string } but if either is undefined the output would read …undefined → undefined…. In practice the service always populates both when it returns skipped: "local_edits", but narrowing the parameter type (or adding a ?? "(unknown)" fallback) would future-proof the message.

Verdict

PASS — warning message is clear, actionable, and consistent in form with other warn-level messages in the same file. Data protection works in both log and JSON modes; the JSON-mode notification gap is a pre-existing issue correctly tracked separately.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Clean, well-designed fix for the data-loss issue. The DDD layering is correct: pure domain combiner in src/domain/extensions/installed_extension_digest.ts (no FS access), infrastructure reader in src/infrastructure/persistence/installed_extension_digest_reader.ts (walks disk, produces entries), and detectLocalEdits injected as a dependency into the auto-update service — proper dependency inversion. The tri-state match | mismatch | no-anchor design with a structured result (not an exception) for the refusal path is a good fit for the "never block command execution for auto-update failures" contract.

Test coverage is thorough: 7 tests for digest determinism/sensitivity, 6 tests for the FS reader (including macOS resource fork exclusion and edge cases), 3 tests for the service-layer tri-state decision tree, and 1 test for the warning message shape. All new files have the AGPLv3 header. License headers, test naming conventions, and file placement all match CLAUDE.md requirements.

Blocking Issues

None.

Suggestions

  1. Export new types from src/libswamp/mod.ts: LocalEditsStatus and DatastoreAutoUpdateSkipReason are part of the public interface of already-exported types (DatastoreAutoUpdateDeps.detectLocalEdits returns Promise<LocalEditsStatus>; DatastoreAutoUpdateResult.skipped is typed as DatastoreAutoUpdateSkipReason). External consumers implementing DatastoreAutoUpdateDeps or inspecting the result would need these types. Consider adding them to the re-export block at lines 568–571.

  2. Symlinks silently excluded from digest: walk() in installed_extension_digest_reader.ts only processes entry.isFile entries. In Deno, symlinks have isSymlink: true / isFile: false, so a user-added symlink wouldn't change the digest. Unlikely in practice for extension directories, but worth a brief note or a test confirming the intentional exclusion.

  3. Pre-existing (not introduced by this PR): resolve_datastore.ts:44 imports maybeAutoUpdateDatastoreExtension from the internal libswamp path ../libswamp/extensions/datastore_auto_update.ts rather than from ../libswamp/mod.ts, which conflicts with the CLAUDE.md import boundary rule. Not a regression — just noting for a future cleanup pass.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial Review

Critical / High

None found.

Medium

  1. Duplicated isMacOsResourceFork — divergence invalidates every anchor (src/infrastructure/persistence/installed_extension_digest_reader.ts:78 and src/libswamp/extensions/pull.ts:211): Both files define an identical private isMacOsResourceFork(name) helper. The install-time digest (via pull.ts's copyDir) and the check-time digest (via the reader's walk) must agree on which files exist. If someone modifies one function but not the other — e.g., adding .DS_Store filtering to the installer but not the reader — every existing filesChecksum anchor silently becomes invalid, and all auto-updates start reporting false "local edits detected" for every extension. This isn't a current bug, but it's a maintenance landmine with blast radius proportional to the number of installed extensions. Consider extracting a shared shouldSkipEntry(name) predicate or at minimum a cross-reference comment.

Low

  1. TOCTOU between detectLocalEdits and pullExtension (src/libswamp/extensions/datastore_auto_update.ts:155-178): Between detectLocalEdits returning "match" and pullExtension executing, the user could edit a file. The protection degrades silently to the pre-fix behavior (overwrite). Inherent in any check-then-act pattern; acceptable here because auto-update is a convenience feature, not a security boundary.

  2. Symlinks silently excluded from digest (src/infrastructure/persistence/installed_extension_digest_reader.ts:61-75): walk only processes entry.isFile and entry.isDirectory. In Deno's readDir, symlinks have isSymlink: true but isFile: false and isDirectory: false, so they are silently skipped. This is consistent with the installer's copyDir (which also only handles files and directories), so install-time and check-time digests agree. But a user who adds a symlinked file to the extension tree would not trigger the protection. Very unlikely in practice.

  3. Double lockfile read per auto-update (src/cli/resolve_datastore.ts:101-103 and 119): getInstalledVersion and detectLocalEdits each independently call readUpstreamExtensions(lockfilePath). Two reads per check is a minor inefficiency, not a correctness issue.

Verdict

PASS — The code is well-structured and handles edge cases thoughtfully. The tri-state detectLocalEdits contract is clean, the degradation paths ("no-anchor", catch → "no-anchor") correctly prevent the safety check from ever blocking a user command, and the lockfile anchor is written at the right point in the install lifecycle (after all file writes, before the lockfile commit). The one medium finding is a maintenance concern, not a shipping blocker.

@stack72 stack72 merged commit c17d3e4 into main Apr 18, 2026
10 checks passed
@stack72 stack72 deleted the worktree-shimmying-mapping-quasar branch April 18, 2026 00:58
ianarsenault pushed a commit to ianarsenault/swamp that referenced this pull request Apr 21, 2026
systeminit#1191)

## Summary

Third in the data-loss family (sibling of systeminit#121/systeminit#1187 and systeminit#126/systeminit#1190).
`swamp open`'s web UI install callback calls `pullExtension` with
`force: true` and, pre-fix, had no local-edits check — so hitting `POST
/api/extensions/install` for an already-installed extension would
silently overwrite anything the user had edited under
`.swamp/pulled-extensions/<name>/`.

Port systeminit#126's `filesChecksum` anchor + detectLocalEdits pattern into a
shared libswamp helper and consult it from the web UI before the
force-pull. On mismatch, throw a typed `LocalEditsError`; the HTTP
handler maps it to **409 Conflict** with a body that names the extension
and the opt-in remediation (`swamp extension pull <name> --force`).

The ui.ts Install button only renders for **uninstalled** extensions
today, so there's no UI click path that reaches this bug — the fix is
defense-in-depth for the HTTP endpoint itself, which is reachable by
curl, scripts, or any future UI change (e.g. an "Update / Reinstall"
button). Scope and fix-shape intentionally mirror systeminit#126.

### What changes
- New `src/libswamp/extensions/local_edits.ts` — shared helper
(`detectLocalEditsForExtension`, `LocalEditsStatus`, `LocalEditsError`),
re-exported through `libswamp/mod.ts`
- `src/cli/resolve_datastore.ts` — systeminit#126's inline `detectLocalEdits`
closure collapses to a one-line call into the new helper (pure DRY)
- `src/cli/commands/open.ts` — `installExtension` callback runs the
local-edits check before force-pull; throws `LocalEditsError` on
mismatch
- `src/serve/open/http.ts` — `handleExtensionInstall` maps
`LocalEditsError` → 409, unchanged 500 fallthrough for other errors
- Unit tests at the new helper and at the HTTP handler layer

### What's out of scope
- `extension_update.ts:118` and `extension_install.ts:114` still use
`force:true`; the former is an explicit user command, the latter is
lockfile-restore. Refusing them would break their contracts.
- The dependency-with-corrupt-lockfile edge case (parent reinstall
recurses into a dep missing from `upstream_extensions.json` but present
on disk with edits) — matches systeminit#126's scope. Fixing it would require
pushing the check into `installExtension` itself, which would wrongly
refuse the opt-in paths above.
- End-to-end UAT for the web UI refuse flow: filed as
[systeminit/swamp-uat#150](systeminit/swamp-uat#150).

## Test Plan

- [x] `deno check`, `deno lint`, `deno fmt`, full `deno run test` (4455
passed), `deno run compile`
- [x] New unit coverage: 6 tests in
`src/libswamp/extensions/local_edits_test.ts` (match, mismatch,
no-anchor × 3 conditions, LocalEditsError message shape)
- [x] New handler coverage: 2 tests in `src/serve/open/http_test.ts`
(409 on `LocalEditsError`, 500 on plain `Error` as a regression fence)
- [x] Manual E2E with locally compiled binary: scratch repo → `swamp
extension pull @stack72/system-extensions` → edit a file → `curl POST
/api/extensions/install` → got 409 with the expected remediation body.
`swamp extension pull --force` → curl again → 200.
- [x] systeminit#126's `datastore_auto_update_test.ts` and
`resolve_datastore_test.ts` both still green after the helper refactor

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant