Skip to content

feat: auto-update @swamp/ datastore extensions once per day#942

Merged
stack72 merged 3 commits intomainfrom
feat/datastore-auto-update-939
Mar 30, 2026
Merged

feat: auto-update @swamp/ datastore extensions once per day#942
stack72 merged 3 commits intomainfrom
feat/datastore-auto-update-939

Conversation

@stack72
Copy link
Copy Markdown
Contributor

@stack72 stack72 commented Mar 30, 2026

Summary

Fixes #939 — automatically checks for and pulls updates to @swamp/ datastore extensions once per day.

When a @swamp/ datastore extension is loaded during command startup, swamp now checks if the installed version is outdated (using a 24h-cooldown cache). If a newer version is available on the registry, it auto-pulls and hot-reloads before the DatastoreProvider is instantiated.

Scope: Only @swamp/ collective datastore extensions. Does not apply to models, workflows, vaults, drivers, reports, or third-party datastores.

Key design decisions:

  • Check placed inside resolveDatastoreConfig() between resolveDatastoreType() and createProvider() — ensures updated registry entry is used when provider is instantiated
  • Helper function maybeAutoUpdateSwampDatastore() called from all three @swamp/ resolution paths (env var, .swamp.yaml renamed, .swamp.yaml custom)
  • Cache stored in .swamp/extension-update-checks.json — shared via S3 for remote datastores, so all machines share the 24h cooldown
  • Registry errors and pull failures are silently ignored — never blocks command execution

Test plan

  • 5 unit tests for cache staleness logic
  • 3 unit tests for file-based cache repository
  • 8 unit tests for orchestration function (skip non-@swamp, cache fresh, update available, already latest, no installed version, registry error, pull error, 24h re-check)
  • All 3729 existing tests pass
  • deno check, deno lint, deno fmt all pass

🤖 Generated with Claude Code

Add a daily staleness check for @swamp/ datastore extensions during
datastore resolution. When the check cache is older than 24 hours,
queries the registry for the latest version and auto-pulls if outdated.

- New domain type: per-extension update check cache with 24h cooldown
- New orchestration function: maybeAutoUpdateDatastoreExtension()
- Helper maybeAutoUpdateSwampDatastore() called from all three @swamp/
  resolution paths in resolveDatastoreConfig(), placed between
  resolveDatastoreType() and createProvider() so updated code is used
- Only triggers for @swamp/ collective, never for third-party extensions
- Registry errors and pull failures are silently ignored

Closes #939

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
stack72 and others added 2 commits March 30, 2026 20:38
The previous implementation used resolver.resolve() which only installs
missing extensions — it doesn't update already-installed ones. Switch to
installExtension() with force:true + UserDatastoreLoader hot-reload,
matching the pattern used by the auto-resolver adapter.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ebug logging

The lockfile path was hardcoded to "models/" but the actual directory
is resolved via resolveModelsDir() (defaults to "extensions/models").
Also add debug logging to diagnose auto-update check flow.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLI UX Review

Blocking

None.

Suggestions

  1. Duplicate log messages on successful update — When an update is pulled, users see two info-level messages back-to-back:

    • datastore_auto_update.ts:120: "Updating @swamp/s3-datastore 2026.03.15.1 → 2026.03.30.1" (before the pull)
    • resolve_datastore.ts:158: "Updated @swamp/s3-datastore 2026.03.15.1 → 2026.03.30.1" (after the pull)

    Only one message is needed. The pre-pull "Updating ..." in datastore_auto_update.ts is redundant since the caller already logs the result. Suggest removing the logger.info at datastore_auto_update.ts:120-123 and keeping only the post-pull "Updated ..." in resolve_datastore.ts.

  2. No user control over auto-update behavior — There's no flag to opt out (e.g., --no-auto-update) or force an immediate check. Not blocking for v1, but worth tracking as the feature matures.

Verdict

PASS — the auto-update runs silently in the background, failures are swallowed, and the only user-visible output is the update log message. The duplicate messaging is minor noise and doesn't block.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial Review

Critical / High

None found.

Medium

  1. Cache written before pull — failed pull suppresses retry for 24h (src/libswamp/extensions/datastore_auto_update.ts:108-113)

    The cache is updated at line 109-113 before pullExtension is called at line 124. If pullExtension throws (network timeout, disk full, checksum mismatch), the outer catch at line 131 returns null, but the cache already records checkedAt: now with the latest version. For the next 24 hours, isExtensionCheckStale() returns false and the update is never retried.

    Breaking scenario: User runs a command while their network is flaky. The registry check succeeds (getting latestVersion), but the archive download in pullExtension fails. The extension stays at the old version, and no retry happens until the cache expires 24 hours later.

    Suggested fix: Move the cache write to after the successful pull (between lines 124 and 126). For the "already at latest" path (line 115-117), the cache write at its current position is fine — split it into two: write-on-up-to-date before the early return, and write-on-successful-pull after pullExtension resolves. This way a pull failure allows an immediate retry on the next command.

  2. Import from internal libswamp path violates architecture rule (src/cli/resolve_datastore.ts:44)

    maybeAutoUpdateDatastoreExtension is imported from ../libswamp/extensions/datastore_auto_update.ts, but CLAUDE.md requires CLI code to import libswamp functions from src/libswamp/mod.ts. The function is already exported from mod.ts (this PR adds it). The import should be:

    import { maybeAutoUpdateDatastoreExtension } from "../libswamp/mod.ts";

Low

  1. Variable shadowing of resolvedRepoDir (src/cli/resolve_datastore.ts:75 and src/cli/resolve_datastore.ts:103)

    resolvedRepoDir is declared with const at line 75 (resolve(repoDir)) in the outer function scope, then declared again at line 103 inside the pullExtension callback with the same resolve(repoDir) expression. The values are identical so behavior is correct, but the inner declaration is unnecessary and could mask a future bug if repoDir were ever reassigned. Use the outer resolvedRepoDir directly in the closure.

  2. Non-atomic cache file writes (src/infrastructure/persistence/extension_update_check_repository.ts:48-51)

    Deno.writeTextFile is not atomic — a crash mid-write leaves a truncated file. The read() method's catch-all means this self-heals (returns {}), so worst case is one extra registry check. Existing pattern in the codebase (update_check_cache.ts) has the same characteristic, so this is consistent.

Verdict

PASS — The core logic is sound and well-tested. The cache-before-pull ordering (Medium #1) is the most actionable finding — it causes a 24h retry blackout on pull failure. The import path (Medium #2) is a straightforward fix. Neither blocks the merge but both are worth addressing.

@stack72 stack72 merged commit 87fb160 into main Mar 30, 2026
10 checks passed
@stack72 stack72 deleted the feat/datastore-auto-update-939 branch March 30, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: auto-update @swamp/ datastore extensions once per day

1 participant