feat: automatic extension resolution with trusted collectives#725
feat: automatic extension resolution with trusted collectives#725
Conversation
When users clone a repo with model/vault configs referencing extension types that aren't installed locally, commands fail with "Unknown model type" errors. This adds lazy auto-resolution: when swamp hits an unknown type whose collective is on an allowlist, it searches the registry, installs the extension, hot-loads it, and continues transparently. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
There was a problem hiding this comment.
Review Summary
This PR implements automatic extension resolution for trusted collectives - a well-designed feature that improves the onboarding experience.
No blocking issues found.
Code Quality ✓
- TypeScript strict mode: properly typed interfaces throughout
- Named exports: correctly used
- No
anytypes in new code - License headers: present on all new files
Domain-Driven Design ✓
ExtensionAutoResolveris correctly implemented as a Domain Service (stateless, orchestrates multiple concerns)- Port interfaces (
ExtensionLookupPort,ExtensionInstallerPort,AutoResolveOutputPort) provide clean separation between domain and infrastructure - Clean hexagonal architecture: domain defines ports, CLI/infrastructure provides adapters
- The ambient context pattern (
auto_resolver_context.ts) is acceptable for CLI applications where full DI is impractical
Test Coverage ✓
- 14 new tests cover the key scenarios: allowlist filtering, direct lookup, search fallback, network errors, re-entrancy guard
- Tests use proper mocking of port interfaces
Security ✓
- Auto-resolution only works for types from trusted collectives (configurable whitelist)
- Default trusted collectives are sensible (
["swamp", "si"]) - No code injection or remote execution risks
Architecture Notes
- Architecture ratchets bumped (
KNOWN_MUTUAL_DEPENDENCIES7→9,KNOWN_PRESENTATION_INFRA_VIOLATIONS39→40) - these are tracked by tests and represent acceptable coupling for this feature
Suggestions (non-blocking)
- Unrelated file:
blog-datastore-internals.mdappears unrelated to extension auto-resolution. Consider moving it to a separate PR to keep this PR focused.
LGTM - well-architected feature with good test coverage and clean DDD patterns.
There was a problem hiding this comment.
Adversarial Review
I systematically traced code paths, analyzed error handling, and looked for edge cases across all 21 changed files. The implementation is generally solid with clean architecture (port interfaces) and comprehensive test coverage.
Critical / High
None found.
Medium
-
src/domain/extensions/extension_auto_resolver.ts:291-299 - Silent failure with misleading output.
Issue: In
installAndLoad, the code callsextensionLookup.getExtension(extensionName)a second time after a successful direct lookup or search. If this second call fails (e.g., transient network issue), the method returnsfalsewith no output. The caller indoResolvethen showsoutput.notFound(normalizedType), which is misleading—the extension was found but couldn't be fetched for installation.Breaking example:
- User runs
swamp model create @swamp/aws/ec2/instance my-instance - Direct lookup finds
@swamp/aws(first getExtension call succeeds) - Second getExtension call in installAndLoad fails (transient timeout)
- User sees: "no matching extension found in registry"
- Actual cause: Network hiccup between finding and installing
Suggested fix: Add a distinct output path for "found but failed to get install info":
const extInfo = await extensionLookup.getExtension(extensionName); if (!extInfo) { output.installFailed(extensionName, "extension info unavailable"); return false; }
- User runs
Low
-
src/domain/extensions/extension_auto_resolver.ts:327-330 - Overly broad network error classification.
private isNetworkError(error: unknown): boolean { if (error instanceof TypeError) { return true; } // ... }
Issue: All
TypeErrorinstances are treated as network errors. Whilefetch()does throw TypeError on network failure, TypeErrors can also occur from accessing properties on undefined, type coercion issues, etc.Impact: Only affects error message presentation—users might see "network error" for non-network issues. Low severity since the fallback behavior (showing install instructions) is still helpful.
-
src/domain/extensions/auto_resolver_context.ts:26 - Global singleton without synchronization.
Issue: The resolver is stored in module-level state with no locking. If
runCli()were called multiple times concurrently (e.g., in tests or programmatic usage), the resolver could be overwritten mid-resolution.Impact: Theoretical only—the CLI is single-threaded in practice, and the existing test suite passes. Mentioning for completeness.
Verdict
PASS — No blocking issues. The medium finding is a UX improvement opportunity, not a correctness bug. The code handles the core auto-resolution flow correctly, the re-entrancy guard is properly implemented with try/finally, edge cases (empty collectives, missing marker, non-@ types) are handled, and tests cover the main paths.
) Closes #665 ## Summary Moves the `aws-sm`, `azure-kv`, and `1password` vault providers from built-in types to extension vaults published at swamp.club. After this change, only `local_encryption` (and `mock` for testing) remain as built-in vault types. The three cloud/external vault providers are now independently versioned extensions that auto-resolve from the registry on first use. ### What changed **Removed from core:** - Deleted `aws_vault_provider.ts`, `azure_kv_vault_provider.ts`, `onepassword_vault_provider.ts` and their test files (-1,545 lines) - Removed `aws-sm`, `azure-kv`, `1password` from `BUILT_IN_VAULT_TYPES` in `vault_types.ts` — only `local_encryption` remains - Removed their switch cases from `VaultService.registerVault()` - Removed `@aws-sdk/client-secrets-manager`, `@azure/identity`, `@azure/keyvault-secrets` from `deno.json` dependencies **Migration path via `RENAMED_VAULT_TYPES`:** - `aws` / `aws-sm` → `@swamp/aws-sm` - `azure` / `azure-kv` → `@swamp/azure-kv` - `1password` → `@swamp/1password` When `VaultService.fromRepository()` loads an existing vault config with an old type name, it remaps to the `@swamp/*` extension type and auto-resolves it from the registry (installed by PR #725's auto-resolution infrastructure). **`vault create` simplified:** - Removed `--region`, `--vault-url`, `--op-vault`, `--op-account` flags - All extension vault types now use `--config <json>` for provider configuration - `resolveProviderConfig()` only handles `local_encryption` now **`ensureDefaultVaults()` is now a no-op:** - Previously auto-created an AWS vault when `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_REGION` were set - This behavior is removed since AWS is now an extension **Error messages updated:** - "No vaults configured" error now suggests `swamp extension pull @swamp/aws-sm` instead of setting AWS env vars ### Published extensions The three vault providers have been published to swamp.club as: - `@swamp/[email protected]` — shells out to `op` CLI, no npm SDK deps - `@swamp/[email protected]` — uses `@aws-sdk/[email protected]` - `@swamp/[email protected]` — uses `@azure/[email protected]` + `@azure/[email protected]` Source lives at https://github.com/systeminit/swamp-extensions ## User impact ### Existing users with vault configs on disk **No action required.** Existing `.swamp/vault/*.yaml` files with `type: aws-sm`, `type: azure-kv`, or `type: 1password` continue to work. On first use, swamp will: 1. Log a deprecation warning about the old type name 2. Remap it to the `@swamp/*` extension type 3. Auto-resolve and install the extension from the registry 4. Load the vault and proceed normally ### Creating new vaults The CLI syntax changes from dedicated flags to `--config <json>`: ```bash # Before swamp vault create aws-sm my-vault --region us-east-1 swamp vault create azure-kv my-vault --vault-url https://myvault.vault.azure.net/ swamp vault create 1password my-vault --op-vault Engineering # After swamp vault create @swamp/aws-sm my-vault --config '{"region":"us-east-1"}' swamp vault create @swamp/azure-kv my-vault --config '{"vault_url":"https://myvault.vault.azure.net/"}' swamp vault create @swamp/1password my-vault --config '{"op_vault":"Engineering"}' ``` ### Offline users Users without registry access can manually install extensions by placing the `.ts` source files in `extensions/vaults/`. ### Binary size The compiled binary no longer includes the AWS SDK, Azure SDK, or 1Password provider code. These dependencies are now bundled into the extensions at publish time. ## Known issues - Azure Key Vault extension bundle fails to load in compiled binary due to large bundle size (#733) - Auto-resolver fails when `extensions/models/` directory doesn't exist for vault-only extensions (#734) ## Verification - `deno check` — passes - `deno lint` — passes - `deno fmt` — passes - `deno run test` — 3138 passed, 0 failed - `deno run compile` — binary compiled successfully - Manual testing: auto-resolution verified for all three extensions (1password fails at `op` CLI check, aws-sm fails at credential check — both expected) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
## Summary Adds CLI commands to manage trusted collectives for extension auto-resolution, solving a discoverability problem where the feature was completely invisible to both users and AI agents. ### The Problem A user asked Claude about trusted collectives in their swamp repo, and Claude responded: *"The trusted collectives feature doesn't exist in swamp yet."* Despite the feature being fully implemented (PRs #725 and #727), it was only configurable by manually editing `.swamp.yaml` — no CLI command, no `--help` text, no discoverability path. ### The Solution Four new commands under `swamp extension trust`: ```bash swamp extension trust list # Show explicit, membership, and resolved collectives swamp extension trust add <collective> # Add a collective to the trusted list swamp extension trust rm <collective> # Remove a collective from the trusted list swamp extension trust auto-trust <on|off> # Enable/disable membership auto-trust ``` The `list` command shows the full picture — explicit collectives from `.swamp.yaml`, membership collectives from auth, and the resolved/merged effective list. This means even on a fresh repo with no config, a user sees that `swamp` and `si` are trusted by default. ### Architecture Follows the libswamp + renderer pattern (issue #739): | Layer | Files | Purpose | |-------|-------|---------| | **Shared types** | `src/libswamp/extensions/trust.ts` | `DEFAULT_TRUSTED`, `TrustModifyData`, `TrustModifyEvent`, `resolveTrustedCollectives()` | | **Generators** | `src/libswamp/extensions/trust_{list,add,rm,auto_trust}.ts` | Pure business logic with injected deps | | **Renderers** | `src/presentation/renderers/trust_{list,modify,auto_trust}.ts` | Log + Json output modes | | **CLI commands** | `src/cli/commands/extension_trust*.ts` | Pure wiring (deps → generator → renderer) | ### Refactoring - **Extracted `resolveTrustedCollectives()`** from `src/cli/mod.ts` into `src/libswamp/extensions/trust.ts` — it was domain logic living in the CLI layer - **Moved 8 tests** from `src/cli/mod_test.ts` to `src/libswamp/extensions/trust_test.ts` to follow the code - **Single `TrustModifyEvent`** shared by both `trust_add` and `trust_rm` generators (identical event shapes, no reason for separate types) - **`DEFAULT_TRUSTED`** defined once, used everywhere ### Documentation Updates - `design/extension.md` — documents CLI commands in "Trusted Collectives" section - `design/repo.md` — cross-references CLI commands from `trustedCollectives` config - 7 skill files updated with `swamp extension trust` references for discoverability ## Test Plan - [x] 26 generator tests (`src/libswamp/extensions/`) — all pass - [x] 16 renderer tests (`src/presentation/renderers/trust_*_test.ts`) — all pass - [x] 8 `resolveTrustedCollectives` tests moved to proper location — all pass - [x] Existing `mod_test.ts` tests still pass (57 tests, down from 65 after moving 8) - [x] `deno check` passes on all files - [x] `deno lint` passes - [x] `deno fmt` applied 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Summary
Closes #665
When users clone a repo with model or vault configurations referencing extension types that aren't installed locally, commands fail with cryptic "Unknown model type" or "Unsupported vault type" errors. This PR adds lazy auto-resolution: when swamp encounters an unknown type whose collective is on a trusted allowlist, it transparently searches the extension registry, installs the matching extension, hot-loads it into the live registries, and continues execution — no manual
swamp extension pullneeded.What changes
trustedCollectivesconfig in.swamp.yaml— defaults to["swamp", "si"]so official extensions auto-resolve out of the box. Users can add more collectives or set[]to disable.ExtensionAutoResolverdomain service — standalone service with port interfaces (ExtensionLookupPort,ExtensionInstallerPort,AutoResolveOutputPort) that keeps the domain layer clean of CLI/presentation imports.resolveModelType()/resolveVaultType()helper functions — drop-in replacements formodelRegistry.get()at 7 choke points across CLI commands and the workflow execution service.UserModelLoader.loadModels()andUserVaultLoader.loadVaults()gain askipAlreadyRegisteredoption so re-running model discovery after install doesn't error on already-registered types.VaultService.fromRepository()— resolves missing@-prefixed vault types beforeregisterVault(), keepingregisterVault()itself sync.Design decisions
Standalone helper, not embedded in registries —
ModelRegistryandVaultTypeRegistryremain pure sync data structures. Resolution is a domain service that choke points call explicitly. This is more DDD-aligned: the registry is a repository (stores/retrieves), resolution is a domain service (orchestrates).Port interfaces for clean architecture — The domain service defines
ExtensionLookupPort,ExtensionInstallerPort, andAutoResolveOutputPortinterfaces. Concrete adapters in the CLI layer wire the HTTP client,installExtension(), model loaders, and output renderers. This keeps domain → CLI/presentation dependency arrows pointing the right direction.Two-step type-to-extension resolution — First tries direct lookup by progressively stripping trailing segments (e.g.,
@swamp/aws/ec2/instance→ try@swamp/aws/ec2, then@swamp/aws). Falls back to search with collective filter if direct lookup fails. This handles both exact extension names and partial matches.Always install latest — Auto-resolution always installs the latest version unless the user has explicitly pinned via
extension pull @name@version. This is the right default for trusted collectives where you control releases.Re-entrancy guard — A
Set<string>of types currently being resolved prevents infinite loops if transitive dependencies trigger further resolution.Collective not allowlisted = silent skip — No auto-resolution is attempted for non-allowlisted collectives. The existing "Unknown model type" error shows as-is. This is intentional — we don't want to suggest the feature exists for untrusted collectives.
User impact
@swamp/digitalocean/droplet, runswamp model method run, and it just works — the extension installs automatically on first use.trustedCollectives: ["swamp", "si"]only kicks in for@swamp/*and@si/*types. Users who don't use extensions see no difference.trustedCollectives: []in.swamp.yamlto disable entirely.Testing
Automated tests (14 new)
@swamp/aws/ec2before@swamp/aws)notFoundoutput when nothing matchesnetworkErroroutput on fetch failures (TypeError, timeouts)@prefixed types (e.g.,swamp/echo/v2)resolveModelTypereturns existing definitions without resolverresolveModelTypereturns undefined for unknown types without resolverresolveVaultTypereturns true for existing vault typesresolveVaultTypereturns false for unknown types without resolverresolveVaultTypeskips non-@typesManual end-to-end test
Compiled the binary, initialized a fresh repo in
/tmp, and ran:Result: auto-resolved
@swamp/digitaloceanfrom the registry, installed@swamp/[email protected], hot-loaded 32 models, and successfully created themy-dropletdefinition — all in one command with clear status output:Full suite
All 3018 tests pass (14 new + 3004 existing), including architecture boundary and DDD layer rule tests.
Verification
deno check— passesdeno lint— passesdeno fmt— passesdeno run test— 3018 passed, 0 faileddeno run compile— binary compiled successfully🤖 Generated with Claude Code