Skip to content

fix: centralize S3 connection config to fix missing endpoint/forcePathStyle#917

Merged
stack72 merged 1 commit intomainfrom
push-uxklvmyksmpk
Mar 29, 2026
Merged

fix: centralize S3 connection config to fix missing endpoint/forcePathStyle#917
stack72 merged 1 commit intomainfrom
push-uxklvmyksmpk

Conversation

@adamhjk
Copy link
Copy Markdown
Contributor

@adamhjk adamhjk commented Mar 29, 2026

Summary

  • Introduce S3ConnectionConfig value object in the domain layer to
    centralize S3 connection parameters (bucket, prefix, region, endpoint,
    forcePathStyle) in one type
  • S3DatastoreConfig now extends S3ConnectionConfig, eliminating field
    duplication
  • Fix bug where createModelLock, acquireModelLocks, and
    createDatastoreLock silently dropped endpoint and forcePathStyle,
    breaking S3-compatible object stores (DigitalOcean Spaces, MinIO, etc.)
    for per-model locking and breakglass commands
  • Replace primitive-obsession signatures on S3DatastoreVerifier and
    DatastoreSetupDeps with S3ConnectionConfig, so adding a new
    connection parameter only requires updating one type

Test Plan

  • New tests: S3Client accepts S3ConnectionConfig, S3DatastoreConfig
    (structural subtyping), and minimal config
  • Existing setup, sync, and repo_context tests pass unchanged
  • Full test suite passes (3685 tests)
  • deno check, deno lint, deno fmt clean
  • deno run compile succeeds

🤖 Generated with Claude Code

…hStyle in per-model locks

## Summary

- Introduce `S3ConnectionConfig` value object in the domain layer to
  centralize S3 connection parameters (bucket, prefix, region, endpoint,
  forcePathStyle) in one type
- `S3DatastoreConfig` now extends `S3ConnectionConfig`, eliminating field
  duplication
- Fix bug where `createModelLock`, `acquireModelLocks`, and
  `createDatastoreLock` silently dropped `endpoint` and `forcePathStyle`,
  breaking S3-compatible object stores (DigitalOcean Spaces, MinIO, etc.)
  for per-model locking and breakglass commands
- Replace primitive-obsession signatures on `S3DatastoreVerifier` and
  `DatastoreSetupDeps` with `S3ConnectionConfig`, so adding a new
  connection parameter only requires updating one type

## Test Plan

- [x] New tests: S3Client accepts S3ConnectionConfig, S3DatastoreConfig
  (structural subtyping), and minimal config
- [x] Existing setup, sync, and repo_context tests pass unchanged
- [x] Full test suite passes (3685 tests)
- [x] `deno check`, `deno lint`, `deno fmt` clean
- [x] `deno run compile` succeeds

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLI UX Review

Blocking

None.

Suggestions

None.

Verdict

PASS — This PR is a pure internal refactoring with a bug fix. No commands, flags, help text, output formats, or error messages were changed. The only user-visible effect is that S3-compatible datastores (DigitalOcean Spaces, MinIO, etc.) now correctly pass endpoint and forcePathStyle through all code paths, fixing broken behavior for those users.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Blocking Issues

None.

Suggestions

  1. Minor: s3Conn variable in setup.ts:267 — The intermediate s3Conn variable explicitly lists all fields from input. Since input already has the right structural shape (it contains bucket, prefix, region, endpoint, forcePathStyle), you could potentially pass input directly, similar to how repo_context.ts passes config/datastoreConfig directly. However, the explicit construction does make the subset of fields being used clear, so this is purely a style preference — fine either way.

Summary

Clean, well-motivated PR. The S3ConnectionConfig value object is a textbook DDD improvement — it eliminates primitive obsession, centralizes connection parameters in an immutable value object, and fixes the real bug where endpoint/forcePathStyle were silently dropped in createModelLock, acquireModelLocks, and createDatastoreLock. The structural subtyping approach (passing S3DatastoreConfig where S3ConnectionConfig is expected) is idiomatic TypeScript and avoids unnecessary adapter code. Tests are properly placed and cover the key subtyping scenarios.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial Review

Critical / High

None.

Medium

None.

Low

  1. src/infrastructure/persistence/s3_client_test.ts: Tests only verify construction, not behavior. The three new tests assert assertExists(client) — i.e., that the constructor doesn't throw. They don't verify that endpoint and forcePathStyle are actually plumbed through to the underlying AwsS3Client. Since the whole point of this PR is fixing silently dropped connection params, a test that actually asserts the AWS SDK receives the correct config would be more valuable. However, this is mitigated by the fact that the constructor logic is straightforward (explicit field reads, not dynamic spreading), and the existing integration tests cover the real S3 paths.

  2. src/libswamp/datastores/setup.ts:267-273: s3Conn includes undefined optional fields. When input.prefix, input.region, input.endpoint, or input.forcePathStyle are undefined, the constructed S3ConnectionConfig object will have keys with undefined values (e.g., { prefix: undefined }) rather than absent keys. This is fine in practice — the S3Client constructor handles undefined correctly via ?? and conditional spreads — but it's a minor difference from passing a config that simply omits those keys. No behavioral impact.

Verdict

PASS. This is a clean, well-scoped fix. The core bug (silently dropping endpoint and forcePathStyle in createModelLock, acquireModelLocks, and createDatastoreLock) is real and correctly fixed. The new S3ConnectionConfig value object is a sound DDD refactoring — structural subtyping ensures S3DatastoreConfig works wherever S3ConnectionConfig is expected without any adapter code. No extra properties leak into the AWS SDK constructor since the S3Client class explicitly reads only the fields it needs. All callers have been updated consistently.

@stack72 stack72 merged commit 0e8efc6 into main Mar 29, 2026
10 checks passed
@stack72 stack72 deleted the push-uxklvmyksmpk branch March 29, 2026 23:16
stack72 added a commit that referenced this pull request Mar 30, 2026
…tension

Remove all built-in S3 datastore code from the core binary and delegate
S3 storage entirely to the `@swamp/s3-datastore` extension. This
follows the same pattern as PR #736 which moved vault providers to
extensions.

## What changed

### Built-in S3 code removed
- Deleted 6 infrastructure files: `s3_client.ts`, `s3_lock.ts`,
  `s3_cache_sync.ts`, `s3_datastore_verifier.ts`, and their tests
- Removed `S3DatastoreConfig`, `S3ConnectionConfig` from the type system
- Removed `@aws-sdk/client-s3` from `deno.json` (no longer bundled)
- Removed all `config.type === "s3"` branches from `repo_context.ts`,
  `resolve_datastore.ts`, `setup.ts`, `sync.ts`, `status.ts`
- Removed `swamp datastore setup s3` CLI command

### Generic extension setup command added
- New `swamp datastore setup extension <type> --config '{...}'` command
  works with any extension datastore, not just S3
- Validates config against extension schema, verifies health via
  provider, migrates data via sync service, updates `.swamp.yaml`
- Supports legacy type names: `swamp datastore setup extension s3 ...`
  auto-remaps to `@swamp/s3-datastore`

### Backward compatibility preserved
- `type: s3` in `.swamp.yaml` auto-remaps to `@swamp/s3-datastore`
  with deprecation warning (from Phase 1+2 in PR #910)
- `SWAMP_DATASTORE=s3:bucket/prefix` env var format still works
- Extension auto-resolves on first use for logged-in users

### Docs and skills updated
- `design/datastores.md`: S3 now documented as extension, not built-in
- `README.md`: Updated setup examples to use extension command
- `.claude/skills/swamp-repo/SKILL.md`: Updated command reference

## Design choices

### Why the provider pattern is better than built-in
The old code created `S3Client` at 6+ call sites in `repo_context.ts`,
manually destructuring config fields each time. PR #917 found that
`endpoint` and `forcePathStyle` were silently dropped at several sites,
breaking S3-compatible stores (DigitalOcean Spaces, MinIO). The
extension's provider pattern stores config once at construction and
passes it through everywhere — structurally immune to this class of bug.

### Sync behavioral differences
The built-in S3 path used `pullAll()`/`pushAll()` for sync operations.
The extension path uses `pullChanged()`/`pushChanged()` via the
`DatastoreSyncService` interface. The "changed" variants are better for
normal use (only transfer what's needed). For empty caches, they're
equivalent since everything is "changed". The built-in also had
`pullChangedForModel()` for per-model scoped pulls — an optimization
not available through the generic interface, acceptable trade-off for
now.

### `datastore status` no longer shows S3-specific fields
Bucket, prefix, region, endpoint are no longer displayed in
`swamp datastore status`. The generic extension interface doesn't
expose backend-specific details. The health check still works via the
provider's `createVerifier()`.

### Known limitation: offline + legacy S3 config
If a user has `type: s3` in `.swamp.yaml`, is offline, and the extension
isn't cached, `resolveDatastoreConfig` throws before any command runs.
The error message is clear ("Install it with: swamp extension pull
@swamp/s3-datastore"). This is an edge case — the extension is cached
after first use.

## Test plan

- [x] `deno check` — clean
- [x] `deno lint` — 910 files, clean
- [x] `deno fmt` — 998 files, clean
- [x] `deno run test` — 3672 passed, 0 failed
- [x] `deno run compile` — binary compiles without `@aws-sdk/client-s3`
- [x] Manual test: compiled binary against real S3-backed repo, extension
      auto-resolved and command completed cleanly
- [ ] CI passes

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants