Skip to content

Latest commit

 

History

History
317 lines (226 loc) · 19.3 KB

File metadata and controls

317 lines (226 loc) · 19.3 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.3.4] - 2026-04-15

Changed

  • Repository maintenance updates committed from the current working tree.
  • Version-bearing manifests synchronized to 0.3.4.

[0.3.3] — 2026-04-05

Fixed

  • tests/test_live.sh: Added uid=1000,gid=1000 to the --tmpfs /data mount so the syslog container user (uid 1000) can write the SQLite database; previously the tmpfs was owned by root, causing unable to open database file: /data/syslog.db and CI health-check timeout
  • .github/workflows/docker-publish.yml: Trivy scan now references steps.meta.outputs.version (e.g. main) instead of the bare github.sha (full 40-char SHA); the image is pushed with the branch/tag name, not the full commit SHA, so the old ref caused MANIFEST_UNKNOWN scan failures

[0.3.2] — 2026-04-04

Fixed

  • Version sync: Aligned .claude-plugin/plugin.json, .codex-plugin/plugin.json, and gemini-extension.json from 0.2.6 to 0.3.2 to match Cargo.toml

Added

  • tests/TEST_COVERAGE.md: Test coverage documentation
  • tests/mcporter/: MCPorter test infrastructure
  • Full documentation structure: Added plugin-lab template docs (README, CLAUDE.md, references, runbooks)

[0.3.1] — 2026-04-04

Fixed

  • src/config.rs: cleanup_chunk_size upper bound replaced from i64::MAX with operational limit of 1_000_000; values above this hold the SQLite write lock indefinitely. Error message now explains why the limit exists.

[0.3.0] — 2026-04-04

Added

  • src/db.rs: DbStats now includes phantom_fts_rows — count of FTS5 index entries without a matching live log row (merge lag indicator, visible via get_stats)
  • src/db.rs: schema_migrations table guards idempotent migrations; DROP TRIGGER migration now runs exactly once per database (version 1)
  • src/db.rs: Composite index idx_logs_hostname_received_at ON logs(hostname, received_at) — makes reconcile_hosts MIN/MAX queries O(1) instead of O(rows_per_host)
  • CLAUDE.md: FTS5 phantom row gotcha with GDPR/HIPAA compliance guidance

Fixed

  • src/db.rs: P1 — delete_oldest_logs_chunk rewritten with subquery DELETE; old dynamic IN-list exceeded SQLite's 1000-node expression depth limit at default cleanup_chunk_size=2000, silently failing every storage enforcement cycle
  • src/db.rs: P1 — fts_incremental_merge now runs ceil(deleted_rows/5000) iterations capped at 20; escalates to forced rebuild after 3 consecutive failures
  • src/db.rs: reconcile_hosts moved outside the enforcement delete loop — batches all host updates to one call per enforcement cycle instead of one per chunk
  • src/syslog.rs: TCP rejection warn! is now rate-limited (once on first rejection, once per 10s thereafter with total_rejected count) to prevent log storms under connection floods

Changed

  • src/config.rs: StorageConfig::for_test demoted to pub(crate)
  • src/mcp.rs: TestHarness struct wraps AppState + TempDir in tests to prevent accidental early TempDir drop; all 10 test functions updated
  • src/mcp.rs: test_storage_config wrapper removed; callers use StorageConfig::for_test directly

[0.2.6] — 2026-04-04

Changed

  • src/db.rs: Extracted fts_incremental_merge() helper — eliminates duplicated FTS merge string across purge_old_logs and enforce_storage_budget
  • src/mcp.rs: test_state() now delegates to test_state_with_token(None); mcp_post() gains optional auth param — auth integration tests no longer inline the request builder

Fixed

  • src/config.rs: Added accepts_cleanup_chunk_size_at_i64_max boundary test; tightened overflow test to assert error message; added SYSLOG_MCP_CLEANUP_CHUNK_SIZE to defaults_are_applied_without_env_vars clear list and assertion
  • src/db.rs: Migrated test_storage_config() to StorageConfig::for_test()
  • src/syslog.rs: TryAcquireError::Closed branch now logs at error! before breaking
  • CHANGELOG.md: Corrected v0.2.2 date (2026-04-042026-04-03)

[0.2.5] — 2026-04-03

Added

  • src/mcp.rs: 9 HTTP-level integration tests for all 6 MCP tools and auth middleware using tower::util::ServiceExt::oneshot — covers health endpoint, initialize, tools/list, get_stats, tail_logs, search_logs, unknown method error, auth rejection (missing token), and auth success (correct token)
  • Cargo.toml: tower 0.5 added to dev-dependencies for axum router integration testing

[0.2.4] — 2026-04-03

Fixed

  • src/db.rs: FTS5 write-lock contention during retention purge and storage-budget bulk deletes — removed logs_ad (AFTER DELETE) and logs_au (AFTER UPDATE) triggers that fired per-row inside 10k-chunk transactions, starving the batch writer. Added migration to drop triggers from existing databases. FTS5 phantom rows are cleaned up by incremental merge (syslog-mcp-eg5)

Added

  • src/db.rs: Incremental FTS merge (merge=500,250) after storage-budget enforcement bulk deletes, matching the existing purge_old_logs pattern

[0.2.3] — 2026-04-03

Fixed

  • src/syslog.rs: TCP accept loop blocked when connection semaphore was at capacity — replaced blocking acquire_owned().await with non-blocking try_acquire_owned() so the accept loop rejects new connections immediately instead of stalling for up to 300s (idle timeout)

[0.2.2] — 2026-04-03

Fixed

  • src/mcp.rs: summarize_json_value panicked on multi-byte UTF-8 input (non-ASCII syslog messages) — replaced &raw[..limit] with a char-boundary-aware walk-back loop; added test covering Greek/CJK input
  • src/db.rs: Storage enforcement deleted 1 row per cycle (extremely slow for large overages) — now configurable via SYSLOG_MCP_CLEANUP_CHUNK_SIZE (default 2000); WAL checkpoint moved outside the recovery loop
  • src/config.rs: Added validation rejecting cleanup_chunk_size == 0 (would cause an infinite enforcement loop)
  • Clippy: Fixed 4 -D warnings errors blocking cargo testderivable_impls on Config::Default, match_like_matches_macro in is_transient_sqlite_lock, needless_late_init for close_reason, len_zero in batch_writer

Added

  • src/config.rs: cleanup_chunk_size field in StorageConfig with env var SYSLOG_MCP_CLEANUP_CHUNK_SIZE (default 2000 rows per enforcement chunk)
  • src/config.rs: #[cfg(test)] StorageConfig::for_test() constructor — centralizes test config; mcp.rs and syslog.rs test helpers now delegate to it
  • docs/sessions/2026-04-03-mcp-test-code-review-simplify.md: Full session documentation

[0.2.1] - 2026-04-03

Fixed

  • OAuth discovery 401 cascade: BearerAuthMiddleware was blocking GET /.well-known/oauth-protected-resource, causing MCP clients to surface generic "unknown error". Added WellKnownMiddleware (RFC 9728) to return resource metadata.

Added

  • docs/AUTHENTICATION.md: New setup guide covering token generation and client config.
  • README Authentication section: Added quick-start examples and link to full guide.

[0.2.0] — 2026-03-31

Added

  • docker-compose.yml / .env.example / README.md: SYSLOG_UID and SYSLOG_GID env vars — container now runs as a configurable user/group (default 1000:1000) for bind-mounted data directories
  • src/db.rs: StorageBudgetState struct — write-blocked flag and storage metrics shared via Arc<Mutex<>> across syslog and MCP modules
  • src/db.rs: Transient SQLite lock retry in batch insert — 3-attempt backoff (25/100/250ms) on SQLITE_BUSY / SQLITE_LOCKED before failing
  • src/db.rs: configure_connection_pragmas() helper — WAL mode and PRAGMA setup extracted from init_pool so every pooled connection is configured consistently
  • src/main.rs: Initial storage budget enforcement check on startup before accepting syslog traffic
  • src/main.rs: background_interval() helper — interval fires after the first period (not at t=0), preventing a burst on startup
  • src/syslog.rs: start_with_storage_state() replaces start() — storage state shared with batch writer for write-blocking under pressure

Fixed

  • src/syslog.rs: TCP handler now enforces max_message_size per line instead of per connection — persistent forwarders (rsyslog, syslog-ng) that reuse a single TCP session no longer hit the connection-level byte limit and cause an OOM/disconnect
  • src/mcp.rs: Auth rejection now logs method, path, and has_auth_header for diagnostics

Changed

  • src/db.rs, src/main.rs, src/mcp.rs, src/syslog.rs: Structured tracing fields added throughout — storage enforcement, batch insert, retention purge, TCP/UDP listeners, health check, and MCP request lifecycle all emit structured events
  • .dockerignore: Reorganized with categorized sections; AI tooling dirs (.claude, .omc, .lavra, .beads) explicitly excluded
  • .gitignore: Reorganized with categorized sections; editor dirs, cache, doc artifacts, and worktree dirs added

[0.1.9] — 2026-03-30

Changed

  • Breaking: env var rename — dropped figment's nested SYSLOG_MCP_SECTION__KEY format for flat SYSLOG_* and SYSLOG_MCP_* prefixes. See .env.example for the new names.
  • src/config.rs: Replaced figment with toml crate + manual env var overlay — simpler, supports two prefixes
  • src/config.rs: Merged udp_bind/tcp_bind into host + port (UDP and TCP always share the same address)
  • src/config.rs: Renamed flush_interval_ms to flush_interval
  • docker-compose.yml: Host-side ports use ${SYSLOG_PORT} and ${SYSLOG_MCP_PORT} env vars
  • docker-compose.yml: Data volume uses ${SYSLOG_MCP_DATA_VOLUME} (defaults to named volume syslog-mcp-data)
  • docker-compose.yml: Replaced environment: block with env_file: .env
  • docker-compose.yml: Removed SWAG labels; network uses external: true
  • Dockerfile: SYSLOG_MCP_STORAGE__DB_PATHSYSLOG_MCP_DB_PATH
  • Cargo.toml: figment dependency replaced with toml

Added

  • src/config.rs: SyslogConfig::bind_addr() and McpConfig::bind_addr() helper methods
  • src/config.rs: validate_host() rejects host strings containing ports
  • src/config.rs: 2 new tests — env_var_overrides_syslog_port, host_with_port_is_rejected

0.1.7 — 2026-03-30

Fixed

  • src/db.rs: Retention purge now uses received_at (server clock) instead of timestamp (device clock) — prevents misconfigured device clocks from causing immediate purge or infinite retention (syslog-mcp-x6l)
  • src/db.rs: Added composite (severity, timestamp) index for get_error_summary query performance (syslog-mcp-ctj)
  • src/db.rs: std::collections::HashMap imported at module level instead of inline paths (syslog-mcp-rva)
  • src/mcp.rs: /health endpoint now runs SELECT 1 instead of COUNT(*) over entire logs table (syslog-mcp-068)
  • src/mcp.rs: severity_to_num moved to db.rs as single source of truth (syslog-mcp-nu6)
  • src/mcp.rs: 401 response uses JSON-RPC 2.0 envelope; replaced futures crate with futures-core (syslog-mcp-zr4)
  • src/syslog.rs: TCP accept error now uses exponential backoff (100ms → 5s cap) instead of flat 100ms sleep (syslog-mcp-ve1)
  • src/syslog.rs: looks_like_timestamp now validates digit positions, not just separator offsets (syslog-mcp-qus)
  • src/syslog.rs: Removed false "octet-counting" claim from TCP listener doc comment (syslog-mcp-jsv)
  • src/syslog.rs: Flush retry adds 250ms pause to avoid hammering a failing DB (syslog-mcp-rjt)
  • src/config.rs: Renamed parse_addr to validate_addr for clarity (syslog-mcp-e5m)
  • bin/smoke-test.sh: assert_no_error now fails on non-JSON output instead of silently passing (syslog-mcp-tef)
  • Cargo.toml: Removed unused ws feature from axum; removed unused json feature from tracing-subscriber (syslog-mcp-3ou, syslog-mcp-avg)
  • docker-compose.yml: SWAG labels updated to swag=enable + url/port/proto format (syslog-mcp-j4m)

Added

  • src/db.rs: PRAGMA wal_checkpoint(PASSIVE) after hourly purge to prevent unbounded WAL growth (syslog-mcp-dah)
  • src/db.rs: pub fn severity_to_num() for reuse across modules (syslog-mcp-nu6)
  • src/config.rs: batch_size and flush_interval_ms fields in SyslogConfig with serde defaults (syslog-mcp-7uv)
  • src/db.rs: 4 new unit tests — timestamp range filtering, severity_to_num edge cases, error summary severity filter, severity_in filter (syslog-mcp-063, syslog-mcp-v9r, syslog-mcp-3su, syslog-mcp-94p)
  • bin/backup.sh: WAL-safe SQLite backup script with cron scheduling and 30-day pruning (syslog-mcp-8zi)
  • docs/runbooks/deploy.md: Rolling update, rollback, health check, and pre-deploy checklist (syslog-mcp-8np)
  • .env.example: Added max_message_size, batch_size, flush_interval_ms documentation (syslog-mcp-vri)
  • README.md: SSE endpoint stub behavior documented; Docker network prereq documented (syslog-mcp-3t7, syslog-mcp-7r4)
  • CLAUDE.md: CEF hostname trust boundary, batch writer failure path, correlate_events 999 limit cap documented (syslog-mcp-dum, syslog-mcp-2oj, syslog-mcp-y1n)

0.1.6 — 2026-03-30

Security

  • src/main.rs: Redact api_token from startup log — log individual fields with auth_enabled=bool instead of printing full config struct (syslog-mcp-4yw)
  • src/mcp.rs: Add optional Bearer token auth middleware; restrict CORS to localhost origins only (syslog-mcp-gm3)

Fixed

  • Dockerfile: Fix ENV SYSLOG_MCP__STORAGE__DB_PATHSYSLOG_MCP_STORAGE__DB_PATH — double-underscore prefix was silently ignored by figment (syslog-mcp-s9b)
  • src/syslog.rs: Drop TCP lines exceeding max_message_size to prevent OOM from unbounded lines (syslog-mcp-zu9)
  • src/syslog.rs: Warn when CEF heuristic fires but all fields extract as None — malformed CEF body now emits a log line instead of silently falling back (syslog-mcp-w5e)
  • src/syslog.rs: Cap TCP connections at 512 with semaphore + 300s wall-clock timeout per connection (syslog-mcp-ct2)
  • src/db.rs: Chunked DELETE + incremental FTS merge to release WAL write-lock during retention purge (syslog-mcp-75i)
  • src/config.rs: Replace blocking to_socket_addrs() DNS call with non-blocking SocketAddr::parse() at config load time
  • Dockerfile: Run container as non-root user uid/gid 10001 (syslog-mcp-ab8)
  • .lavra/memory/recall.sh: Remove stray local keyword outside function scope (syslog-mcp-1mg)

Added

  • .github/workflows/ci.yml: GitHub Actions CI — fmt check, clippy -D warnings, test, cargo audit (syslog-mcp-7ee)
  • src/db.rs: 7 unit tests covering insert, FTS search, severity filter, purge, stats, host aggregation (syslog-mcp-sd0)
  • .env.example: Document SYSLOG_MCP_MCP__API_TOKEN bearer token option

0.1.5 — 2026-03-28

Fixed

  • syslog.rs: Normalize stored timestamps to UTC (dt.with_timezone(&Utc)) — mixed-offset sources no longer misorder SQLite rows or break retention purges
  • smoke-test.sh: --url flag now creates a temp mcporter config so health checks and tool calls always target the same server; guard $2 dereference under set -u; fix limit=0 boundary test that was silently passing limit=1
  • recall.sh: Fix --all --recent ordering (archive first → newest entries last in tail); use grep -F for literal bead matching; fix auto-build to source + kb_sync (subprocess call was a no-op)
  • knowledge.jsonl: Strip embedded shell command fragments from content and bead fields

Changed

  • knowledge-db.sh: Quoted temp file path in sqlite3 .import; consolidated 7→1 jq invocations per JSONL line and 2→1 per beads-import row
  • .gitignore: Narrow *.db to data/*.db to avoid hiding fixture files
  • README.md / CLAUDE.md: Correct env var prefix SYSLOG_MCP__SYSLOG_MCP_
  • docker-compose.yml: Switch network from internal syslog-mcp to external jakenet
  • Session docs: blank lines after subsection headings; complete rollback command

0.1.4 — 2026-03-28

Added

  • Session docs for syslog host onboarding (tootie, dookie, squirts, steamy-wsl, vivobook-wsl) and systemd service cleanup

0.1.3 — 2026-03-28

Fixed

  • Clippy type_complexity errors: introduced LogBatchEntry type alias for the 8-field batch tuple (src/db.rs, src/syslog.rs)
  • ORDER BY timestampORDER BY l.timestamp for consistency with table alias in non-FTS search path
  • #[allow(dead_code)]#[expect(dead_code, reason = "...")] on jsonrpc field for self-cleaning lint suppression

Changed

  • Removed single-insert insert_log in favour of batch-only path via insert_logs_batch
  • search_logs non-FTS path now uses FROM logs l alias, consistent with the FTS join path
  • syslog_loose::parse_message updated to explicit Variant::Either API; timestamp handling simplified from 5-arm IncompleteDate match to direct dt.to_rfc3339()
  • Removed unused imports (NaiveDateTime, StreamExt, error/info from tracing, uuid, thiserror, axum-extra, tower)
  • Removed dead idx += 1 at end of tail_logs

0.1.2 — 2026-03-27

Added

  • Project documentation (SETUP.md, docs/)
  • Lavra project config and codebase profile (.lavra/)
  • Beads issue tracking init (.beads/)
  • Session doc for 2026-03-27 repo init and restructure

Changed

  • Updated Rust base image in Dockerfile

Fixed

  • Removed root-level source files after src/ migration (duplicate artifact cleanup)

0.1.1 — 2026-03-27

Changed

  • Restructured project to standard Rust layout (src/ modules)
  • Migrated flat source files into src/config.rs, src/db.rs, src/mcp.rs, src/syslog.rs, src/main.rs

0.1.0 — 2026-03-27

Added

  • Initial release: syslog receiver + MCP server in Rust
  • UDP + TCP syslog listeners on port 1514 (RFC 3164 / RFC 5424 / loose via syslog_loose)
  • SQLite storage with FTS5 full-text index, WAL mode, and hourly retention purge
  • Six MCP tools over JSON-RPC 2.0 (POST /mcp):
    • search_logs — FTS5 search with host/severity/app/time filters
    • tail_logs — most recent N entries
    • get_errors — error/warning summary grouped by host and severity
    • list_hosts — all known hosts with first/last seen and log counts
    • correlate_events — cross-host event correlation in a time window
    • get_stats — DB stats (total logs, size, time range)
  • SSE endpoint (GET /sse) for legacy MCP transport
  • Health check endpoint (GET /health)
  • figment-based config (config.toml + SYSLOG_MCP_ env vars)
  • Docker Compose deployment with bind-mounted ./data/ volume
  • Batch writer with mpsc channel, 100-entry batches, 500ms flush interval