smg turns source code into a semantic graph of modules, classes, functions, and their relationships. It already scans, queries, analyzes, diffs, exports, and enforces a useful baseline architecture graph for agents and humans.

Really, this is for LM agents -- it's supposed to give them better (and more efficient) eyes on the structure of your codebase, and allow them to poke around and prod it as a graph of semantic entities.

It's not ... say, Smalltalk ... but it'll have to do for today.

Warning: the PyPI package named smg is "Shepherd Model Gateway," an unrelated project. Do not run pip install smg or uv tool install smg without --from — you will get the wrong package. Always install from this git repo or a local checkout.

Install as a CLI tool (all supported languages):

uv tool install smg \
  --from git+https://github.com/femtomc/smg \
  --with tree-sitter \
  --with tree-sitter-python \
  --with tree-sitter-javascript \
  --with tree-sitter-typescript \
  --with tree-sitter-c \
  --with tree-sitter-zig \
  --with xxhash \
  --with watchdog

Python-only variant (smaller install):

uv tool install smg \
  --from git+https://github.com/femtomc/smg \
  --with tree-sitter --with tree-sitter-python --with xxhash

For development from a local checkout:

git clone https://github.com/femtomc/smg && cd smg
uv pip install -e ".[scan,dev]"

cd your-project
smg init
smg scan src/

# Ask questions
smg about MyClass           # What is this?
smg usages MyClass          # Where is it used?
smg impact MyClass          # What breaks if I change it?
smg between api.routes db   # How do these relate?

# Analyze
smg analyze                 # Architectural analysis with hotspot detection
smg diff                    # What changed? (with rename/move detection)
smg export dsm > deps.csv   # Dependency Structure Matrix export
smg blame MyClass           # Who last touched this?
smg context MyClass --tokens 8000  # Pack source for an LLM prompt

# Enforce
smg rule add acyclic --invariant no-cycles
smg check                   # Enforce architectural rules

This release ships the baseline architecture workflow plus three additions that used to be release candidates:

• Quantified rules via smg rule add --forall ... --assert ...
• Declared concept/group analysis via smg concept ... and smg analyze --concepts
• Minimal witnesses in smg check text and JSON output

The main remaining limitation is explicit: concept analysis is an analysis surface, not yet a smg check invariant. smg analyze --concepts reports unsanctioned cross-concept edges, but it does not fail CI on its own.

Run these commands from the repo root. --with-editable . keeps the CLI and imports pointed at your checkout while uv resolves the required extras on demand.

# Test suite
uv run --with-editable . --extra dev --extra scan python -m pytest -q

# Type checking
uv run --with-editable . --extra dev --extra scan python -m pyright

The reproducible validation command for code changes in this repository is:

uv run --with-editable . --extra dev --extra scan python -m pytest -q

When you want to dogfood the architecture workflow on this repository itself, refresh the local graph and run:

uv run --with-editable . --extra dev --extra scan python -m smg scan src tests --clean
uv run --with-editable . --extra dev --extra scan python -m smg check --format json
uv run --with-editable . --extra dev --extra scan python -m smg analyze --concepts --summary --format json

Concept declarations are stored in .smg/concepts when you add them locally. If that file is absent, smg analyze --concepts reports an empty declaration set instead of mutating .smg/graph.jsonl.

smg stores a graph of code entities in .smg/graph.jsonl at your project root. Each line is a node (module, class, function, ...) or a typed edge (contains, calls, imports, inherits, ...).

There are three ways to populate the graph:

1. smg scan -- tree-sitter parses source files and extracts symbols, containment, imports, inheritance, and call graph automatically.
2. Manual CLI -- agents or humans add nodes and edges directly with smg add and smg link.
3. Both -- scan for the baseline, then layer on domain-specific relationships (e.g. "tests", "endpoint", custom types).

Every node and edge is tagged with source: "scan" or source: "manual". When rescanning, only scan-sourced nodes are cleaned -- manual annotations survive. If a rescan deletes a node that had manual edges, those orphaned edges are reported.

All languages extract: classes/structs, functions, methods, constants, containment, imports, inheritance, call graph, and per-function metrics. Adding a language means writing a langs/<language>.py extractor and a BranchMap -- the metrics engine and scanner are shared.

Source code has two kinds of identity: its exact text (what you see in a file), and its structure (what the compiler sees after discarding names, whitespace, and comments). smg computes both during extraction and stores them on every function, method, and class node:

• Content hash -- xxHash64 of the entity's raw source bytes. Two entities with the same content hash have identical source code, byte for byte.
• Structure hash -- xxHash64 of a normalized AST walk that strips comments, identifiers, and literals but preserves node types and nesting. Two entities with the same structure hash have the same control flow shape -- the same branches, loops, and call structure -- even if variable names, strings, or comments differ.

This separation is the key design decision. A content hash tells you "these are the same code." A structure hash tells you "these do the same thing, structurally." The gap between them is where renames, comment edits, and cosmetic refactors live.

The Unison language pioneered the idea that code should be identified by the hash of its AST, not by its name. In Unison, a function's hash is its identity -- names are just human-readable labels in a separate namespace. This makes rename refactoring free (renaming updates a mapping, not code) and eliminates rebuilds (cache everything by hash, invalidate nothing unless the definition changes). The idea is described in Unison's documentation and draws on de Bruijn's name-free representation of bound variables (de Bruijn, 1972).

smg applies the same core insight -- structural identity independent of names -- but in a different context and at a different fidelity:

• Unison hashes for identity. The hash is authoritative and permanent. It includes types, de Bruijn indices for local variables, and recursive hashes of all dependencies. Unison can do this because it controls the compiler and type system.
• smg hashes for detection. The hash is a heuristic for matching entities across graph snapshots. It includes AST node types and nesting structure but not types or dependency hashes, because smg operates as an external analysis tool across languages without full semantic analysis.
• smg keeps both hashes. Unison collapses "same code" and "same structure, different names" into one hash (both are the same identity). smg distinguishes them because smg diff wants to report that a rename happened -- the content hash says "the code changed," the structure hash says "but the shape didn't," and together they tell you it was a rename.

Structural hashing powers three things:

1. Rename/move detection in smg diff -- when a function disappears and a structurally identical one appears elsewhere, they are matched as a rename rather than a deletion + addition (see Structural diff).
2. Change classification -- same content hash means no real change (metadata-only update). Same structure hash with different content hash means a cosmetic change (rename, comment edit). Different structure hash means a real structural change.
3. Stability tracking -- structure hashes persist in the graph across scans, so you can track which entities have structurally changed over time, independent of formatting or naming churn.

smg analyze runs graph-theoretic, OO, smell-detection, and git-churn analyses in a single pass, then synthesizes a ranked list of hotspots.

smg analyze                        # all analyses
smg analyze --module auth          # scope to auth.* nodes
smg analyze --since HEAD~5         # only analyze changed nodes
smg analyze --summary --top 5      # hotspots and key findings only
smg analyze --churn-days 180       # look back 6 months for churn (default: 90)
smg analyze --format json          # structured output for agents

The CK suite (Chidamber & Kemerer, 1994) and Martin's package metrics (Martin, 1994) quantify class-level and module-level design quality.

Every function and method node includes AST-based metrics in its metadata, computed automatically during scan:

Language-agnostic -- metrics are computed from tree-sitter ASTs using a per-language BranchMap that maps node types to semantic roles.

All analyses feed into a composite hotspot ranking that scores nodes by a weighted combination of complexity (WMC, cyclomatic complexity), coupling (CBO, fan-out), cohesion (LCOM4), centrality (betweenness, PageRank), churn (commit touch frequency), and module distance from the main sequence. The resulting ranked list surfaces the areas most likely to cause problems when modified.

smg integrates structural graph analysis with git history to track how code changes over time.

smg diff compares the current graph against a git ref and reports structural changes: added, removed, changed, and renamed/moved nodes and edges.

smg diff              # vs HEAD (last commit)
smg diff HEAD~3       # vs 3 commits ago
smg diff main         # vs another branch

Rename and move detection uses a three-phase matching algorithm on unmatched added/removed nodes:

1. Content hash match -- if a removed node and an added node have identical source bytes (same content hash), the entity was purely renamed or moved with no code changes. Reported as an "exact" match.
2. Structure hash match -- if there's a unique structural match (same AST shape but different content, e.g. a variable was renamed inside the function), the entity was renamed with minor edits. Reported as a "structural" match.
3. Fuzzy name similarity -- for still-unmatched entities of the same type, smg computes Jaccard similarity on whitespace-split tokens of their fully-qualified names. Pairs above 0.8 similarity are classified as renamed/moved. This catches cases where both the entity's name and its body changed significantly but the name tokens mostly overlap (e.g. app.utils.parse_config -> app.helpers.parse_config).

Anything unmatched after all three phases is reported as a plain addition or deletion.

smg analyze integrates temporal data from git history. For each entity in the graph, it counts how many commits modified the entity's line range over a configurable time window (default: 90 days).

Entities that are both structurally central (high complexity, high coupling) AND frequently changed are the most dangerous hotspots -- they are hard to modify correctly AND are being modified often. Churn feeds into the hotspot ranking at both class level (> 10 touches) and function level (> 5 touches with cyclomatic complexity > 10).

smg analyze --churn-days 30   # last month only
smg analyze --churn-days 365  # full year

smg blame maps graph entities to the most recent git commit that touched their source lines:

# Single entity
smg blame SemGraph
# smg.graph.SemGraph [class]
#   src/smg/graph.py:14-251
#   69a55085a058 [email protected] (2024-01-15)
#   Optimize scan and analysis hot paths

# All entities in a file
smg blame src/smg/graph.py

For a single entity, it runs git log -1 -L <start>,<end>:<file> to find the commit that last modified the entity's line range. For a file, it blames every entity in that file and displays a table sorted by line number. Output is JSON when piped.

When preparing a prompt for an LLM, the question is: which source code should go in, given a limited context window? smg context uses the dependency graph to answer this -- it packs relevant source code into a token budget, prioritizing by graph proximity to a target entity.

smg context auth.service --tokens 8000

The algorithm walks outward from the target:

1. Target entity -- full source (always included, even if it exceeds the budget alone)
2. Direct dependencies and dependents (1-hop) -- full source, downgraded to signature if over budget
3. 2-hop neighbors -- signature only (the function/class declaration line)
4. 3-hop neighbors -- summary only (name, type, file location, docstring)

If the budget fills up at any tier, remaining entries are downgraded to the next level. Output is structured JSON when piped, syntax-highlighted source when in a terminal. The token counter defaults to len(text) / 4 (~4 chars per token) but is pluggable via the library API.

smg analyze tells you what is true about your architecture. smg rule and smg check let you declare what should be true -- and find where the code departs from intent.

This is inspired by Alloy's approach to software design (Jackson, 2012): state structural properties declaratively, then check them automatically. When a rule is violated, smg reports the specific offending edges or nodes -- concrete counterexamples, not just "violated: true."

Forbid coupling edges that match a glob pattern:

# No module in infra/ may depend on app/
smg rule add layering --deny "infra.* -> app.*"

# Controllers must not directly call repository code
smg rule add no-direct-db --deny "*.controller -[calls]-> *.repository"

Patterns use fnmatch syntax over fully-qualified node names. The optional [rel] filter restricts to a specific relationship type; without it, all coupling edges (calls, imports, inherits, implements, depends_on) are checked.

smg rule add acyclic --invariant no-cycles
smg rule add reachable --invariant no-dead-code --entry-points "main,cli.*"
smg rule add layered --invariant no-layering-violations

smg rule add service-fan-out --forall "*.service" --assert "fan_out <= 5"
smg rule add simple-handlers --forall "api.handlers.*" --assert "cyclomatic_complexity <= 10"

Quantified rules match subjects by glob and evaluate a small expression language over graph metrics, OO metrics, and scanner metadata. The current release supports per-subject universal checks; it does not yet support graph-wide aggregates such as count(...) or sum(...).

smg check                  # check all rules
smg check layering         # check one rule
smg check --format json    # structured output for agents / CI

Exit code 0 means all rules pass; exit code 1 means at least one violation. This makes smg check suitable as a CI gate:

smg scan src/ --clean && smg check

Failures include a witnesses list in JSON and show the first few witnesses in text output. Deny and layering rules emit edge witnesses, cycle rules emit minimal cycles, and quantified rules emit predicate witnesses with the observed metric facts.

Rules are stored as JSONL records in .smg/rules. The file is created on first use and stays separate from .smg/graph.jsonl.

Declare higher-level module groups in a sidecar store, then lift the dependency graph to concept-level summaries and cross-concept witnesses.

smg concept add cli --prefix smg.cli
smg concept add core --prefix smg.graph --prefix smg.model
smg concept add core-api --prefix smg.api --sync-point smg.api.boundary
smg concept list
smg analyze --concepts

Concept declarations live in .smg/concepts, not in .smg/graph.jsonl. Use --sync-point to mark node-name prefixes that are allowed to cross a concept boundary. The concepts section is added to smg analyze --format json only when --concepts is requested.

Agents should treat smg as a codebase database. The typical workflow:

# Orient
smg overview                           # graph stats, top connected nodes
smg about auth.service                 # context card for a node

# Investigate
smg usages auth.service                # who uses this?
smg impact auth.service                # what depends on this?
smg between api.routes db.models       # how do these connect?
smg diff                               # what changed since last commit?
smg blame auth.service                 # who last touched this?
smg analyze --summary --top 5          # key architectural findings
smg query deps auth.service            # forward transitive dependencies

# Build LLM context
smg context auth.service --tokens 8000 # source packed by graph proximity

# Enforce
smg rule add layering --deny "infra.* -> app.*"
smg check                              # exit 1 on violation

# Mutate
smg add endpoint /api/login --doc "Login endpoint" --meta method=POST
smg link api.routes calls auth.service
smg scan --changed                     # incremental rescan

Output is automatically JSON when piped, rich text in terminal. No flags needed.

package, module, class, function, method, interface, variable, constant, type, endpoint, config -- plus any custom string.

calls, inherits, implements, contains, depends_on, imports, returns, accepts, overrides, decorates, tests -- plus any custom string.

.smg/graph.jsonl -- one JSON object per line:

{"kind":"node","name":"app.core.Engine","type":"class","file":"src/app/core.py","line":12,"metadata":{"source":"scan","content_hash":"a1b2c3d4e5f6a7b8","structure_hash":"f8e7d6c5b4a39281","metrics":{...}}}
{"kind":"edge","source":"app.core","rel":"contains","target":"app.core.Engine","metadata":{"source":"scan"}}

Rules and concept declarations use separate JSONL sidecars, created on first use:

{"kind":"rule","name":"service-fan-out","type":"quantified","selector":"*.service","assertion":"fan_out <= 5"}
{"kind":"concept","name":"cli","prefixes":["app.cli"],"sync_points":["app.cli.surface"]}

Keeping rules and concepts out of .smg/graph.jsonl preserves graph counts and analyses across rescans. All three stores are human-readable and written atomically via temp file + rename.

Node names are fully qualified (app.core.Engine.run), but you can use short names:

smg about Engine          # Matches app.core.Engine if unambiguous
smg show run              # Error if multiple matches -- lists candidates

smg scan skips common non-source directories by default (.git, node_modules, __pycache__, .venv, etc.). Extend with --exclude GLOB flags or a .smgignore file at the project root (gitignore syntax).

• Agent-first: JSON by default when piped, structured output, exit codes for branching
• Gradual disclosure: about -> show -> query -- start simple, drill down as needed
• Language-agnostic: tree-sitter grammars for any language, BranchMap protocol for metrics
• Incremental: --changed rescans only modified files, watch for live updates
• Provenance-aware: scan vs manual annotations tracked, manual edges survive rescans
• Zero config: smg init && smg scan . works on any supported project
• Git-friendly: JSONL is diffable, sorted deterministically, written atomically

• de Bruijn, N.G. (1972). Lambda Calculus Notation with Nameless Dummies. Indagationes Mathematicae, 75(5), 381--392.
• McCabe, T.J. (1976). A Complexity Measure. IEEE TSE, SE-2(4), 308--320.
• Tarjan, R.E. (1972). Depth-First Search and Linear Graph Algorithms. SIAM Journal on Computing, 1(2), 146--160.
• Seidman, S.B. (1983). Network Structure and Minimum Degree. Social Networks, 5(3), 269--287.
• Chidamber, S.R. & Kemerer, C.F. (1994). A Metrics Suite for Object Oriented Design. IEEE TSE, 20(6), 476--493.
• Martin, R.C. (1994). OO Design Quality Metrics: An Analysis of Dependencies. OOPSLA '94 Workshop.
• Hitz, M. & Montazeri, B. (1995). Measuring Coupling and Cohesion in Object-Oriented Systems. Proc. ISACC '95.
• Brin, S. & Page, L. (1998). The Anatomy of a Large-Scale Hypertextual Web Search Engine. Computer Networks, 30(1--7), 107--117.
• Fowler, M. (1999). Refactoring: Improving the Design of Existing Code. Addison-Wesley.
• Brandes, U. (2001). A Faster Algorithm for Betweenness Centrality. Journal of Mathematical Sociology, 25(2), 163--177.
• Jaccard, P. (1912). The Distribution of the Flora in the Alpine Zone. New Phytologist, 11(2), 37--50.
• Jackson, D. (2012). Software Abstractions: Logic, Language, and Analysis. MIT Press.
• Campbell, G.A. (2018). Cognitive Complexity: An Overview and Evaluation. Proc. TechDebt '18, ACM.

MIT