Skip to content

[2/3] Refactor ModelConfiguration into a slot-aware component that auto-derives labels, descriptions, and JSON templates from the form field path, consolidating inherited values into a single typed object#3029

Open
dimaMachina wants to merge 15 commits intorm-onProviderOptionsChange-onModelChangefrom
rm-onProviderOptionsChange-onModelChange2

Conversation

@dimaMachina
Copy link
Copy Markdown
Collaborator

@dimaMachina dimaMachina commented Apr 6, 2026

Summary

Refactors ModelConfiguration into a self-contained, slot-aware component that auto-derives its label, description, placeholder, and JSON template from the form field path — eliminating prop sprawl across all consumers.

Changes

model-configuration.tsx (core)

  • Introduces ModelConfigurationSlot ('base' | 'structuredOutput' | 'summarizer') with lookup tables for labels, descriptions, and placeholders.
  • getModelConfigurationSlot() infers the slot from the name path (e.g. "models.structuredOutput""structuredOutput").
  • getJsonPlaceholder() selects the correct JSON template based on model name (Azure detection) and slot type — consolidating logic previously duplicated in consumers.
  • All five template imports are now centralized here.
  • Props simplified to control, name, optional label/description overrides, inherited, and disabled.
  • Uses useController for model, providerOptions, fallbackModels, and allowedProviders sub-fields instead of individual value/onChange prop drilling.
  • Extracts AllowedProvidersSection and FallbackModelsSection as internal sub-components.
  • Conditionally renders AzureConfigurationSection when the model starts with azure/.
  • Gates fallback models and allowed providers behind capabilities?.modelFallback?.enabled and gateway-routability checks.
  • Fully generic over TFieldValues/TTransformedValues for reuse across agent and project forms.

azure-configuration-section.tsx

  • Extracted as a standalone component encapsulating Azure OpenAI-specific configuration (Resource Name vs. Base URL).

metadata-editor.tsx

  • Removes direct template imports. Computes a single inherited object per model slot and passes it to ModelConfiguration with a path-based name.

model-section.tsx

  • Adds getInheritance() helper implementing a 5-level cascading fallback chain (specific slot agent → specific slot project → own base → agent base → project base).
  • Adds basePath prop ('models' | \subAgents.${string}.models``) so the same component serves both agent-level and sub-agent-level configuration.

project-models-section.tsx

  • Splits into BaseModelSection, StructuredOutputModelSection, SummarizerModelSection sub-components.
  • Structured output and summarizer sections watch the base model to compute inherited values.
  • Auto-opens the collapsible on validation errors.

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Claude Opus𝕏

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 6, 2026

🦋 Changeset detected

Latest commit: 067fd9e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
Name Type
@inkeep/agents-manage-ui Patch
@inkeep/agents-api Patch
@inkeep/agents-cli Patch
@inkeep/agents-core Patch
@inkeep/agents-email Patch
@inkeep/agents-mcp Patch
@inkeep/agents-sdk Patch
@inkeep/agents-work-apps Patch
@inkeep/ai-sdk-provider Patch
@inkeep/create-agents Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agents-api Ready Ready Preview, Comment Apr 7, 2026 3:05pm
agents-docs Ready Ready Preview, Comment Apr 7, 2026 3:05pm
agents-manage-ui Ready Ready Preview, Comment Apr 7, 2026 3:05pm

Request Review

@dimaMachina dimaMachina changed the title Rm on provider options change on model change2 refactor ModelConfiguration2 Apr 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 6, 2026

Preview URLs

Use these stable preview aliases for testing this PR:

These point to the same Vercel preview deployment as the bot comment, but they stay stable and easier to find.

Raw Vercel deployment URLs

@dimaMachina dimaMachina marked this pull request as ready for review April 6, 2026 16:49
@dimaMachina
Copy link
Copy Markdown
Collaborator Author

@pullfrog update PR description based on changes in this PR

@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog Bot commented Apr 6, 2026

Updated the PR description for #3029 with a detailed summary of the ModelConfiguration refactoring — covering all 5 changed files, the slot-based auto-configuration pattern, prop simplification, inheritance computation, and extracted sub-components.

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Claude Opus𝕏

@dimaMachina
Copy link
Copy Markdown
Collaborator Author

@claude add a changeset based on changes in this PR

@inkeep-internal-ci
Copy link
Copy Markdown
Contributor

inkeep-internal-ci Bot commented Apr 6, 2026

Claude finished @dimaMachina's task in 48s —— View job


Adding changeset

  • Analyze PR changes
  • Create changeset

A changeset already exists at .changeset/refactor-model-configuration.md (added in commit ea2829b):

---
"@inkeep/agents-manage-ui": patch
---

Simplify ModelConfiguration component by removing onProviderOptionsChange and onModelChange callbacks in favor of direct form state management

No additional action needed.


Copy link
Copy Markdown
Contributor

@pullfrog pullfrog Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean refactor that consolidates the ModelConfiguration prop surface. The inherited bag, slot-derived defaults, and useId() migration are all sound. One potential runtime issue with duplicate React keys in the fallback models list.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

Comment thread agents-manage-ui/src/components/shared/model-configuration.tsx Outdated
Comment thread agents-manage-ui/src/components/shared/model-configuration.tsx
@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog Bot commented Apr 6, 2026

TL;DR — Refactors ModelConfiguration from a heavily-parameterized component into a convention-driven one that derives its label, description, placeholder, clearability, and JSON template from the form field name path. This replaces ~10 individual props with a single inherited object and a slot-based lookup system, dramatically simplifying every call site.

Key changes

  • Introduce slot-based convention system in ModelConfiguration — the component infers its role (base / structuredOutput / summarizer) from the form field name, looking up labels, descriptions, and placeholders from static maps instead of requiring callers to pass them.
  • Consolidate four inherited-value props into a single inherited objectinheritedValue, inheritedProviderOptions, inheritedFallbackModels, and inheritedAllowedProviders are replaced by one typed ModelConfigurationInheritedValues prop with nullable fallbackModels and allowedProviders fields.
  • Replace editorNamePrefix with React useId() — eliminates a threading prop across ModelConfiguration, AzureConfigurationSection, AllowedProvidersSection, and FallbackModelsSection in favor of stable, SSR-safe auto-generated IDs.
  • Centralize JSON template resolution — the per-caller getJsonPlaceholder callback is removed; a single internal getJsonPlaceholder({ model, slot }) function handles all template variants.
  • Rename options to providerOptions in model-section.tsx — aligns the getInheritance() return shape with the new ModelConfigurationInheritedValues interface across all 6 return branches.
  • Extract shared inheritance variables in consumersmetadata-editor.tsx pre-computes baseInherited, structuredOutputInherited, and summarizerInherited objects, eliminating ~20 lines of chained nullish coalescing from the JSX.
  • Fix duplicate React keys and hardcoded IDsFallbackModelsSection key changed from model to `${model}-${index}`; AllowedProvidersSection radio button IDs changed from hardcoded "providers-all" / "providers-specific" to useId().
  • Flatten render tree for provider options and fallback sections — Azure config, provider options editor, AllowedProvidersSection, and FallbackModelsSection are wrapped in a single Fragment under the effectiveModel guard instead of being separate sibling conditionals, reducing duplicated checks.
  • Collapse GATEWAY_ROUTABLE_PROVIDERS_SET to single-line declaration — removes the multi-line new Set([…]) and explicit ReadonlySet<string> annotation in favor of a concise inline form.
  • Add changesetpatch bump for @inkeep/agents-manage-ui documenting the refactor.

Summary | 7 files | 14 commits | base: rm-onProviderOptionsChange-onModelChangerm-onProviderOptionsChange-onModelChange2


Slot-based self-configuration for ModelConfiguration

Before: Every <ModelConfiguration> call site had to specify label, description, placeholder, canClear, editorNamePrefix, and a getJsonPlaceholder callback — repeating the same domain knowledge across three consumer files.
After: The component derives a slot from its form field name path (e.g. models.base"base") and uses static lookup maps (MODEL_CONFIGURATION_LABELS, MODEL_CONFIGURATION_DESCRIPTIONS, MODEL_CONFIGURATION_PLACEHOLDERS) to self-configure. canClear is derived as slot !== 'base', and label / description accept optional overrides via ?? fallback.

getModelConfigurationSlot(name) splits the field path on . and maps the last segment to a ModelConfigurationSlot union type. A centralized getJsonPlaceholder({ model, slot }) function selects the correct Azure or standard template based on the model provider and slot, replacing callback props that were duplicated across six call sites.

How does JSON template selection work now?

The internal getJsonPlaceholder({ model, slot }) function checks if the model string starts with azure/ and branches on the slot to pick the right template import. This keeps all template imports consolidated in model-configuration.tsx instead of scattered across consumers.

model-configuration.tsx


Inherited values as a single typed object

Before: Four separate props (inheritedValue, inheritedProviderOptions, inheritedFallbackModels, inheritedAllowedProviders) were threaded from each consumer, with complex fallback chains inlined in JSX.
After: A single inherited?: ModelConfigurationInheritedValues prop carrying { model, providerOptions, fallbackModels, allowedProviders }. The component handles ?? undefined coercion internally so callers don't need to.

Consumer files now build the inherited object once. In metadata-editor.tsx, shared variables (model, fallbackModels, allowedProviders) are extracted from the watch data and reused across structuredOutputInherited and summarizerInherited, eliminating ~20 lines of duplicated nullish coalescing chains. project-models-section.tsx collapses its <ModelConfiguration> invocations from ~12 props each to a handful.

metadata-editor.tsx · model-section.tsx · project-models-section.tsx


Replace editorNamePrefix with useId()

Before: A string editorNamePrefix prop was passed down to generate HTML element IDs like ${editorNamePrefix}-azure-resource-name — and AllowedProvidersSection used hardcoded IDs ("providers-all", "providers-specific") that would collide when multiple instances render.
After: ModelConfiguration, AzureConfigurationSection, AllowedProvidersSection, and FallbackModelsSection each use React's useId() hook to generate stable, globally unique IDs with no prop threading or collision risk.

FallbackModelsSection also fixes its list item key from model to `${model}-${index}` — preventing React reconciliation issues when duplicate fallback model names appear — and consolidates two adjacent conditional blocks into a single ternary.

azure-configuration-section.tsx · model-configuration.tsx · models.ts

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Claude Opus𝕏

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(1) Total Issues | Risk: Low


🟠⚠️ Major (1) 🟠⚠️

Inline Comments:

  • 🟠 Major: model-configuration.tsx:265 React key collision risk with duplicate fallback models

💭 Consider (1) 💭

Inline Comments:

  • 💭 Consider: model-section.tsx:118 Inconsistent use of explicit description prop vs slot-based derivation

Overview

This is a well-executed refactor that consolidates the ModelConfiguration component's prop interface from 8+ individual props down to a cleaner API with a single inherited object prop and slot-based derivation for defaults. Key improvements:

Props consolidation: inheritedValue, inheritedProviderOptions, inheritedFallbackModels, inheritedAllowedProviders, editorNamePrefix, getJsonPlaceholder, canClear, isRequiredinherited object + automatic derivation

ID generation: Migrated from manual prefix-based IDs to React's useId() hook — consistent with codebase patterns

Centralized logic: Labels, descriptions, placeholders, and JSON templates now derived from slot type (base/structuredOutput/summarizer)

Net reduction: -87 lines while maintaining functionality

The one actionable issue is the React key change in FallbackModelsSection that could cause problems with duplicate models.


💡 APPROVE WITH SUGGESTIONS

Summary: Clean refactor with good API simplification. The React key issue (using model alone instead of model-${index}) is the only actionable item — low probability of occurring in practice but worth fixing for correctness.


Discarded (3)
Location Issue Reason Discarded
azure-configuration-section.tsx:16-19 JSON.parse without try-catch Pre-existing code, not introduced by this PR
model-section.tsx:18-21 Props use any type for projectModels/agentModels Pre-existing code, not introduced by this PR
metadata-editor.tsx:69-107 Inline object creation causes new references React Compiler handles memoization per CLAUDE.md guidelines
Reviewers (4)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-frontend 5 0 0 0 1 0 4
pr-review-standards 1 0 0 0 0 0 1
pr-review-types 4 0 0 0 0 0 4
pr-review-consistency 5 0 0 0 1 0 4
Total 15 0 0 0 2 0 13

Note: High discard rate due to (1) duplicate findings across reviewers for the same React key issue, (2) pre-existing issues out of PR scope, and (3) issues handled by React Compiler.

Comment thread agents-manage-ui/src/components/shared/model-configuration.tsx Outdated
@@ -119,21 +116,13 @@ export function ModelSection({ projectModels, agentModels, control, basePath }:
</div>
}
description="Primary model for general sub agent responses"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 Consider: Inconsistent use of explicit description prop

Issue: This call site explicitly passes description="Primary model for general sub agent responses", while other call sites (metadata-editor.tsx, project-models-section.tsx) rely on the new slot-based derivation pattern.

Why: The refactored ModelConfiguration component automatically derives description from the slot type (base → "Primary model for general agent responses"). Having some call sites override while others use derivation creates a split-world pattern.

Fix: If "sub agent" vs "agent" is an intentional distinction for this context, this override is appropriate. Otherwise, consider removing the explicit prop to use slot-based derivation consistently:

<ModelConfiguration
  control={control}
  name={`${basePath}.base`}
  inherited={{...}}
  label={...}
/>

Refs:

@github-actions github-actions Bot deleted a comment from claude Bot Apr 6, 2026
@itoqa
Copy link
Copy Markdown

itoqa Bot commented Apr 6, 2026

Ito Test Report ✅

9 test cases ran. 3 additional findings, 6 passed.

Overall, 9 test cases were executed with 6 passing and 3 failing (all Medium severity), indicating generally solid behavior but with important persistence issues still open. The key findings were that allowed-providers mode/list changes and fallback-model clears can fail to persist correctly after save/reload (including rapid toggle scenarios), while project creation and agents-route loading, model inheritance reset behavior, Azure field isolation/label targeting, mobile deep-link interruption handling, hostile providerOptions validation, and invalid URL tampering all behaved as expected without confirmed defects.

✅ Passed (6)
Category Summary Screenshot
Adversarial Hostile providerOptions payload remained inert and malformed JSON submission was rejected before create completed. ADV-3
Adversarial Invalid project/agent deep links rendered 404/error states without leaking editable model data, including after back/forward navigation. ADV-4
Edge Azure one-of fields remained independent across two slots, and each Resource Name/Base URL label focused the correct in-slot input. EDGE-3
Edge Mobile deep-link settings flow remained stable after refresh/back-forward; unsaved provider-options edits were not persisted and controls stayed interactive. EDGE-6
Happy-path Re-validation confirmed project creation persisted and the new project's /agents route loaded successfully. ROUTE-2
Happy-path Re-execution verified structured/summarizer inheritance and clear-to-inherited behavior works as expected. ROUTE-4
ℹ️ Additional Findings (3)

These findings are unrelated to the current changes but were observed during testing.

Category Summary Screenshot
Adversarial 🟠 Rapid provider-mode toggles can persist stale provider lists after save/reload. ADV-2
Edge 🟠 Allowed providers state can diverge after save/reload when mode and list mutations are applied. EDGE-4
Edge 🟠 Clearing fallback models does not reliably persist an empty list after save/reload. EDGE-5
🟠 Provider mode toggles can persist stale provider constraints
  • What failed: Final persisted mode/list can be inconsistent with the last UI state, leaving stale explicit providers after reload.
  • Impact: Rapid operator changes can leave hidden routing drift that is hard to detect in review. This increases risk of wrong-provider execution in live agent calls.
  • Steps to reproduce:
    1. Open Allowed providers in project settings and switch between All providers and Specific providers multiple times.
    2. Mutate the provider list while controls are visible, then finish in All providers mode.
    3. Save and reload, then verify the persisted mode/list matches the final UI state.
  • Stub / mock context: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code analysis: I checked the mode-toggle and submit serialization code paths. All-mode transitions clear the list in form state, but payload construction only sends allowedProviders when populated, so clearing can be lost during persistence.
  • Why this is likely a bug: The code cannot reliably represent an explicit clear/reset of provider constraints, so mode-toggle persistence can desync.

Relevant code:

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 132-136)

if (val === 'all') {
  setSpecificMode(false);
  onAllowedProvidersChange([]);
} else {

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 406-408)

function onAllowedProvidersChange(providers: string[]) {
  allowedProvidersField.onChange(providers.length ? providers : undefined);
}

agents-manage-ui/src/components/projects/form/project-form.tsx (lines 71-73)

...(data.models.base.allowedProviders?.length && {
  allowedProviders: data.models.base.allowedProviders.filter(Boolean),
}),
🟠 Allowed providers state can diverge after save/reload
  • What failed: The saved provider configuration can come back in a stale state after reload instead of matching the final UI selection before submit.
  • Impact: Teams cannot trust provider routing configuration to persist exactly as edited. This can cause production traffic to run against unintended providers.
  • Steps to reproduce:
    1. Open a project settings page and expand Configure default models.
    2. Switch to Specific providers, then add/reorder/remove providers.
    3. Save the project, reload the page, and compare the persisted provider list and mode to the final pre-save state.
  • Stub / mock context: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code analysis: I reviewed the shared model configuration state transitions and project submit serialization. The UI clears provider selection by writing an empty array/undefined, but submit serialization only includes allowedProviders when the array has length, which can skip explicit clear operations.
  • Why this is likely a bug: The code path drops explicit "clear/reset" provider updates, so persisted backend state can diverge from the user-selected final state.

Relevant code:

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 132-139)

onValueChange={(val) => {
  if (val === 'all') {
    setSpecificMode(false);
    onAllowedProvidersChange([]);
  } else {
    setSpecificMode(true);
  }
}}

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 406-408)

function onAllowedProvidersChange(providers: string[]) {
  allowedProvidersField.onChange(providers.length ? providers : undefined);
}

agents-manage-ui/src/components/projects/form/project-form.tsx (lines 71-73)

...(data.models.base.allowedProviders?.length && {
  allowedProviders: data.models.base.allowedProviders.filter(Boolean),
}),
🟠 Clearing fallback models does not reliably persist
  • What failed: The empty fallback list is not reliably persisted as an explicit cleared state after submit.
  • Impact: Users cannot reliably remove fallback behavior once configured. Requests may continue using unexpected fallback chains.
  • Steps to reproduce:
    1. Open project settings and expand Configure default models.
    2. Add a fallback model, then remove it so the list is empty.
    3. Save and reload, then verify whether the empty fallback state persisted.
  • Stub / mock context: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code analysis: I inspected fallback mutation handlers and project form serialization. The UI clears fallback values to undefined when the list is empty, and submit payload generation only includes fallbackModels when length is non-zero, so clear-intent can be omitted.
  • Why this is likely a bug: Clearing fallback models should serialize an explicit empty value, but the current code omits that update.

Relevant code:

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 397-399)

function onFallbackModelsChange(models: string[]) {
  fallbackModelsField.onChange(models.length ? models : undefined);
}

agents-manage-ui/src/components/projects/form/project-form.tsx (lines 68-70)

...(data.models.base.fallbackModels?.length && {
  fallbackModels: data.models.base.fallbackModels.filter(Boolean),
}),

agents-manage-ui/src/components/projects/form/project-form.tsx (lines 79-81)

...(data.models.structuredOutput.fallbackModels?.length && {
  fallbackModels: data.models.structuredOutput.fallbackModels.filter(Boolean),
}),

Commit: 5181496

View Full Run


Tell us how we did: Give Ito Feedback

@dimaMachina dimaMachina marked this pull request as draft April 6, 2026 23:50
claude[bot]

This comment was marked as resolved.

@github-actions github-actions Bot deleted a comment from claude Bot Apr 7, 2026
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low


Delta Review: Changes Since Last Review

This commit (903da0513) contains trivial code polish with no functional changes:

Inlined intermediate variables (model-configuration.tsx:466, 473): Removed resolvedLabel and resolvedDescription variables — now uses nullish coalescing directly in JSX props. Pure style preference, no behavior change.

Simplified type annotation (models.ts:92): Removed explicit ReadonlySet<string> type from GATEWAY_ROUTABLE_PROVIDERS_SET. TypeScript infers Set<string> which is functionally equivalent for the usage pattern (.has() calls only).


Prior Issues Status

Issue Status
React key collision (model-configuration.tsx:265) ✅ Fixed (earlier commit)
Inconsistent shouldUnregister usage ⏳ Minor — doesn't affect runtime in most cases
Explicit description prop (model-section.tsx:119) ✅ Intentional — "sub agent" vs "agent" context distinction

✅ APPROVE

Summary: Trivial cosmetic polish — 5 lines added, 9 removed, zero behavior change. The overall refactor successfully consolidates ModelConfiguration into a cleaner, slot-aware component with good API ergonomics. All prior issues have been addressed or acknowledged as minor/intentional. Ship it! 🚀


Reviewers (0)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
Total 0 0 0 0 0 0 0

Note: Sub-reviewers not dispatched — delta contains only cosmetic code polish (inlined variables, removed type annotation) with no functional changes.

@dimaMachina
Copy link
Copy Markdown
Collaborator Author

@claude add a changeset based on changes exactly from this PR

@inkeep-internal-ci
Copy link
Copy Markdown
Contributor

inkeep-internal-ci Bot commented Apr 7, 2026

Claude finished @dimaMachina's task in 1m 22s —— View job


Adding changeset

  • Analyze PR changes
  • Create changeset
  • Push changes

Updated the existing changeset at .changeset/refactor-model-configuration.md to reflect the full scope of this PR:

---
"@inkeep/agents-manage-ui": patch
---

Refactor ModelConfiguration into a slot-aware component that auto-derives labels, descriptions, and JSON templates from the form field path, consolidating inherited values into a single typed object and fixing React key collisions in fallback models

| Branch

Co-authored-by: Dimitri POSTOLOV <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>
@vercel vercel Bot temporarily deployed to Preview – agents-docs April 7, 2026 15:00 Inactive
Comment thread .changeset/refactor-model-configuration.md Outdated
@dimaMachina dimaMachina changed the title refactor ModelConfiguration2 [2/2] Refactor ModelConfiguration into a slot-aware component that auto-derives labels, descriptions, and JSON templates from the form field path, consolidating inherited values into a single typed object Apr 7, 2026
@github-actions github-actions Bot deleted a comment from claude Bot Apr 7, 2026
@github-actions github-actions Bot deleted a comment from claude Bot Apr 7, 2026
@itoqa
Copy link
Copy Markdown

itoqa Bot commented Apr 7, 2026

Ito Test Report ❌

14 test cases ran. 1 failed, 1 additional finding, 12 passed.

Across a unified run of 14 test cases, 12 passed and 2 failed, with broad confirmation that default-model and sub-agent settings persist correctly across save/reload, inheritance and inherited-to-explicit transitions behave as expected, slot-aware labels/templates render correctly, and rapid toggles/submits, navigation stress, malformed JSON recovery (non-Azure flow), and mobile sidepane editing remained stable.
The two key defects were a Critical authorization gap that lets viewer/read-only users perform project and agent mutations when the management bypass secret is enabled (effective system-level privilege escalation) and a High-severity Azure configuration bug where malformed providerOptions text is passed to unguarded JSON.parse and crashes the settings UI into the error boundary.

❌ Failed (1)
Category Summary Screenshot
Adversarial ⚠️ Azure provider options flow can crash settings via unguarded JSON parsing of editor text. ADV-1
⚠️ XSS payload injection in provider options and Azure fields
  • What failed: The UI hits the tenant error boundary (Something went wrong) instead of safely handling invalid provider-options text as inert input.
  • Impact: Admin users can lose in-progress configuration work because the settings screen crashes while editing Azure model options. This blocks reliable configuration updates until invalid text is removed or the page is recovered.
  • Steps to reproduce:
    1. Open model configuration in project settings or agent settings.
    2. Select an azure/* model so Azure OpenAI fields are shown.
    3. Enter non-JSON text in provider options and trigger a render/update path.
    4. Observe the page hit the tenant error boundary instead of safely handling invalid input.
  • Stub / mock context: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code analysis: The model configuration path forwards raw JSON-editor text directly into provider options, then the Azure section synchronously calls JSON.parse without a guard when providerOptions is a string. Any malformed/non-JSON text throws during render and bubbles into the error boundary.
  • Why this is likely a bug: Production UI render logic throws on valid user-editable intermediate input states, which should be tolerated instead of crashing the page.

Relevant code:

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 450-452)

function handleProviderOptionsStringChange(nextValue = '') {
  onProviderOptionsChange(nextValue);
}

agents-manage-ui/src/components/shared/model-configuration.tsx (lines 485-491)

{effectiveModel.startsWith('azure/') && (
  <AzureConfigurationSection
    providerOptions={effectiveProviderOptions}
    onProviderOptionsChange={handleProviderOptionsStringChange}
    disabled={disabled || isUsingInheritedOptions}
  />
)}

agents-manage-ui/src/components/shared/azure-configuration-section.tsx (lines 16-19)

const providerOptionsObj =
  typeof providerOptions === 'string'
    ? JSON.parse(providerOptions || '{}')
    : providerOptions || {};
✅ Passed (12)
Category Summary Screenshot
Adversarial Deep-link refresh and repeated back/forward navigation preserved a stable, usable sub-agent editor with no visible corruption. ADV-3
Adversarial Rapid repeated submits on project and agent saves ended in one coherent persisted configuration after reload checks. ADV-4
Edge Malformed provider options JSON did not crash the UI; after correction to valid JSON, the form remained usable and save-ready. EDGE-1
Edge Rapid provider toggles, mutation, and reload preserved intended allowed-provider and fallback state without UI corruption. EDGE-3
Edge Rapid fallback add/cancel/remove/edit flows settled to a valid persisted state with no phantom rows after reload. EDGE-4
Edge Sidepane remained usable at 390x844; model and JSON edits saved and persisted after reload. EDGE-5
Logic Slot-specific labels, descriptions, and placeholders render correctly for base, structured output, and summarizer model sections. LOGIC-1
Logic Provider option templates correctly vary by slot and Azure vs non-Azure model selection. LOGIC-2
Logic Inherited provider options were cleared correctly when transitioning from inherited to explicit selection of the same effective model, and reload preserved the clean explicit state. LOGIC-3
Happy-path Explicit model/provider options persisted after save+reload, and inherited->explicit same-model transition cleared stale provider options. ROUTE-1
Happy-path Edited project default models (including Azure/non-Azure slot changes), saved successfully, and reloaded with persisted values intact. ROUTE-2
Happy-path Sub-agent structured-output override persisted after save/reload, clear-to-inherited restoration worked, and Azure-to-non-Azure transition behavior remained consistent. ROUTE-3
ℹ️ Additional Findings (1)

These findings are unrelated to the current changes but were observed during testing.

Category Summary Screenshot
Adversarial 🚨 Read-only permission boundary is bypassed, allowing viewer-level mutation of project and agent settings. ADV-2
🚨 Read-only users can mutate project and agent settings via bypass-authenticated UI requests
  • What failed: Read-only users can persist project model changes and create agents, but they should be blocked from edit actions.
  • Impact: Any environment with the bypass secret configured can unintentionally grant write access through the UI, defeating project-level edit authorization. This can lead to unauthorized configuration changes by users expected to be read-only.
  • Steps to reproduce:
    1. Log in as a viewer-level project user.
    2. Open /default/projects/app-playground/settings and edit model configuration.
    3. Click Update project and confirm the change persists after reload.
    4. Open /default/projects/app-playground/agents and create a new agent successfully.
  • Stub / mock context: A viewer account and project membership were seeded to exercise the read-only path, and no route interception or mocked API responses were used while validating the authorization behavior.
  • Code analysis: The UI request layer appends a bearer bypass secret to management API calls whenever the environment variable is present. The API auth middleware treats that token as system, and project permission middleware explicitly skips authorization checks for system, which creates a direct path for unauthorized edits.
  • Why this is likely a bug: Production permission checks are bypassed by code path, not by flaky runtime behavior, so viewer users can be elevated to system-level authorization when the bypass secret is present.

Relevant code:

agents-manage-ui/src/lib/api/api-config.ts (lines 64-71)

const defaultHeaders: HeadersInit = {
  'Content-Type': 'application/json',
  ...options.headers,
  ...(cookieHeader && { Cookie: cookieHeader }),
  ...(process.env.INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET && {
    Authorization: `Bearer ${process.env.INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET}`,
  }),
};

agents-api/src/middleware/manageAuth.ts (lines 70-79)

// 1. First, check if it's the bypass secret
if (
  env.INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET &&
  token === env.INKEEP_AGENTS_MANAGE_API_BYPASS_SECRET
) {
  logger.info({}, 'Bypass secret authenticated successfully');

  // Set system user context for bypass authentication
  c.set('userId', 'system');

agents-api/src/middleware/projectAccess.ts (lines 57-60)

if (userId === 'system' || userId.startsWith('apikey:')) {
  await next();
  return;
}

Commit: 067fd9e

View Full Run


Tell us how we did: Give Ito Feedback

@dimaMachina dimaMachina changed the title [2/2] Refactor ModelConfiguration into a slot-aware component that auto-derives labels, descriptions, and JSON templates from the form field path, consolidating inherited values into a single typed object [2/3] Refactor ModelConfiguration into a slot-aware component that auto-derives labels, descriptions, and JSON templates from the form field path, consolidating inherited values into a single typed object Apr 8, 2026
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity.
It will be closed in 7 days if no further activity occurs.

If this PR is still relevant:

  • Rebase it on the latest main branch
  • Add a comment explaining its current status
  • Request a review if it's ready

Thank you for your contributions!

@github-actions github-actions Bot added the stale label Apr 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant