Skip to content

Allow artifacts to be passed as tool arguments#2189

Merged
tim-inkeep merged 19 commits intomainfrom
feature/artifacts-as-arguments
Feb 27, 2026
Merged

Allow artifacts to be passed as tool arguments#2189
tim-inkeep merged 19 commits intomainfrom
feature/artifacts-as-arguments

Conversation

@tim-inkeep
Copy link
Copy Markdown
Contributor

Adds support for passing saved artifacts by reference ({ "$artifact": "...", "$tool": "..." }) as tool call arguments. Before execution, ArtifactParser.resolveArgs() recursively walks the tool args and replaces any artifact reference objects with the full artifact data. This prevents the LLM from needing to re-transcribe or inline large artifact payloads when chaining tool calls.

Updated system prompts (both v1 PromptConfig variants) instruct the model to always use the reference object form rather than writing artifact data inline.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Feb 19, 2026

🦋 Changeset detected

Latest commit: e8ee4d9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages
Name Type
@inkeep/agents-api Patch
@inkeep/agents-manage-ui Patch
@inkeep/agents-cli Patch
@inkeep/agents-core Patch
@inkeep/agents-mcp Patch
@inkeep/agents-sdk Patch
@inkeep/agents-work-apps Patch
@inkeep/ai-sdk-provider Patch
@inkeep/create-agents Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown

vercel Bot commented Feb 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agents-api Ready Ready Preview, Comment Feb 27, 2026 7:29pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
agents-manage-ui Ignored Ignored Preview Feb 27, 2026 7:29pm
agents-docs Skipped Skipped Feb 27, 2026 7:29pm

Request Review

@vercel vercel Bot temporarily deployed to Preview – agents-docs February 20, 2026 22:07 Inactive
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(8) Total Issues | Risk: High

🔴❗ Critical (2) ❗🔴

🔴 1) resolveArgs()Missing unit test coverage for core feature

Issue: The resolveArgs() method (ArtifactParser.ts lines 220-254) is the primary new feature enabling artifact-as-argument functionality, but it has no direct unit tests. This method handles: (1) detecting { $artifact, $tool } sentinel objects, (2) calling artifactService.getArtifactFull() to fetch full data, (3) recursive traversal of nested objects/arrays, (4) fallback behavior when resolution fails.

Why: Without tests, several regressions could go undetected: nested artifact refs might not resolve correctly, resolution failures silently return the original sentinel object (tool receives { $artifact: 'x', $tool: 'y' } instead of failing), arrays with mixed refs and values might be corrupted. This is the primary feature of the PR and is called on every tool execution path.

Fix: Add dedicated unit tests covering:

  • Top-level artifact ref resolution
  • Nested artifact refs in objects
  • Artifact refs in arrays
  • Resolution failure fallback behavior
  • Primitive passthrough
describe('resolveArgs', () => {
  it('resolves top-level artifact ref to full data', async () => {
    mockArtifactService.getArtifactFull.mockResolvedValue({ data: { title: 'Resolved' } });
    const result = await parser.resolveArgs({ $artifact: 'art-1', $tool: 'tool-1' });
    expect(result).toEqual({ title: 'Resolved' });
  });

  it('resolves nested artifact ref in object', async () => {
    mockArtifactService.getArtifactFull.mockResolvedValue({ data: { content: 'Full' } });
    const result = await parser.resolveArgs({ doc: { $artifact: 'art-1', $tool: 'tool-1' }, other: 'value' });
    expect(result).toEqual({ doc: { content: 'Full' }, other: 'value' });
  });

  it('returns original ref when resolution fails', async () => {
    mockArtifactService.getArtifactFull.mockResolvedValue(null);
    const ref = { $artifact: 'missing', $tool: 'tool' };
    const result = await parser.resolveArgs(ref);
    expect(result).toEqual(ref);
  });
});

Refs:

Inline Comments:

  • 🔴 Critical: artifact-components.mdx:118 Incorrect import — SDK exports functionTool, not tool
  • 🔴 Critical: artifact-components.mdx:121 Function name should be functionTool
  • 🔴 Critical: visual-builder/artifact-components.mdx:134 Same incorrect import
  • 🔴 Critical: visual-builder/artifact-components.mdx:136 Same function name fix

🟠⚠️ Major (3) 🟠⚠️

🟠 1) ArtifactParser.ts:220-235Silent failure when artifact resolution fails

Issue: When resolveArgs() fails to resolve an artifact reference (line 231-235), it logs a warning but returns the original { $artifact, $tool } sentinel object unchanged. The tool then receives this sentinel object as its input instead of the expected data shape.

Why: This creates a silent failure mode where tools may crash or behave unexpectedly because they receive { $artifact: "...", $tool: "..." } instead of the actual artifact content. The docs and prompts promise full resolution but don't explain what happens when resolution fails. Debugging requires correlating the warning log with the tool failure.

Fix: Consider either:

  1. Throw an error that propagates to the tool execution error handler (Agent.ts lines 729-749) which already handles tool errors gracefully
  2. Return a clearly marked error object like { $artifactResolutionFailed: true, reason: 'Artifact not found', ... } that tools can check
  3. Document the behavior and instruct tool authors to handle the fallback case

Refs:

🟠 2) agents-docs/Documentation doesn't explain sentinel format for developers

files: agents-docs/content/typescript-sdk/structured-outputs/artifact-components.mdx, agents-docs/content/visual-builder/structured-outputs/artifact-components.mdx

Issue: The new "Passing Artifacts to Tools" section documents that tools receive complete artifact data, but it does not explain the agent-side syntax { "$artifact": "...", "$tool": "..." } that triggers this resolution. The sentinel format is only documented in the system prompts seen by LLMs, not in developer-facing docs.

Why: Developers reading this documentation may not understand that artifact passing is automatic/system-resolved. They may wonder how they should configure their tool's inputSchema to accept artifacts, or whether they need to handle artifact resolution themselves.

Fix: Add a clarifying subsection explaining:

  1. The sentinel format { "$artifact": "artifact-id", "$tool": "tool-call-id" } that agents use in tool arguments
  2. That the runtime automatically resolves this to full artifact data before the tool executes
  3. What happens if resolution fails (per the behavior noted above)

Example addition:

"When the agent decides to pass an artifact to your tool, it uses a special reference format internally. The system automatically resolves this reference and provides the complete artifact data to your execute function. You simply define your inputSchema with the expected shape of the artifact data—no special handling is required."

Refs:

Inline Comments:

  • 🟠 Major: Agent.ts:1856 collectProjectArtifactComponents() omits current agent's own artifact components

🟡 Minor (2) 🟡

Inline Comments:

  • 🟡 Minor: .changeset/petite-ends-trade.md:5 Changeset message should use sentence case
  • 🟡 Minor: conversations.ts:908 Artifact IDs injected into XML without escaping

💭 Consider (5) 💭

💭 1) ArtifactParser.ts:18-22StreamPart uses data?: any - weak type safety

Issue: The StreamPart interface uses data?: any which provides no type safety for the artifact data payload. Downstream consumers cannot safely narrow the data field.

Why: Code that expects data.artifactId or data.typeSchema has no compile-time guarantee these fields exist. This is a design choice that trades type safety for flexibility.

Fix: Consider defining a discriminated union for StreamPart with specific data shapes if stronger typing is desired.

Refs: ArtifactParser.ts:18-22

💭 2) types.ts:39Naming inconsistency: allProjectArtifactComponents vs artifactComponents

Issue: The new property allProjectArtifactComponents uses a different naming convention than the existing artifactComponents property. The "all" prefix is unusual in this interface.

Fix: Consider projectArtifactComponents for consistency, or clarify the distinction in the comment.

Refs: types.ts:38-39

💭 3) artifact-retrieval-guidance.xml:5-17Concept economy: "FOUR WAYS" vs "three modes" mismatch

Issue: The guidance lists "FOUR WAYS TO ACCESS ARTIFACT DATA" but PromptConfig describes "three modes of use" (create, reference, pass). The mismatch may confuse LLMs about the conceptual model.

Fix: Align the framing: "Two ways to ACCESS existing artifact data (reference in text for preview, pass to tool for full) plus get_reference_artifact for explicit retrieval. One way to CREATE new artifacts."

Refs: artifact-retrieval-guidance.xml:5-17

💭 4) Agent.ts:1877-1879, ArtifactParser.ts:269-272Empty catch blocks without logging

Issue: Several catch blocks silently swallow errors without logging: collectProjectArtifactComponents() (Agent.ts:1877), parseCreateAttributes() (ArtifactParser.ts:269-272).

Fix: Add debug-level logging to help diagnose issues: catch (error) { logger.debug({ error }, 'Failed to parse, using fallback'); ... }

💭 5) ArtifactParser.ts:217-235Sentinel format is first-of-kind convention

Issue: The { "$artifact": "...", "$tool": "..." } sentinel is a novel pattern in the codebase. Existing stream event schemas use discriminated unions with a type field rather than $-prefixed keys. This establishes a precedent.

Fix: Consider documenting why this format was chosen, or adding a constant for the sentinel keys to prevent drift between prompt and runtime.


🚫 REQUEST CHANGES

Summary: This is a well-architected feature that adds valuable capability for artifact-as-argument passing. However, two blocking issues need resolution before merge:

  1. Critical docs bug: The code examples import tool which doesn't exist in the SDK (should be functionTool). Users will get build errors.

  2. Missing test coverage: The core resolveArgs() method has no unit tests despite being the primary feature and affecting every tool execution path.

Additionally, the silent failure behavior when artifact resolution fails (returning the sentinel object unchanged) may cause confusing downstream errors and should at minimum be documented, or ideally made more explicit.

Discarded (8)
Location Issue Reason Discarded
PromptConfig.ts:418-438 First-contact legibility - timing rules confusing Prompt guidance is comprehensive; timing is explained, just detailed
Agent.ts:2034-2040 Updated tool description may confuse LLM Description is helpful and accurate; negation is common in LLM guidance
PromptConfig.ts:414-534 Extensive prompt instructions may exceed token limits Token estimation exists; this is a design tradeoff, not a bug
conversations.ts:894 reconstructMessageText exported but could be private Function is tested separately, export is intentional
ArtifactParser.ts:131-145 typeSchema property lacks schema definition data: z.any() in stream schemas is intentional flexibility
collectProjectArtifactComponents Method returns any[] Type looseness is consistent with existing patterns
buildTypeSchemaMap Accepts any[] Same pattern as above
reconstructMessageText Accepts any Defensive runtime handling is appropriate here
Reviewers (9)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-tests 3 1 0 0 0 0 0
pr-review-docs 4 1 0 0 4 0 0
pr-review-product 5 1 1 0 0 0 2
pr-review-errors 4 0 1 0 0 0 2
pr-review-architecture 4 0 1 0 0 0 1
pr-review-llm 5 0 0 0 1 0 3
pr-review-consistency 7 0 2 0 1 0 3
pr-review-types 4 0 1 0 0 0 3
pr-review-standards 1 0 0 0 1 0 0
Total 37 3 5 0 7 0 14

Note: Multiple reviewers flagged the same issues (silent failure, missing tests) — these were deduplicated in the summary.

For example, a document processing tool can access the full `content` array from a citation artifact even though `content` is a non-preview field:

```typescript
import { tool } from "@inkeep/agents-sdk";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CRITICAL: Incorrect import - SDK exports functionTool, not tool

Issue: The code example imports tool from @inkeep/agents-sdk, but this export does not exist. The SDK exports functionTool for creating function tools with execute handlers.

Why: Users copying this example will get a build/runtime error: Module '@inkeep/agents-sdk' has no exported member 'tool'. This blocks developers from implementing the documented feature.

Fix:

Suggested change
import { tool } from "@inkeep/agents-sdk";
import { functionTool } from "@inkeep/agents-sdk";

Refs:

import { tool } from "@inkeep/agents-sdk";
import { z } from "zod";

const summarizeTool = tool({
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CRITICAL: Function name should be functionTool, not tool

Issue: Continuation of the import fix — the function call should also use functionTool.

Why: Consistent with the import fix above.

Fix:

Suggested change
const summarizeTool = tool({
const summarizeTool = functionTool({

Refs:

When a tool receives an artifact as an argument, it has access to all fields — not just the preview ones. For example, a document processing tool can read the full `content` array even though `content` is a non-preview field:

```typescript
import { tool } from "@inkeep/agents-sdk";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CRITICAL: Incorrect import - SDK exports functionTool, not tool

Issue: Same issue as the TypeScript SDK docs — the SDK exports functionTool, not tool.

Why: Users copying this example will get a build error.

Fix:

Suggested change
import { tool } from "@inkeep/agents-sdk";
import { functionTool } from "@inkeep/agents-sdk";

Refs:

```typescript
import { tool } from "@inkeep/agents-sdk";

const summarizeTool = tool({
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 CRITICAL: Function name should be functionTool, not tool

Issue: Continuation of the import fix.

Fix:

Suggested change
const summarizeTool = tool({
const summarizeTool = functionTool({

Comment thread agents-api/src/domains/run/agents/Agent.ts Outdated
Comment thread .changeset/petite-ends-trade.md Outdated
"@inkeep/agents-api": patch
---

Allowed Artifacts to be Passed as Tool Arguments
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor: Changeset message should use sentence case

Issue: The changeset message uses title case ("Allowed Artifacts to be Passed as Tool Arguments") rather than sentence case per CLAUDE.md guidelines.

Why: Consistency with changelog conventions.

Fix:

Suggested change
Allowed Artifacts to be Passed as Tool Arguments
Allow artifacts to be passed as tool arguments

Refs:

if (part.type === 'data') {
try {
const data = typeof part.data === 'string' ? JSON.parse(part.data) : part.data;
if (data?.artifactId && data?.toolCallId) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor: Artifact IDs injected into XML without escaping

Issue: The reconstructMessageText function injects artifactId and toolCallId directly into XML-like tags without HTML/XML escaping. If these IDs contain special characters (quotes, angle brackets), they could malform the XML or cause parsing issues.

Why: While artifact IDs are typically safe alphanumeric strings, defensive escaping prevents edge-case bugs if IDs contain unexpected characters.

Fix: Consider escaping the values:

// Helper function
const escapeXml = (s: string) => s.replace(/&/g, '&amp;').replace(/"/g, '&quot;').replace(/</g, '&lt;').replace(/>/g, '&gt;');

// In the function
return `<artifact:ref id="${escapeXml(data.artifactId)}" tool="${escapeXml(data.toolCallId)}" />`;

Refs:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pullfrog Does this handle all MessagePart types? WHat are we missing?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Investigated: handles text and data (artifact refs) parts. Missing file parts (silently dropped — acceptable for text-based history reconstruction but could add [file: name] placeholders). Also noted a type vs kind field name discrepancy between runtime data and TypeScript types — no runtime bug since the param is any, but worth aligning when this function gets properly typed.

Pullfrog  | View workflow run | Using Claude Code | Triggered by Pullfrogpullfrog.com𝕏

@github-actions github-actions Bot deleted a comment from claude Bot Feb 20, 2026
@tim-inkeep
Copy link
Copy Markdown
Contributor Author

Need to check claude's comment

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(1) Total Issues | Risk: Low

🟡 Minor (0) 🟡

No new minor issues in the delta.

💭 Consider (0) 💭

No new considerations in the delta.

🕐 Pending Recommendations (1)

  • 🟡 conversations.ts:908 Artifact IDs injected into XML without escaping — defensive escaping prevents edge-case bugs if IDs contain unexpected characters

💡 APPROVE WITH SUGGESTIONS

Summary: Great work addressing the prior review feedback! The critical and major issues have all been resolved:

Tests addedresolveArgs() now has comprehensive unit test coverage (lines 276-333)
Docs fixed — Both artifact-components.mdx files now correctly import functionTool
Agent fix appliedcollectProjectArtifactComponents() now seeds with current agent's components first
Changeset corrected — Now uses sentence case per guidelines
Silent failure documented — The fallback behavior is now explicitly tested, confirming it's intentional design

The only remaining item is the XML escaping suggestion (Minor) which is a defensive hardening measure for an unlikely edge case. This can be addressed in a follow-up if desired.

This PR is ready to merge! 🚀

Discarded (0)

No findings discarded in this delta review.

Reviewers (3)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-standards 0 0 0 0 0 0 0
pr-review-tests 0 0 0 0 0 0 0
pr-review-errors 0 0 0 0 0 0 0
Total 0 0 0 0 0 0 0

Note: This was a delta re-review. Reviewers focused on changes since the prior review at ae38f8d. Prior issues were verified as addressed rather than re-raised.

@github-actions github-actions Bot deleted a comment from claude Bot Feb 23, 2026
@vercel vercel Bot temporarily deployed to Preview – agents-docs February 23, 2026 04:23 Inactive
@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog Bot commented Feb 27, 2026

Addressed review feedback on collectProjectArtifactComponents():

  • Extracted an addUnique helper to eliminate the duplicated dedup loop
  • Renamed allcollected for clarity
  • No behavioral changes — same try/catch, same fallback, same return type

Pushed as refactor: improve collectProjectArtifactComponents clarity.

Pullfrog  | View workflow run | Using Claude Code | Triggered by Pullfrogpullfrog.com𝕏

Comment thread agents-api/src/domains/run/services/ArtifactParser.ts Outdated
@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog Bot commented Feb 27, 2026

Addressed the review comment on ArtifactParser.ts:

  • Renamed typeSchemaMapartifactSchemasByType
  • Renamed inner fields previewShape/fullShapepreview/full
  • Updated the corresponding test file (ArtifactParser.typeSchema.test.ts)
  • Verified no type errors via tsc --noEmit

Pullfrog  | View workflow run | Using Claude Code | Triggered by Pullfrogpullfrog.com𝕏

@vercel vercel Bot temporarily deployed to Preview – agents-docs February 27, 2026 19:02 Inactive
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

This is a delta re-review scoped to changes since the last automated review (Feb 24), which approved this PR.

Delta Changes Reviewed

Commit Change Status
cd8c2631b Remove any types, use ArtifactComponentApiInsert[] ✅ Correctly applied
cd8c2631b Remove excess comment from types.ts ✅ Done
a5216cfc5 Clarify reconstructMessageText JSDoc ✅ Done
1f3952be0 Merge from main ✅ No conflicts

Verification

  • Type changes: collectProjectArtifactComponents() now correctly uses ArtifactComponentApiInsert[] for return type, local variable, and cast — properly addressing prior review feedback
  • Comment cleanup: Excess comment removed from allProjectArtifactComponents in types.ts
  • JSDoc fix: reconstructMessageText now accurately describes its purpose (handling multi-part content with artifact refs)
  • Changeset: Now uses sentence case per guidelines

🕐 Pending Recommendations (1)

  • 🟡 conversations.ts:909 Artifact IDs injected into XML without escaping — defensive hardening for unlikely edge case (can be addressed in follow-up)

💭 Consider (1)

Inline Comments:

  • 💭 Consider: Agent.ts:1854-1884 Clarity improvements for collectProjectArtifactComponents() (suggestions provided per @amikofalvy's feedback)

✅ APPROVE

Summary: All prior review feedback has been addressed correctly. The delta changes are type annotation improvements and documentation clarifications with no behavioral impact. The one remaining minor suggestion (XML escaping) is a defensive hardening measure for an unlikely edge case that can be addressed in a follow-up.

This PR is ready to merge! 🚀

Reviewers (1)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-standards 0 0 0 0 0 0 0
Total 0 0 0 0 1 1 0

Note: This was a delta re-review focused on changes since the Feb 24 approval. No new issues were found in the delta.

Comment on lines +1854 to +1884
private collectProjectArtifactComponents(): ArtifactComponentApiInsert[] {
const project = this.executionContext.project;
try {
const agentDefinition = project.agents[this.config.agentId];
if (!agentDefinition) {
return this.artifactComponents;
}

const seen = new Set<string>();
const all: ArtifactComponentApiInsert[] = [];
for (const ac of this.artifactComponents) {
if (ac.name && !seen.has(ac.name)) {
seen.add(ac.name);
all.push(ac);
}
}
for (const subAgent of Object.values(agentDefinition.subAgents)) {
if ('artifactComponents' in subAgent && subAgent.artifactComponents) {
for (const ac of subAgent.artifactComponents as ArtifactComponentApiInsert[]) {
if (ac.name && !seen.has(ac.name)) {
seen.add(ac.name);
all.push(ac);
}
}
}
}
return all;
} catch {
return this.artifactComponents;
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 Consider: Clarity improvements for collectProjectArtifactComponents()

Re: @amikofalvy's question — here are some ideas to improve clarity:

1. Extract helper for deduplication logic:

private addUniqueComponents(
  components: ArtifactComponentApiInsert[],
  seen: Set<string>,
  all: ArtifactComponentApiInsert[]
): void {
  for (const ac of components) {
    if (ac.name && !seen.has(ac.name)) {
      seen.add(ac.name);
      all.push(ac);
    }
  }
}

2. Add JSDoc explaining the purpose:

/**
 * Collects artifact components from this agent and all its sub-agents,
 * deduplicating by name (first occurrence wins).
 * Used for building the type schema map in prompts.
 */

3. Use more descriptive variable names:

  • alluniqueComponents or collectedComponents
  • seenseenNames

4. Consider returning early if no subAgents:

if (Object.keys(agentDefinition.subAgents).length === 0) {
  return this.artifactComponents;
}

These are optional polish suggestions — the current implementation is functionally correct.

@vercel vercel Bot temporarily deployed to Preview – agents-docs February 27, 2026 19:02 Inactive
@github-actions github-actions Bot deleted a comment from claude Bot Feb 27, 2026
Comment on lines +5 to +21
FOUR WAYS TO ACCESS ARTIFACT DATA:

1. **SUMMARY DATA**: Read the summary_data directly from available artifacts for basic information
2. **FULL DATA**: Use the get_artifact tool to retrieve complete artifact data (both summary_data and full_data) when you need detailed information
1. artifact:ref in text → PREVIEW FIELDS ONLY
Cites the artifact inline in your response. Only summary/preview fields appear in context.

**REUSE EXISTING ARTIFACTS WHEN POSSIBLE:**
- Look for artifacts with similar topics, names, or descriptions
- Check if existing artifacts can answer the current question
- Use existing artifact data instead of creating duplicates
- Only create new artifacts if existing ones don't contain the needed information
- Prioritize reusing relevant existing artifacts over creating new ones
2. artifact:create in text → PREVIEW FIELDS ONLY
Saves a new artifact from a tool result. Only summary/preview fields are captured in context.

HOW TO USE ARTIFACT DATA:
- Read summary_data from available artifacts for quick answers
- Use get_artifact tool when you need comprehensive details
- Extract specific information to answer user questions accurately
- Reference artifacts when citing the information source
- Combine information from multiple existing artifacts when relevant
3. { "$artifact": "id", "$tool": "toolCallId" } as a tool argument → FULL DATA
Passes the artifact directly to a tool. The system resolves the complete artifact data automatically — no retrieval needed.

4. get_reference_artifact tool → FULL DATA
Explicitly fetches the complete artifact data into your context. Only call this when you specifically need to read the actual value of a non-preview field that is not visible in summary_data.

AVOID DUPLICATE ARTIFACTS:
- Check available_artifacts before creating new ones for the same source
- Reuse existing artifact IDs when referencing information already saved
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tim-inkeep Is this instruction clear enough to illustrate when to use each method of accessing artifact data.

Example: Use X when Y for each use case

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amikofalvy from my testing i tested with the 3 different scenarios. my only concern (which is still a concern with our existing prompt) is dumber models maybe not providing the right output.

@amikofalvy
Copy link
Copy Markdown
Collaborator

@pullfrog
CI / ci (pull_request) is failing address and fix

@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog Bot commented Feb 27, 2026

Fixed the CI typecheck failure in agents-api.

Root cause: In collectProjectArtifactComponents() (Agent.ts:1878), subAgent.artifactComponents is a string[] of artifact component IDs (per the sub-agent schema), but was being cast directly as ArtifactComponentApiInsert[]. TypeScript rejected this because string and the full artifact component object type don't overlap.

Fix: Instead of the invalid cast, the code now resolves each string ID by looking it up in the project-level artifactComponents map, then filters out any unresolved entries. This matches the pattern used elsewhere in the codebase (e.g. getArtifactComponentsForSubAgent in project.ts).

Verification: tsc --noEmit passes cleanly with zero errors.

Pullfrog  | View workflow run | Using Claude Code | Triggered by Pullfrogpullfrog.com𝕏

Copy link
Copy Markdown
Collaborator

@amikofalvy amikofalvy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few questions for you that need answers before we can ship

@itoqa
Copy link
Copy Markdown

itoqa Bot commented Feb 27, 2026

Ito Test Report ✅

27 test cases ran. 27 passed.

All tests for PR #2189 "Allow artifacts to be passed as tool arguments" passed successfully. The verification covered the core artifact pass-by-reference functionality, system prompt generation with type schemas, cross-agent artifact component aggregation, conversation history reconstruction, documentation updates, and comprehensive edge cases. Unit tests confirmed correct behavior for artifact resolution, schema validation, and message reconstruction. Integration tests verified the API creates projects with artifact components and that the code paths for resolving artifact references in tool arguments work as expected.

✅ Passed (27)
Test Case Summary Timestamp Screenshot
ROUTE-1 Verified resolveArgs in ArtifactParser.ts replaces artifact refs with full data. 12 unit tests pass. Project with ResearchDoc artifact component created via API. 10:46 ROUTE-1_10-46.png
ROUTE-2 Verified ArtifactParser.buildArtifactDataPart attaches typeSchema with preview/full shapes in data-artifact SSE events. 11:12 ROUTE-2_11-12.png
ROUTE-3 Verified PromptConfig.generateArtifactXml uses artifact.xml template with type and type_schema tags. 9 unit tests passed. 11:13 ROUTE-3_11-13.png
ROUTE-4 Verified collectProjectArtifactComponents in Agent.ts aggregates artifact components from sub-agents. 35 Agent tests passed. 11:14 ROUTE-4_11-14.png
ROUTE-5 Verified reconstructMessageText in conversations.ts produces artifact:ref tags from data parts. 13 conversation tests passed. 11:18 ROUTE-5_11-18.png
ROUTE-6 Verified system prompt guidance includes pass-to-tool instructions with $artifact/$tool format and FULL FIELDS note. 11:19 ROUTE-6_11-19.png
DOCS-1 Verified TS SDK Artifact Components page has 'Accessing Artifact Data' section with 3 bullet points and 'Passing Artifacts to Tools' section with summarizeTool code example. 15:29 DOCS-1_15-29.png
DOCS-2 Verified Visual Builder Artifact Components page has 'Accessing Artifact Data', 'Passing Artifacts to Tools' sections, and Preview Fields with context window lean text. 15:51 DOCS-2_15-51.png
DOCS-3 Verified Chat API page has updated data-artifact Events section mentioning preview fields, inPreview: true, and links to Artifact Components page. 16:13 DOCS-3_16-13.png
DOCS-4 Verified Data Operations page has Field notes subsection with summaryData/fullData explanations and link to Artifact Components page. 16:39 DOCS-4_16-39.png
DOCS-5 Verified Concepts page has updated Artifacts bullet mentioning preview fields, non-preview fields, pass to tools, and link to artifact components. 17:01 DOCS-5_17-01.png
DOCS-6 Verified Agent Settings page has new paragraph about preview() marking fields as immediately available, with code example comments. 17:23 DOCS-6_17-23.png
EDGE-1 Unit test passed: resolveArgs returns original ref when resolution returns null. 19:10 EDGE-1_19-10.png
EDGE-2 Unit test passed: resolves nested artifact ref inside an object. 19:11 EDGE-2_19-11.png
EDGE-3 Unit test passed: resolves artifact refs inside arrays while leaving plain values unchanged. 19:12 EDGE-3_19-12.png
EDGE-4 Unit test verified: typeSchema omitted when artifact type is not in component map. 12:43 EDGE-4_12-43.png
EDGE-5 Unit test verified: typeSchema omitted when no artifactComponents provided. Project without artifacts created via API. 12:45 EDGE-5_12-45.png
EDGE-6 Code verified: collectProjectArtifactComponents falls back to this.artifactComponents when agentId not found. 35 Agent tests passed. 12:46 EDGE-6_12-46.png
EDGE-7 Unit test passed: handles data parts with JSON string data correctly. 19:15 EDGE-7_19-15.png
EDGE-8 Unit test passed: ignores data parts with unparseable JSON string gracefully. 19:15 EDGE-8_19-15.png
EDGE-9 Code inspection confirms: when streamRequestId is null, artifactParser is null and args pass through unmodified. 19:16 EDGE-9_19-16.png
EDGE-10 Unit test passed: artifact XML includes type unknown when artifact has no type field. 19:17 EDGE-10_19-17.png
ADV-1 Unit test passed: non-string values in $artifact/$tool fields pass through unchanged. 19:20 ADV-1_19-20.png
ADV-2 Unit test passed: empty object does not trigger artifact resolution. 19:21 ADV-2_19-21.png
ADV-3 Unit test passed: artifact IDs embedded as-is in XML tags (acceptable for LLM context). 19:23 ADV-3_19-23.png
ADV-4 All 17 schema-validation unit tests passed: buildSchemaShape handles missing type values correctly. 20:49 ADV-4_20-49.png
ADV-5 All 12 ArtifactParser tests passed: resolveArgs correctly skips objects with $artifact but missing $tool key. 21:58 ADV-5_21-58.png
📋 View Recording

Screen Recording

@tim-inkeep tim-inkeep merged commit d62c5b0 into main Feb 27, 2026
11 checks passed
@tim-inkeep tim-inkeep deleted the feature/artifacts-as-arguments branch February 27, 2026 20:39
@github-actions
Copy link
Copy Markdown
Contributor

🔎💬 Inkeep AI search and chat service is syncing content for source 'Inkeep Agent Framework Docs'

dimaMachina pushed a commit that referenced this pull request Mar 2, 2026
* allowed artifacts as tool arguments

* upadted prompt

* Added Artifact Components as Tool Arguments

* fixed linter

* added test

* updated

* updated

* style: auto-format with biome

* docs: clarify reconstructMessageText JSDoc

* address review: remove any types and excess comment

* refactor: rename typeSchemaMap to artifactSchemasByType

* refactor: improve collectProjectArtifactComponents clarity

* style: auto-format with biome

* fix: correct type cast in collectProjectArtifactComponents

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants