> For the complete documentation index, see [llms.txt](https://docs.graphlit.dev/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.graphlit.dev/api-guides/use-cases/conversations/conversation-prompt-agent.md).

# Prompt Agent

## User Intent

"I want to use AI with tool calling but don't need real-time streaming"

## Operation

* **SDK Method**: `graphlit.promptAgent()`
* **GraphQL**: N/A (composite operation)
* **Entity Type**: Conversation
* **Common Use Cases**: Batch processing, background tasks, tool calling without streaming, server-side AI

## TypeScript (Canonical)

```typescript
import { Graphlit } from 'graphlit-client';
import { ModelServiceTypes, SpecificationTypes } from 'graphlit-client/dist/generated/graphql-types';

const graphlit = new Graphlit();

// Define tools
const tools: ToolDefinitionInput[] = [
  {
    name: 'searchDocuments',
    description: 'Search through ingested documents',
    schema: JSON.stringify({
      type: 'object',
      properties: {
        query: { type: 'string', description: 'Search query' }
      },
      required: ['query']
    })
  }
];

// Implement tool handlers
const toolHandlers: Record<string, Function> = {
  searchDocuments: async (args: { query: string }) => {
    const results = await graphlit.queryContents({
      search: args.query,
      limit: 5
    });
    
    return {
      count: results.contents.results.length,
      results: results.contents.results.map(c => c.name)
    };
  }
};

// Prompt agent with tools
const result = await graphlit.promptAgent(
  'Find information about API rate limiting',  // prompt
  undefined,  // conversationId (optional - creates new)
  undefined,  // specification (optional - uses default)
  tools,      // tool definitions
  toolHandlers // tool implementations
);

console.log(`AI Response: ${result.message}`);
console.log(`Tools called: ${result.toolCalls?.length || 0}`);
console.log(`Tokens used: ${result.usage?.totalTokens || 0}`);
```

## Parameters

### promptAgent

* **`prompt`** (string): User's question or message
* **`conversationId`** (string): Optional conversation ID
  * If omitted: Creates new conversation
  * If provided: Continues existing conversation
* **`specification`** (EntityReferenceInput): Optional LLM configuration
* **`tools`** (ToolDefinitionInput\[]): Tool definitions for function calling
* **`toolHandlers`** (Record\<string, Function>): Tool implementations
  * Key = tool name
  * Value = async function that executes tool

## Response

```typescript
{
  conversationId: string;           // Conversation ID (new or existing)
  message: string;                  // AI's complete response
  conversationMessage?: {           // Detailed message info
    message: string,
    role: 'ASSISTANT',
    tokens: number,
    model: string,
    completionTime: number
  },
  toolCalls?: ToolCall[];          // Tools that were called
  toolResults?: ToolResult[];      // Results from tool executions
  usage?: {                        // Token usage
    totalTokens: number,
    promptTokens: number,
    completionTokens: number,
    cost?: number
  },
  metrics?: {                      // Performance metrics
    totalTime: number,
    ttft: number,
    llmTime: number,
    toolTime: number,
    toolExecutions: number,
    rounds: number
  }
}
```

## Developer Hints

### promptAgent vs streamAgent

| Feature          | promptAgent                     | streamAgent                |
| ---------------- | ------------------------------- | -------------------------- |
| **Streaming**    | Returns complete response       | Token-by-token streaming   |
| **Tool calling** | Supported                       | Supported                  |
| **Latency**      | Higher (wait for full response) | Lower (first token faster) |
| **Use case**     | Background, batch, server-side  | Chat UI, real-time         |
| **Result**       | Single return value             | Event callbacks            |

```typescript
// Use promptAgent when:
// - Processing in background
// - Batch operations
// - Don't need streaming
// - Simpler code structure

// Use streamAgent when:
// - Chat UI with streaming
// - Need real-time progress
// - User-facing interactions
```

### Tool Execution is Automatic

Unlike `promptConversation` which doesn't call tools, `promptAgent` automatically:

1. Detects when tools should be called (based on prompt)
2. Calls your tool handlers
3. Sends results back to LLM
4. Returns final response

```typescript
// You provide tools + handlers
const result = await graphlit.promptAgent(prompt, undefined, undefined, tools, toolHandlers);

// Graphlit automatically:
// 1. LLM decides to call searchDocuments
// 2. Calls your searchDocuments handler
// 3. Passes result back to LLM
// 4. Returns final response incorporating tool results
```

### Checking Tool Calls

```typescript
const result = await graphlit.promptAgent(prompt, undefined, undefined, tools, toolHandlers);

// Check which tools were called
if (result.toolCalls && result.toolCalls.length > 0) {
  console.log(`Tools called: ${result.toolCalls.length}`);
  
  result.toolCalls.forEach((toolCall, index) => {
    const toolResult = result.toolResults?.[index];
    
    console.log(`Tool: ${toolCall.name}`);
    console.log(`Args: ${JSON.stringify(toolCall.arguments)}`);
    console.log(`Result: ${JSON.stringify(toolResult?.result)}`);
    
    if (toolResult?.error) {
      console.error(`Error: ${toolResult.error}`);
    }
  });
}
```

### ⚡ Performance Metrics

```typescript
const result = await graphlit.promptAgent(prompt, undefined, undefined, tools, toolHandlers);

console.log('Performance:');
console.log(`  Total time: ${result.metrics?.totalTime}ms`);
console.log(`  LLM time: ${result.metrics?.llmTime}ms`);
console.log(`  Tool time: ${result.metrics?.toolTime}ms`);
console.log(`  Tool executions: ${result.metrics?.toolExecutions}`);
console.log(`  Rounds: ${result.metrics?.rounds}`);
```

**Rounds**: Number of LLM calls (1 = no tools, 2+ = tool calling with follow-ups)

## Variations

### 1. Simple Agent without Tools

Use as synchronous alternative to promptConversation:

```typescript
const result = await graphlit.promptAgent(
  'What are the key findings in the document?'
  // No tools - just RAG
);

console.log(result.message);
```

### 2. Multi-Tool Agent

Provide multiple tools for complex tasks:

```typescript
const tools = [
  {
    name: 'searchDocuments',
    description: 'Search ingested documents',
    schema: JSON.stringify({
      type: 'object',
      properties: {
        query: { type: 'string' }
      },
      required: ['query']
    })
  },
  {
    name: 'getCurrentDate',
    description: 'Get current date and time',
    schema: JSON.stringify({
      type: 'object',
      properties: {}
    })
  },
  {
    name: 'calculateSum',
    description: 'Add numbers together',
    schema: JSON.stringify({
      type: 'object',
      properties: {
        numbers: { type: 'array', items: { type: 'number' } }
      },
      required: ['numbers']
    })
  }
];

const toolHandlers = {
  searchDocuments: async (args: {query: string}) => {
    // Implementation
  },
  getCurrentDate: async () => {
    return { date: new Date().toISOString() };
  },
  calculateSum: async (args: {numbers: number[]}) => {
    return { sum: args.numbers.reduce((a, b) => a + b, 0) };
  }
};

const result = await graphlit.promptAgent(
  'Search for information about pricing, then calculate the total cost',
  undefined,
  undefined,
  tools,
  toolHandlers
);
```

### 3. Multi-Turn Conversation with Tools

Continue conversation across multiple prompts:

```typescript
let conversationId: string | undefined;

// First turn
const result1 = await graphlit.promptAgent(
  'Search for recent news about AI',
  conversationId,
  undefined,
  tools,
  toolHandlers
);

conversationId = result1.conversationId;
console.log(result1.message);

// Second turn (with context from first)
const result2 = await graphlit.promptAgent(
  'Summarize the top 3 articles you found',
  conversationId,  // Same conversation
  undefined,
  tools,
  toolHandlers
);

console.log(result2.message);
```

### 4. Agent with Custom Model

Use specific LLM:

```typescript
// Create specification
const specResponse = await graphlit.createSpecification({
  name: 'Claude Sonnet',
  type: SpecificationTypes.Completion,
  serviceType: ModelServiceTypes.Anthropic,
  anthropic: {
    model: AnthropicModels.Claude_3_5Sonnet
  }
});

// Use with promptAgent
const result = await graphlit.promptAgent(
  'Analyze this data and provide insights',
  undefined,
  { id: specResponse.createSpecification.id },  // Custom model
  tools,
  toolHandlers
);
```

### 5. Error Handling for Tool Failures

Handle tool execution errors gracefully:

```typescript
const toolHandlers = {
  riskyOperation: async (args: any) => {
    try {
      // Operation that might fail
      const data = await fetchExternalAPI(args.url);
      return { success: true, data };
    } catch (error) {
      // Return error info for LLM to handle
      return {
        success: false,
        error: error.message
      };
    }
  }
};

const result = await graphlit.promptAgent(
  'Fetch data from external API',
  undefined,
  undefined,
  tools,
  toolHandlers
);

// Check if any tools failed
const failedTools = result.toolResults?.filter(tr => tr.error);
if (failedTools && failedTools.length > 0) {
  console.warn(`${failedTools.length} tools failed`);
  // LLM's response will acknowledge the failures
}

console.log(result.message);
```

### 6. Batch Processing with promptAgent

Process multiple queries in parallel:

```typescript
const prompts = [
  'What is the main topic of document 1?',
  'Summarize document 2',
  'Extract key dates from document 3'
];

const results = await Promise.all(
  prompts.map(prompt =>
    graphlit.promptAgent(prompt, undefined, undefined, tools, toolHandlers)
  )
);

results.forEach((result, index) => {
  console.log(`Query ${index + 1}: ${result.message}`);
  console.log(`Tokens: ${result.usage?.totalTokens}`);
});
```

## Common Issues

**Issue**: Tool not being called even though it seems relevant\
**Solution**: Ensure tool description clearly explains when/why to use it. Add more specific use cases in description.

**Issue**: Tool called but result not used in response\
**Solution**: Check tool handler returns data in expected format. Log tool results to debug.

**Issue**: `toolHandlers` function not executing\
**Solution**: Verify function name matches tool name exactly (case-sensitive). Ensure function is async.

**Issue**: Response takes longer than expected\
**Solution**: Tools are called synchronously. If tools take time (API calls, DB queries), total time increases. Use streamAgent for better UX.

**Issue**: Conversation ID changes unexpectedly\
**Solution**: If conversationId is undefined, promptAgent creates new conversation. Always pass conversationId for multi-turn.

**Issue**: Tool parameters not matching schema\
**Solution**: LLM may provide parameters differently than expected. Add validation in tool handler:

```typescript
const toolHandlers = {
  myTool: async (args: any) => {
    if (!args.requiredParam) {
      return { error: 'Missing required parameter' };
    }
    // ... rest of implementation
  }
};
```

## Production Example

**Batch processing with tool calling**:

```typescript
const result = await graphlit.promptAgent(
  userPrompt,
  conversationId,
  specificationId,
  tools as ToolDefinitionInput[],
  toolHandlers
);

// Track conversation ID (may be created by promptAgent)
if (result.conversationId && result.conversationId !== conversationId) {
  conversationId = result.conversationId;
  console.log(`New conversation created: ${conversationId}`);
}

// Log tool execution
if (result.toolCalls && result.toolCalls.length > 0) {
  console.log(`Executed ${result.toolCalls.length} tool calls:`);
  
  for (const toolCall of result.toolCalls) {
    const toolResult = result.toolResults?.find(tr => tr.name === toolCall.name);
    
    console.log(`  - ${toolCall.name}: ${toolResult?.error ? 'failed' : 'completed'}`);
    
    if (toolResult?.error) {
      console.error(`    Error: ${toolResult.error}`);
    }
  }
}

// Return response with metrics
return {
  message: result.message,
  tokens: result.usage?.totalTokens,
  cost: result.usage?.cost,
  duration: result.metrics?.totalTime
};
```