Building Enterprise AI Chatbots: A Deep Dive into Multi-Agent Architecture

Modern enterprise chatbots have evolved far beyond simple question-answering systems. Today’s AI assistants need to handle complex queries, search across multiple data sources, reason about user intent, and deliver accurate, contextual responses. This article explores the architectural patterns that make this possible, drawing from real-world enterprise implementations.

The Challenge: Beyond Simple Chatbots

Traditional chatbots follow a straightforward pattern: receive a question, search a knowledge base, return an answer. But enterprise requirements demand much more:

• Multi-source data retrieval: Information scattered across websites, product databases, knowledge bases, and documents
• Intent understanding: Distinguishing between product inquiries, job searches, financial questions, and general information requests
• Contextual responses: Maintaining conversation history and understanding follow-up questions
• Scalability: Handling thousands of concurrent users with sub-second response times
• Accuracy: Providing reliable information with proper source attribution

The Multi-Agent Architecture Pattern

The solution lies in a multi-agent architecture where specialized agents collaborate to handle different aspects of a query. Let’s break down the key components:

1. The Plan Agent: Orchestrating the Response

At the heart of the system sits a Plan Agent that analyzes incoming queries and determines which specialized agents should be activated. Think of it as a dispatcher that routes requests to the right experts.

export class PlanAgent {
    private readonly modelInterface: BedrockModelInterface;
    private readonly agents: ISearchAgent[];

    public async createExecutionPlan(request: Request): Promise<string> {
        const systemPrompt = `You are an agent working at the entry of a conversational AI system.
Your task is to identify the user's intent and select the agents needed to fulfill the query.

AVAILABLE AGENTS:
${this.agents.map(agent => `- ${agent.getName()} - ${agent.getDescription()}`).join('\n')}

RESPONSE FORMAT:
<plan>
  <agents>
    <agent>agent_name</agent>
  </agents>
  <user_intent>
    Clear description of what the user wants
  </user_intent>
</plan>`;

        return await this.modelInterface.callModel(systemPrompt, request);
    }
}

The Plan Agent uses an LLM to understand the query and outputs a structured execution plan in XML format. This approach provides:

• Flexibility: Easy to add new agents without changing the core logic
• Transparency: Clear reasoning about why certain agents were selected
• Fallback handling: Default agents when intent is unclear

2. Specialized Search Agents

Each search agent is an expert in retrieving specific types of information:

interface ISearchAgent {
    getName(): string;
    getDescription(): string;
    getData(request: Request, context: RequestContext): Promise<SearchAgentResponse>;
    needsAgents(): string[]; // Dependencies on other agents
}

Common agent types include:

• Product Search Agent: Queries product catalogs and specifications
• Knowledge Base Agent: Searches internal documentation and FAQs
• Web Search Agent: Retrieves information from public websites
• Job Search Agent: Handles career and employment queries
• Finance Agent: Processes financial data and reports

3. Reasoning Agents: Adding Intelligence

Beyond search, reasoning agents add a layer of intelligence:

interface IReasoningAgent {
    getData(request: Request, context: RequestContext): Promise<AgentResponse>;
}

Examples include:

• Persona Agent: Ensures responses match the brand voice
• Blacklist Agent: Filters inappropriate content
• Language Detection Agent: Identifies user language and region
• Forbidden Task Agent: Prevents the system from performing unauthorized actions

4. The Answer Agent: Synthesizing Results

After all agents complete their work, the Answer Agent synthesizes the collected information into a coherent response:

export class AnswerAgent {
    private readonly modelInterface: BedrockModelInterface;

    public async createAnswer(
        promptExtensions: string,
        request: Request,
        context: RequestContext
    ): Promise<string> {
        const systemPrompt = `You are a helpful assistant. Use the following information 
to answer the user's question accurately and concisely.

${promptExtensions}

USER INTENT: ${context.intent}`;

        return await this.modelInterface.callModel(systemPrompt, request);
    }
}

OpenSearch + RAG: The Retrieval Foundation

Retrieval-Augmented Generation (RAG) is the backbone of accurate AI responses. Here’s how OpenSearch powers this:

Vector Search for Semantic Understanding

export class EmbeddingService {
    async getQueryEmbedding(query: string): Promise<number[]> {
        const result = await this.bedrockClient.invokeModel({
            modelId: 'amazon.titan-embed-text-v1',
            body: JSON.stringify({ inputText: query })
        });
        return result.embeddings[0];
    }
}

The embedding service converts text into vector representations, enabling semantic search that understands meaning rather than just keywords.

Hybrid Search Strategy

Combining lexical and semantic search delivers the best results:

export class SemanticSearchStrategy {
    async search(params: SearchParams): Promise<SearchResult> {
        const vector = await this.embeddingService.getQueryEmbedding(params.query);
        
        const body: MsearchBody = {
            knn: {
                embedding: {
                    vector: vector,
                    k: 10
                }
            },
            query: {
                bool: {
                    should: [
                        { match: { title: params.query } },
                        { match: { content: params.query } }
                    ]
                }
            }
        };

        return await this.openSearchClient.search(body);
    }
}

This hybrid approach ensures:

• Semantic matches: Finding conceptually related content
• Exact matches: Catching specific product names or technical terms
• Ranked results: Combining scores for optimal relevance

Parallel Execution for Performance

Enterprise chatbots must respond quickly. The architecture leverages parallel execution:

public async chat(request: Request): Promise<Response> {
    // Start language detection immediately
    const languagePromise = this.languageRegionDetectAgent.detectLanguage(request);
    
    // Create execution plan
    const plan = await this.planAgent.createExecutionPlan(request);
    const agents = this.extractAgentsFromPlan(plan);
    
    // Execute all agents in parallel
    const [reasoningResults, searchResults] = await Promise.all([
        Promise.all(this.reasoningAgents.map(agent => agent.getData(request, context))),
        Promise.all(agents.map(agent => agent.getData(request, context)))
    ]);
    
    // Combine and generate response
    return await this.answerAgent.createAnswer(combinedResults, request, context);
}

Key performance optimizations:

• Parallel agent execution: All selected agents run simultaneously
• Early language detection: Starts before planning completes
• Streaming responses: Results delivered as they become available

Handling Complex Scenarios: The Compare Agent Pattern

Some queries require comparing multiple entities. The Meta-Agent pattern handles this elegantly:

export class CompareAgent implements IAgent {
    async getData(request: Request, context: RequestContext): Promise<AgentResponse> {
        // Extract entities to compare
        const entities = this.extractEntities(context.executionPlan);
        
        // Process each entity in parallel
        const entityPromises = entities.map(entity => 
            this.processEntity(entity, request, context)
        );
        const entityResults = await Promise.all(entityPromises);
        
        // Format comparison results
        return this.formatComparisonResults(entityResults);
    }
    
    private async processEntity(entity: string, request: Request, context: RequestContext) {
        // Determine relevant agents for this entity
        const relevantAgents = await this.determineRelevantAgents(entity);
        
        // Execute searches for this specific entity
        const results = await Promise.all(
            relevantAgents.map(agent => agent.getData(
                { ...request, query: `Information about ${entity}` },
                context
            ))
        );
        
        return { name: entity, results };
    }
}

This pattern enables:

• Dynamic agent selection: Different agents for different entity types
• Parallel entity processing: Compare multiple items simultaneously
• Structured output: Results formatted for easy comparison

AWS Bedrock Integration

The architecture leverages AWS Bedrock for LLM capabilities:

export class BedrockModelInterface {
    private readonly client: BedrockRuntimeClient;
    private readonly modelId: string;

    async callModel(systemPrompt: string, messages: Message[]): Promise<string> {
        const response = await this.client.send(new InvokeModelCommand({
            modelId: this.modelId,
            body: JSON.stringify({
                system: systemPrompt,
                messages: messages,
                max_tokens: 4096,
                temperature: 0.7
            })
        }));
        
        return this.parseResponse(response);
    }
}

Benefits of Bedrock:

• Model flexibility: Switch between Claude, Nova, and other models
• Managed infrastructure: No GPU management required
• Enterprise security: Data stays within AWS boundaries
• Cost optimization: Pay-per-use pricing

Best Practices for Production

1. Implement Robust Error Handling

async getData(request: Request, context: RequestContext): Promise<AgentResponse> {
    try {
        return await this.executeSearch(request, context);
    } catch (error) {
        console.error(`Agent ${this.getName()} failed:`, error);
        return { promptExtension: '', data: [] }; // Graceful degradation
    }
}

2. Add Comprehensive Logging

console.log('used agents:', agents);
console.log('user intent:', userIntent);
console.log('language:', language);
console.log('answer:', answer);

3. Deduplicate Results

function deduplicateHitsByUrl(hits: SearchHit[]): SearchHit[] {
    const uniqueUrls = new Set<string>();
    return hits.filter(hit => {
        if (!hit.url || uniqueUrls.has(hit.url)) return false;
        uniqueUrls.add(hit.url);
        return true;
    });
}

4. Validate Requests

private isValid(request: Request): boolean {
    if (!request?.query?.trim()) return false;
    
    const { history } = request;
    if (!history || history.length === 0) return true;
    
    // Ensure alternating user/assistant messages
    return history.every((msg, i) => 
        i === 0 || msg.role !== history[i - 1].role
    );
}

Conclusion: Building for the Future

Multi-agent architecture represents the state of the art in enterprise AI chatbots. By decomposing complex queries into specialized agents, leveraging OpenSearch for powerful retrieval, and orchestrating everything with modern LLMs, you can build systems that are:

• Scalable: Handle growing data and user loads
• Maintainable: Add new capabilities without rewriting core logic
• Accurate: Combine multiple sources for comprehensive answers
• Fast: Parallel execution for sub-second responses

The patterns described here have been battle-tested in production environments handling millions of queries. Whether you’re building a customer service bot, an internal knowledge assistant, or a product recommendation engine, these architectural principles provide a solid foundation.

---

Building an AI-powered application for your business? At n3tz, we specialize in architecting and implementing sophisticated AI solutions. Get in touch to discuss how we can help bring your vision to life.

[Top]