> ## Documentation Index
> Fetch the complete documentation index at: https://amd-gaia.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Part 3: Deployment & Optimization

> Deploy as API/CLI, optimize performance, and master advanced patterns

<Info>
  **Source Code:** [`hub/agents/python/chat/gaia_agent_chat/agent.py`](https://github.com/amd/gaia/blob/main/hub/agents/python/chat/gaia_agent_chat/agent.py) | [`src/gaia/rag/sdk.py`](https://github.com/amd/gaia/blob/main/src/gaia/rag/sdk.py)
</Info>

<Badge text="development" color="orange" />

<Note>
  **This is Part 3 of 3.** Complete Part 1 and Part 2 first: [Part 1](./part-1-getting-started) | [Part 2](./part-2-advanced-features)
</Note>

* **Time to complete:** 15-20 minutes
* **What you'll learn:** Agent intelligence, advanced patterns, production deployment, and performance optimization

***

## Understanding the Agent's Intelligence

### How the Agent "Thinks"

When you ask a question, the agent goes through a reasoning loop. Here's a real example:

**User:** *"Find the oil manual on my drive and tell me the vision statement"*

**Agent's Internal Reasoning** (this is what the LLM decides):

```json theme={null}
// Step 1: Analyze what's needed
{
  "thought": "User wants oil manual. Need to find it first.",
  "goal": "Locate oil manual on drive",
  "tool": "search_file",
  "tool_args": {"file_pattern": "oil manual"}
}

// Result: Found "Oil-Gas-Regulatory-Manual.pdf"

// Step 2: Index it
{
  "thought": "Found the file. Need to index before querying.",
  "goal": "Index the oil manual",
  "tool": "index_document",
  "tool_args": {"file_path": "C:/Users/user/Documents/Oil-Gas-Regulatory-Manual.pdf"}
}

// Result: Successfully indexed, 150 chunks created

// Step 3: Search for vision
{
  "thought": "Document indexed. Now search for vision statement.",
  "goal": "Find vision statement in oil manual",
  "tool": "query_specific_file",
  "tool_args": {
    "file_path": "C:/Users/user/Documents/Oil-Gas-Regulatory-Manual.pdf",
    "query": "vision statement"
  }
}

// Result: Retrieved 5 chunks, top score: 0.92
// Chunk: "The vision is to be recognized as..."

// Step 4: Formulate answer
{
  "thought": "I have the vision statement from the document",
  "goal": "Provide answer with citation",
  "answer": "According to the Oil & Gas Regulatory Manual, the vision is to be recognized as the leading regulatory authority in energy sector governance... (Page 3)"
}
```

**Note:** This orchestration is handled by the LLM at runtime based on:

1. System prompt (defines agent behavior)
2. Tool schemas (available functions and their parameters)
3. User input (the query to process)

***

### Search Key Generation (Smart Retrieval)

The agent generates multiple variations of your query to improve retrieval.

**User asks:** *"What is the vision of the oil & gas regulator?"*

**Agent generates multiple search keys:**

1. Original: *"What is the vision of the oil & gas regulator?"*
2. Keywords: *"vision oil gas regulator"*
3. Reformulated: *"oil gas regulator vision definition"*
4. Alternate: *"oil gas regulator vision explanation"*

**Then searches with ALL of them and combines results!**

```python theme={null}
# This happens in _generate_search_keys method
def _generate_search_keys(self, query: str) -> List[str]:
    keys = [query]  # Original

    # Extract keywords (remove stop words)
    words = query.lower().split()
    keywords = [w for w in words if w not in STOP_WORDS and len(w) > 2]
    keys.append(" ".join(keywords))

    # Add reformulations
    if query.lower().startswith("what is"):
        topic = query[8:].strip("?").strip()
        keys.append(f"{topic} definition")
        keys.append(f"{topic} explanation")

    return keys
```

**Impact on retrieval:**

* Increases recall by matching different phrasings
* Compensates for keyword mismatches
* Trade-off: More compute (multiple searches) for better coverage

***

## Advanced Patterns

### Pattern 1: Multi-Document Synthesis

<Tabs>
  <Tab title="Example">
    ```python theme={null}
    agent.process_query(
        "Compare safety protocols in oil manual vs gas manual. "
        "What are the key differences?"
    )
    ```
  </Tab>

  <Tab title="Execution">
    ```
    Agent reasoning:
    1. Query both documents for "safety protocols"
    2. Extract relevant sections from each
    3. Synthesize comparative analysis
    4. Cite both sources in response
    ```
  </Tab>

  <Tab title="Implementation">
    The `query_documents` tool searches across ALL indexed documents by default. The LLM receives chunks from multiple sources and synthesizes them.

    For more control, call the RAG SDK directly via `agent.rag` (there is no
    public `agent.execute_tool()` on the base `Agent` class):

    ```python theme={null}
    # Query across all indexed documents
    response = agent.rag.query("safety protocols")
    print(response.text, response.source_files)
    ```

    To target a single file, use the `query_specific_file` tool the agent
    already exposes (`file_path` + `query`) rather than `agent.rag.query()`,
    which searches every indexed document.
  </Tab>
</Tabs>

***

### Pattern 2: Contextual Follow-ups

<Tabs>
  <Tab title="Example">
    ```python theme={null}
    # Initial query
    agent.process_query("What are the installation requirements?")
    # Response: "Python 3.10+, 8GB RAM, 50GB disk..."

    # Follow-up (implicit context)
    agent.process_query("What about for production?")
    # Agent understands context: still discussing installation
    # Retrieves production-specific requirements
    ```
  </Tab>

  <Tab title="How It Works">
    The agent's `process_query` maintains conversation state:

    ```python theme={null}
    # Conversation history is maintained
    self.conversation_history = [
        {"role": "user", "content": "What are install requirements?"},
        {"role": "assistant", "content": "Python 3.10+..."},
        {"role": "user", "content": "What about for production?"}
    ]
    # LLM receives full history as context
    ```
  </Tab>
</Tabs>

***

### Pattern 3: Progressive Discovery

<Tabs>
  <Tab title="Example">
    ```python theme={null}
    # Query 1
    agent.process_query("What documents do you have about AI?")
    # Agent: Searches filesystem, finds AI PDFs, indexes them

    # Query 2
    agent.process_query("Tell me about neural networks")
    # Agent: Searches newly-indexed AI documents

    # Query 3
    agent.process_query("Are there any documents about transformers?")
    # Agent: Searches again, finds more, indexes on-demand
    ```
  </Tab>

  <Tab title="Pattern">
    **Lazy indexing strategy:**

    * Don't index everything upfront
    * Let user queries drive discovery
    * Index documents as they're needed
    * Builds index progressively

    **Benefits:**

    * Faster startup
    * Lower initial memory usage
    * User-driven relevance
  </Tab>
</Tabs>

***

## Deployment Options

### As a Web API

<Tabs>
  <Tab title="Server">
    The OpenAI-compatible API server exposes a **static** `AGENT_MODELS`
    dict; there is no runtime `registry.register()` API. To add your agent,
    append an entry to `src/gaia/api/agent_registry.py` and restart the
    server:

    ```python title="src/gaia/api/agent_registry.py (excerpt)" theme={null}
    AGENT_MODELS = {
        "gaia-code": {
            "class_name": "gaia_agent_routing.agent.RoutingAgent",
            "init_params": {"api_mode": True, "silent_mode": True, "max_steps": 100},
            "description": "Default routing agent",
        },
        # 👇 Your new agent
        "doc-qa": {
            "class_name": "gaia_agent_chat.agent.ChatAgent",
            "init_params": {
                "silent_mode": True,
                # Only kwargs recognised by ChatAgentConfig will be applied;
                # see ChatAgentConfig in hub/agents/python/chat/gaia_agent_chat/agent.py.
                "rag_documents": ["./company_docs"],
            },
            "description": "Document Q&A agent backed by RAG",
        },
    }
    ```

    Then run the server as usual:

    ```bash theme={null}
    gaia api start --port 8080
    ```

    See [API Server](/docs/sdk/infrastructure/api-server) for the full workflow.
  </Tab>

  <Tab title="Client">
    ```python title="client.py" theme={null}
    from openai import OpenAI

    # Connect to your agent API
    client = OpenAI(
        base_url="http://localhost:8080/v1",
        api_key="dummy"  # Not validated in local mode
    )

    # Query the agent
    response = client.chat.completions.create(
        model="doc-qa",
        messages=[
            {"role": "user", "content": "What's in the manual?"}
        ]
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="Deploy">
    ```bash theme={null}
    # Production with gunicorn
    gunicorn api_server:app \
        --workers 4 \
        --bind 0.0.0.0:8080 \
        --timeout 120

    # With Docker (on AI PC)
    docker build -t doc-qa-agent .
    docker run -p 8080:8080 doc-qa-agent
    ```

    <Note>
      **Deployment on AI PCs:** The agent runs entirely locally on Ryzen AI hardware. No cloud dependencies, ensuring data privacy and low latency.
    </Note>
  </Tab>
</Tabs>

***

### As a CLI Tool

<Tabs>
  <Tab title="Implementation">
    ```python title="cli.py" theme={null}
    #!/usr/bin/env python3
    import sys
    from gaia_agent_chat.agent import ChatAgent, ChatAgentConfig

    def main():
        if len(sys.argv) < 2:
            print("Usage: doc-qa <question>")
            print("       doc-qa --interactive")
            sys.exit(1)

        config = ChatAgentConfig(
            watch_directories=["./documents"],
            silent_mode=False
        )
        agent = ChatAgent(config)

        if sys.argv[1] == "--interactive":
            # Interactive mode
            while True:
                question = input("You: ")
                if question.lower() in ['quit', 'exit']:
                    break
                result = agent.process_query(question)
                print(f"Agent: {result.get('answer')}\n")
        else:
            # One-shot query
            question = " ".join(sys.argv[1:])
            result = agent.process_query(question)
            print(result.get('answer'))

    if __name__ == "__main__":
        main()
    ```
  </Tab>

  <Tab title="Package Config">
    ```toml title="pyproject.toml" theme={null}
    [project]
    name = "doc-qa-agent"
    version = "1.0.0"
    # gaia-agent-chat provides ChatAgent and depends on amd-gaia
    dependencies = ["gaia-agent-chat"]

    [project.scripts]
    doc-qa = "my_agent.cli:main"
    ```

    Install with: `uv pip install -e .`
  </Tab>

  <Tab title="Usage">
    ```bash theme={null}
    # One-shot query
    doc-qa "What does the manual say about installation?"

    # Interactive mode
    doc-qa --interactive

    # From anywhere after install
    cd ~/projects
    doc-qa "Search my indexed docs for API examples"
    ```
  </Tab>
</Tabs>

***

## Troubleshooting

<AccordionGroup>
  <Accordion title="No documents indexed" icon="circle-xmark">
    **Symptom:** Agent responds with "No documents indexed"

    **Debugging:**

    ```python theme={null}
    # Verify RAG dependencies
    uv pip install -e ".[rag]"

    # Check document exists
    import os
    print(os.path.exists("./manual.pdf"))  # Should be True

    # Check indexed state
    print(f"Indexed: {agent.indexed_files}")
    print(f"RAG files: {agent.rag.indexed_files}")
    ```

    **Common causes:**

    * Missing RAG dependencies (`uv pip install -e ".[rag]"`)
    * Incorrect file path
    * File not readable
  </Accordion>

  <Accordion title="Poor retrieval quality" icon="magnifying-glass">
    **Symptom:** Agent returns irrelevant information

    **Tuning options:**

    ```python theme={null}
    # Retrieve more chunks
    config = ChatAgentConfig(max_chunks=10)

    # Use semantic chunking (slower, more accurate)
    config = ChatAgentConfig(use_llm_chunking=True)

    # Larger chunks for more context
    config = ChatAgentConfig(chunk_size=800)

    # Debug what's being retrieved
    config = ChatAgentConfig(debug=True)
    ```

    **Check retrieval scores:**

    ```python theme={null}
    response = agent.rag.query("your question")
    print(f"Scores: {response.chunk_scores}")
    # Scores < 0.5 indicate weak matches
    ```
  </Accordion>

  <Accordion title="File watching not working" icon="eye-slash">
    **Symptom:** New files not auto-indexed

    **Solution:**

    ```bash theme={null}
    # Install watchdog dependency
    uv pip install "watchdog>=2.1.0"
    ```

    **Verify:**

    ```python theme={null}
    # Check watchers are active
    print(f"Watching: {agent.watch_directories}")
    print(f"Observers: {len(agent.observers)}")  # Should be > 0

    # Check file handler telemetry
    if agent.file_handlers:
        telemetry = agent.file_handlers[0].get_telemetry()
        print(f"Events: {telemetry}")
    ```
  </Accordion>

  <Accordion title="Slow indexing performance" icon="hourglass">
    **Symptom:** Indexing takes excessive time

    **Optimizations:**

    ```python theme={null}
    # Smaller chunks = faster indexing
    config = ChatAgentConfig(chunk_size=300)

    # Disable LLM chunking (use fast heuristic)
    config = ChatAgentConfig(use_llm_chunking=False)

    # Index incrementally, not all at once
    # Let file watching handle gradual indexing
    ```

    **Benchmark:**

    ```python theme={null}
    import time
    start = time.time()
    agent.rag.index_document("large.pdf")
    print(f"Indexing took: {time.time() - start:.2f}s")
    ```

    **Typical performance on AI PC with Ryzen AI:**

    * Small PDF (10 pages): \~2-3 seconds
    * Medium PDF (50 pages): \~8-12 seconds
    * Large PDF (200 pages): \~30-45 seconds
    * Embedding generation: \~50-100 chunks/second on NPU
  </Accordion>
</AccordionGroup>

***

### Best Practices

<CardGroup cols={2}>
  <Card title="Start Small" icon="seedling">
    Index a few representative documents first, then scale. Full drive indexing is rarely necessary.
  </Card>

  <Card title="Tune for Document Type" icon="sliders">
    Technical docs need larger chunks (600-800 tokens). FAQs work well with smaller chunks (300-400 tokens).
  </Card>

  <Card title="Use Sessions" icon="floppy-disk">
    Session persistence avoids re-indexing on every restart. Critical for large document sets.
  </Card>

  <Card title="Monitor Selectively" icon="eye">
    Watch only actively changing directories. Static archives don't benefit from monitoring.
  </Card>

  <Card title="Security" icon="lock">
    Set `allowed_paths` in production. Prevents path traversal and unauthorized access.
  </Card>

  <Card title="Incremental Development" icon="vial">
    Build step-by-step. Test each component before combining them.
  </Card>
</CardGroup>

***

## Performance Tuning

<Tabs>
  <Tab title="Indexing Speed">
    | Profile  | chunk\_size | max\_chunks | use\_llm\_chunking | Use Case                  | AI PC Performance          |
    | -------- | ----------- | ----------- | ------------------ | ------------------------- | -------------------------- |
    | Fast     | 300         | 3           | False              | Development, quick FAQs   | \~2-3s per PDF (NPU)       |
    | Balanced | 500         | 5           | False              | Production default        | \~5-7s per PDF (NPU)       |
    | Accurate | 800         | 7           | True               | Complex technical queries | \~10-15s per PDF (NPU+LLM) |

    ```python theme={null}
    # Development (fast)
    config = ChatAgentConfig(
        chunk_size=300,
        max_chunks=3,
        use_llm_chunking=False
    )

    # Production (balanced)
    config = ChatAgentConfig(
        chunk_size=500,
        max_chunks=5,
        use_llm_chunking=False
    )

    # Research (accurate)
    config = ChatAgentConfig(
        chunk_size=800,
        max_chunks=7,
        use_llm_chunking=True
    )
    ```
  </Tab>

  <Tab title="Memory Management">
    ```python theme={null}
    # Index/file-size caps live on RAGConfig (used by the RAG SDK), not on
    # ChatAgentConfig. Build a RAGSDK directly to tune them:
    from gaia.rag.sdk import RAGSDK, RAGConfig

    rag = RAGSDK(RAGConfig(
        max_indexed_files=100,   # Cap at 100 files
        max_file_size_mb=50      # Skip files > 50MB
    ))

    # Manual cleanup on a running ChatAgent
    agent.rag.clear_cache()      # Clear cached embeddings/index
    agent.stop_watching()        # Stop file observers
    ```

    **Memory usage estimates:**

    * \~1MB per 100 chunks (embeddings)
    * \~500KB per PDF (extracted text cache)
    * \~10KB per file watcher event

    <Tip>
      **AI PC Performance:** On Ryzen AI systems, NPU handles embedding generation offloading work from CPU/RAM, improving overall system responsiveness.
    </Tip>
  </Tab>

  <Tab title="Query Optimization">
    ```python theme={null}
    # Reduce max_steps for faster responses
    config = ChatAgentConfig(max_steps=5)

    # Disable stats for production
    config = ChatAgentConfig(
        show_stats=False,
        debug=False
    )

    # Use smaller model for speed (efficient on NPU)
    config = ChatAgentConfig(
        model_id="Qwen3-0.6B-GGUF"
    )
    ```

    <Tip>
      **AMD Ryzen AI Optimization:** Smaller models like Qwen3-0.6B are optimized for NPU execution, providing fast inference with minimal power consumption on AI PCs.
    </Tip>
  </Tab>
</Tabs>

***

## What You've Learned

<CardGroup cols={2}>
  <Card title="Agent Architecture" icon="diagram-project">
    The reasoning loop pattern: query → tool selection → execution → synthesis
  </Card>

  <Card title="Tool System" icon="wrench">
    Using `@tool` decorator to register Python functions as LLM-invocable capabilities
  </Card>

  <Card title="RAG Integration" icon="database">
    Vector embeddings, FAISS indexing, and semantic retrieval
  </Card>

  <Card title="Mixin Pattern" icon="puzzle-piece">
    Composing agent functionality via multiple inheritance
  </Card>

  <Card title="File Monitoring" icon="folder-open">
    Watchdog-based reactive indexing with debouncing
  </Card>

  <Card title="Session Management" icon="floppy-disk">
    JSON serialization for state persistence
  </Card>

  <Card title="Security" icon="shield">
    Path validation, symlink detection, and access control
  </Card>

  <Card title="Performance" icon="gauge">
    Tuning chunk size, retrieval count, and chunking strategies
  </Card>
</CardGroup>

***

## Next Steps

<Steps>
  <Step title="Extend with Voice" icon="microphone">
    Add Whisper (ASR) and Kokoro (TTS) for voice interaction.

    See: [Voice Interaction Guide](/docs/guides/talk)
  </Step>

  <Step title="Build Code Agent" icon="code">
    Create an agent with code generation and debugging capabilities.

    See: [Code Generation Agent](/docs/playbooks/code-agent/part-1-introduction)
  </Step>

  <Step title="Build More Agents" icon="users">
    Explore all available agent guides and playbooks.

    See: [User Guides](/docs/guides/index)
  </Step>

  <Step title="Package for Distribution" icon="rocket">
    Package as Electron app for desktop deployment.

    See: [UI Documentation](/docs/deployment/ui)
  </Step>
</Steps>

***

## Source Code Reference

<CardGroup cols={2}>
  <Card title="ChatAgent" icon="code" href="https://github.com/amd/gaia/blob/main/hub/agents/python/chat/gaia_agent_chat/agent.py">
    Main agent implementation with session management and file monitoring
  </Card>

  <Card title="RAGToolsMixin" icon="database" href="https://github.com/amd/gaia/blob/main/src/gaia/agents/tools/rag_tools.py">
    Document indexing and query tools
  </Card>

  <Card title="FileToolsMixin" icon="folder" href="https://github.com/amd/gaia/blob/main/src/gaia/agents/tools/file_monitor_tools.py">
    Directory monitoring implementation
  </Card>

  <Card title="FileSearchToolsMixin" icon="magnifying-glass" href="https://github.com/amd/gaia/blob/main/src/gaia/agents/tools/file_tools.py">
    Multi-phase file discovery across drives
  </Card>
</CardGroup>

***

## Resources

<CardGroup cols={3}>
  <Card title="GitHub" icon="github" href="https://github.com/amd/gaia">
    Source code, issues, and discussions
  </Card>

  <Card title="Discord" icon="discord" href="https://discord.com/channels/1392562559122407535/1402013282495102997">
    Developer community chat
  </Card>

  <Card title="Email" icon="envelope" href="mailto:gaia@amd.com">
    [gaia@amd.com](mailto:gaia@amd.com)
  </Card>
</CardGroup>

***

<small style="color: #666;">
  **License**

  Copyright(C) 2024-2026 Advanced Micro Devices, Inc. All rights reserved.

  SPDX-License-Identifier: MIT
</small>