> ## Documentation Index
> Fetch the complete documentation index at: https://amd-gaia.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# API Reference

> Error handling, thread safety, security, deployment, and complete API for the C++ agent framework

<Info>
  **Source Code:** [`cpp/`](https://github.com/amd/gaia/tree/main/cpp) in the GAIA repository.
</Info>

<Note>
  **See also:** [Overview](/docs/cpp/overview) for architecture, execution flow, and getting started.
</Note>

***

## Error Handling & Recovery

The framework handles failures at every layer — LLM connection, JSON parsing, and tool execution — so your agent doesn't crash on transient errors.

### LLM Connection Failures

If the LLM server is unreachable or returns an error, the agent **retries once automatically**, then exits gracefully:

```
Call LLM → fails → retry once → fails again → return error result
```

The return value on LLM failure:

```json theme={null}
{
  "result": "Unable to complete task due to LLM error: Connection refused",
  "steps_taken": 1,
  "steps_limit": 20
}
```

Your application should check the `result` field — there is no exception to catch. HTTP timeouts: 30s connection, 120s read.

### Malformed JSON Recovery

Local LLMs often return imperfect JSON. The parser applies **six extraction strategies** in sequence:

1. Direct JSON parse
2. Extract from markdown code blocks (` ```json ... ``` `)
3. Bracket-matching — find first complete `{...}` in mixed text
4. Fix common syntax errors (trailing commas, single quotes, missing brackets)
5. Regex extraction of individual fields (`"thought"`, `"tool"`, `"answer"`)
6. Treat entire response as a plain-text conversational answer

This means the agent recovers from most LLM formatting errors without any intervention.

### Tool Execution Errors

When a tool callback throws an exception or returns `{"status": "error", ...}`, the agent enters **error recovery mode**:

1. The error is captured (exceptions are caught, not propagated)
2. The error context is sent back to the LLM: *"Tool execution failed. Please try an alternative approach."*
3. The LLM reasons about the error and may try a different tool or strategy
4. If the LLM cannot recover within `maxSteps`, the agent returns the last error as the result

Tool errors never crash the agent. The error flow:

```cpp theme={null}
try {
    result = tool->callback(args);
} catch (const std::exception& e) {
    result = {{"status", "error"}, {"error", "Tool execution failed: " + e.what()}};
}
// → error context sent to LLM → LLM adapts → loop continues
```

### MCP Auto-Reconnect

If an MCP server disconnects mid-session (process crash, timeout), the agent **reconnects automatically**:

```
MCP tool call → fails → reconnect to server → retry tool call → success or return error
```

The subprocess is re-launched and re-initialized. If reconnection fails, the tool call returns an error and the LLM is notified.

### Loop Detection

The agent detects **infinite tool call loops** — when the LLM calls the same tool with the same arguments 4+ times in a row. When detected, the agent stops and returns:

```
"Task stopped due to repeated tool call loop."
```

***

## Thread Safety

### Blocking Semantics

**`processQuery()` is fully blocking.** It runs the complete agent loop (LLM calls, tool executions, history management) on the calling thread and returns only when a final answer is produced or the step limit is reached.

This means:

* Do **not** call `processQuery()` from a UI thread — it will freeze the UI for the duration of the agent run
* Use a background thread or async wrapper for GUI integration

### Concurrent Agent Instances

**Different `Agent` instances are fully independent** and can run in parallel on separate threads. Each agent owns its own conversation history, tool registry, MCP connections, and output handler.

```cpp theme={null}
// SAFE — separate instances on separate threads
Agent agent1(config1);
Agent agent2(config2);

std::thread t1([&] { agent1.processQuery("query 1"); });
std::thread t2([&] { agent2.processQuery("query 2"); });
t1.join();
t2.join();
```

### Single-Agent Rules

**Do NOT call `processQuery()` concurrently on the same agent instance.** The Agent enforces this with an atomic in-flight guard: the second concurrent call throws `std::runtime_error("Agent::processQuery is not re-entrant")`. Callers must serialize access to a single Agent (or create one per thread/session).

```cpp theme={null}
// NOT SAFE — same instance, two threads
Agent agent(config);
std::thread t1([&] { agent.processQuery("query 1"); });  // race condition
std::thread t2([&] { agent.processQuery("query 2"); });  // race condition
```

Similarly, do not call `connectMcpServer()` or `disconnectMcpServer()` while `processQuery()` is running.

***

## Security Model

### Tool Registration Is Explicit

Only tools registered via `registerTool()` or discovered from a connected MCP server are available. There is no reflection, auto-discovery, or dynamic code execution. The LLM can only call tools that your code has explicitly registered.

### Tool Callback Responsibility

The framework **does not validate tool arguments** before passing them to your callback. Each tool is responsible for:

* Validating its input parameters (types, ranges, formats)
* Sanitizing paths and shell arguments
* Rejecting unexpected or dangerous inputs

Example — a safe file-reading tool:

```cpp theme={null}
toolRegistry().registerTool("read_file", "Read a text file",
    [](const gaia::json& args) -> gaia::json {
        std::string path = args.value("path", "");

        // Validate: reject path traversal
        if (path.find("..") != std::string::npos) {
            return {{"status", "error"}, {"error", "Path traversal not allowed"}};
        }

        // Validate: restrict to allowed directory
        if (path.find("/allowed/dir/") != 0) {
            return {{"status", "error"}, {"error", "Access denied"}};
        }

        // Safe to read
        std::ifstream f(path);
        std::string content((std::istreambuf_iterator<char>(f)),
                             std::istreambuf_iterator<char>());
        return {{"content", content}};
    },
    {{"path", gaia::ToolParamType::STRING, true, "File path to read"}}
);
```

### MCP Server Trust

MCP servers are **trusted implicitly** — all tools they expose are registered without review. Only connect to MCP servers you control. In production, audit the tool list returned by each server before deployment.

### Prompt Injection

The LLM decides which tool to call based on user input and conversation history. A malicious user could craft input that causes the LLM to misuse a tool. Mitigations:

* **Validate in the tool callback** — don't trust the LLM's argument choices blindly
* **Use restrictive tool descriptions** — describe exactly what the tool does and what arguments it accepts
* **Limit tool scope** — register only the tools needed for your use case
* **Consider confirmation flows** — for destructive operations, require user confirmation before executing

### Conversation History

Conversation history persists between `processQuery()` calls on the same agent. Previous queries and tool results are visible to subsequent LLM calls. For multi-user scenarios, create a new `Agent` instance per user session to prevent data leakage.

***

## Production Deployment

### Binary Sizes

Measured with MSVC 2022 Release build (x64):

| Artifact                 | Size         | Notes                                                           |
| ------------------------ | ------------ | --------------------------------------------------------------- |
| `gaia_core.lib` (static) | \~18 MB      | Includes statically linked nlohmann\_json and cpp-httplib       |
| Example executable       | \~400-440 KB | Linked against static library                                   |
| Shared library (DLL)     | Smaller      | Build with `-DBUILD_SHARED_LIBS=ON` — ships only framework code |

The static library is large because it bundles all dependencies. When building as a shared library (DLL), the binary is significantly smaller since dependencies are linked dynamically.

### DLL / Shared Library

The framework supports both static and shared library builds. DLL export macros (`GAIA_API`) are already applied to all public classes:

```bash theme={null}
# Build as shared library (DLL on Windows, .so on Linux)
cmake -B build -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release
```

When consuming the DLL, the `GAIA_API` macro automatically switches from `__declspec(dllexport)` to `__declspec(dllimport)`.

### Install Targets

The CMake install target produces a complete SDK package:

```bash theme={null}
cmake --install build --prefix /path/to/install
```

This creates:

```
/path/to/install/
  include/gaia/          # All public headers
  lib/gaia_core.lib      # Library (static or import lib)
  lib/cmake/gaia_core/   # CMake config for find_package()
  bin/gaia_core.dll      # DLL (shared builds only)
```

Consumers use `find_package(gaia_core)` to link against the installed SDK.

### Runtime Configuration

The LLM endpoint can be configured at runtime via environment variable — no recompilation needed:

```bash theme={null}
# Override the default LLM server URL
set LEMONADE_BASE_URL=http://my-server:8080/api/v1
my_agent.exe
```

All other `AgentConfig` fields are set at construction time. For dynamic configuration, read from a config file or registry in your `makeConfig()` function.

### HTTPS Support

HTTPS is enabled automatically when CMake finds OpenSSL on the system
(`find_package(OpenSSL QUIET)` in `cpp/CMakeLists.txt`). If OpenSSL is not
present, GAIA builds with HTTP-only transport and skips the SSL-specific code
paths. There is no `GAIA_ENABLE_SSL` option — to force-disable OpenSSL, pass
the CMake built-in `-DCMAKE_DISABLE_FIND_PACKAGE_OpenSSL=ON`:

```bash theme={null}
cmake -B build -DCMAKE_DISABLE_FIND_PACKAGE_OpenSSL=ON
```

***

## API Quick Reference

### Agent

```cpp theme={null}
class Agent {
public:
    explicit Agent(const AgentConfig& config = {});
    virtual ~Agent();

    // Main execution — blocking, returns {"result": "...", "steps_taken": N}
    json processQuery(const std::string& userInput, int maxSteps = 0);

    // Vision-language (VLM) overloads — send images alongside text.
    // See the Vision Language Models (VLM) section below for usage.
    json processQuery(const std::string& userInput,
                      const std::vector<Image>& images,
                      int maxSteps = 0);
    json processQuery(const std::vector<Message>& messages, int maxSteps = 0);

    // MCP server management
    bool connectMcpServer(const std::string& name, const json& config);
    void disconnectMcpServer(const std::string& name);
    void disconnectAllMcp();

    // Output handler (for custom UI integration)
    OutputHandler& console();
    void setOutputHandler(std::unique_ptr<OutputHandler> handler);

    // Tool registry access
    const ToolRegistry& tools() const;
    ToolRegistry& toolRegistry();

    // System prompt
    std::string systemPrompt() const;
    void rebuildSystemPrompt();  // call after adding tools dynamically

protected:
    virtual void registerTools() {}           // override to register domain tools
    virtual std::string getSystemPrompt() const;  // override for agent-specific instructions
    void init();  // call at end of subclass constructor
};
```

### ToolRegistry

```cpp theme={null}
class ToolRegistry {
public:
    void registerTool(const std::string& name, const std::string& description,
                      ToolCallback callback, std::vector<ToolParameter> params = {},
                      bool atomic = false,
                      std::optional<ToolPolicy> policy = std::nullopt);

    json executeTool(const std::string& name, const json& args);

    const ToolInfo* findTool(const std::string& name) const;
    bool hasTool(const std::string& name) const;
    bool removeTool(const std::string& name);
    size_t size() const;
    void clear();
};
```

### OutputHandler

Subclass to integrate agent output with your own UI. All methods are virtual:

```cpp theme={null}
class OutputHandler {
public:
    virtual void printProcessingStart(const std::string& query, int maxSteps,
                                      const std::string& modelId = "") = 0;
    virtual void printStepHeader(int stepNum, int stepLimit) = 0;
    virtual void printThought(const std::string& thought) = 0;
    virtual void printGoal(const std::string& goal) = 0;
    virtual void printToolUsage(const std::string& toolName) = 0;
    virtual void printToolComplete() = 0;
    virtual void prettyPrintJson(const json& data, const std::string& title = "") = 0;
    virtual void printError(const std::string& message) = 0;
    virtual void printWarning(const std::string& message) = 0;
    virtual void printInfo(const std::string& message) = 0;
    virtual void printFinalAnswer(const std::string& answer,
                                  const UsageStats& usage = {}) = 0;
    virtual void printCompletion(int stepsTaken, int stepsLimit) = 0;
    // ... plus printStateInfo, printPlan, startProgress/stopProgress, and debug methods
};
```

See the [Custom Agent](/cpp/custom-agent) guide for full OutputHandler examples including headless/embedded usage.

### MCPClient

```cpp theme={null}
class MCPClient {
public:
    static MCPClient fromConfig(const std::string& name, const json& config,
                                int timeout = 30, bool debug = false);

    bool connect();
    void disconnect();
    bool isConnected() const;

    std::vector<MCPToolSchema> listTools(bool refresh = false);
    json callTool(const std::string& toolName, const json& arguments);
};
```

***

## Vision Language Models (VLM)

The C++ SDK supports vision-language models (VLMs) via the OpenAI-compatible
`/chat/completions` endpoint. Images are sent inline as base64 data URIs.

### `gaia::Image`

```cpp theme={null}
#include <gaia/types.h>

// Load from disk (MIME auto-detected from magic bytes).
gaia::Image img = gaia::Image::fromFile("photo.png");

// Or from bytes (explicit MIME or auto-detect).
std::vector<std::uint8_t> bytes = readSomewhere();
gaia::Image img2 = gaia::Image::fromBytes(bytes);                 // auto-detect
gaia::Image img3 = gaia::Image::fromBytes(bytes, "image/jpeg");   // explicit
```

Supported formats: `image/png`, `image/jpeg`, `image/gif`, `image/webp`,
`image/bmp`. Unsupported MIME types and empty buffers throw
`std::invalid_argument`.

**Size cap.** `Image::fromFile` rejects files larger than
`GAIA_MAX_IMAGE_BYTES` (default 20 MiB, compile-time override). It also rejects
non-regular files (directories, symlinks, FIFOs, devices) for safety.

### Sending images with `processQuery`

Two new overloads accept images:

```cpp theme={null}
gaia::AgentConfig cfg;
cfg.modelId     = "Qwen3-VL-4B-Instruct-GGUF";
cfg.contextSize = 32768;   // recommended minimum for VLM
gaia::Agent agent(cfg);

// 1) Convenience overload: text + images
gaia::Image img = gaia::Image::fromFile("photo.png");
gaia::json result = agent.processQuery("Describe this image.", {img});

// 2) Caller-composed messages overload (advanced)
std::vector<gaia::Message> msgs = {
    gaia::Message::fromUser("What is in this image?", {img}),
};
gaia::json result2 = agent.processQuery(msgs);
```

**Context size.** VLM models require a large context window — **32768 is the
recommended minimum**. Smaller values (e.g. 2048) will surface a raw server
error as `std::runtime_error` from `processQuery`.

**History semantics.** Both overloads are stateful and symmetric with the
string overload: they read `conversationHistory_` as request context, and
append the input user messages (with image parts **stripped**) plus the
assistant's final answer. Image base64 is never retained in history.

**Thread safety.** `Agent` is **not re-entrant** — concurrent `processQuery`
calls on the same Agent throw `std::runtime_error`. See [Thread
Safety](#thread-safety) above.

### End-to-end example

See [`cpp/examples/vlm_agent.cpp`](https://github.com/amd/gaia/blob/main/cpp/examples/vlm_agent.cpp):

```bash theme={null}
./build/vlm_agent path/to/image.png "What is in this image?"
```

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Overview" icon="eye" href="/docs/cpp/overview">
    Architecture, execution flow, and getting started
  </Card>

  <Card title="Custom Agent" icon="sliders" href="/docs/cpp/custom-agent">
    Custom prompts, typed tools, MCP servers, and output capture
  </Card>

  <Card title="Integration Guide" icon="puzzle-piece" href="/docs/cpp/integration">
    Consume gaia\_core in your own CMake project
  </Card>

  <Card title="Quickstart" icon="rocket" href="/docs/cpp/quickstart">
    Prerequisites, build steps, and running your first demo
  </Card>
</CardGroup>

***

<small style="color: #666;">
  **License**

  Copyright(C) 2025-2026 Advanced Micro Devices, Inc. All rights reserved.

  SPDX-License-Identifier: MIT
</small>