> ## Documentation Index
> Fetch the complete documentation index at: https://amd-gaia.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Integration

<Info>
  **Source Code:** [`src/gaia/llm/`](https://github.com/amd/gaia/blob/main/src/gaia/llm/)
</Info>

<Note>
  **Import:** `from gaia.llm import LLMClient`
</Note>

***

<Badge text="development" color="orange" />

**Detailed Spec:** [spec/llm-client](/docs/spec/llm-client)

**Purpose:** Unified client for local and cloud LLM providers.

## LLM Client

`LLMClient` is an abstract base class. Use `create_client()` to get the right provider:

```python theme={null}
from gaia.llm import create_client

# Local LLM (Lemonade server — default)
llm = create_client()

# Generate response
response = llm.generate(
    prompt="What is AI?",
    model="Qwen3-0.6B-GGUF"
)
print(response)

# Streaming response
for chunk in llm.generate(prompt="Tell me a story", stream=True):
    print(chunk, end="", flush=True)

# Chat completions format
messages = [
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi! How can I help?"},
    {"role": "user", "content": "Tell me about Python"}
]

response = llm.chat(
    messages=messages,
    model="Qwen3-0.6B-GGUF"
)
print(response)
```

***

## Cloud Providers

```python theme={null}
from gaia.llm import create_client

# Claude API
llm_claude = create_client(use_claude=True)

# OpenAI API
llm_openai = create_client(use_openai=True)

# LiteLLM gateway — 100+ providers (Bedrock, Vertex AI, Groq, Azure, ...)
# Requires: pip install 'gaia[litellm]'
llm_litellm = create_client("litellm", model="anthropic/claude-sonnet-4-6")

# Use same interface
response = llm_claude.generate("Explain Python decorators")
```

***

## Lemonade Client (AMD-Optimized)

**Import:** `from gaia.llm.lemonade_client import DEFAULT_MODEL_NAME, DEFAULT_LEMONADE_URL`

```python theme={null}
from gaia.llm.lemonade_client import DEFAULT_MODEL_NAME, DEFAULT_LEMONADE_URL

# Default configuration
MODEL = DEFAULT_MODEL_NAME  # "Gemma-4-E4B-it-GGUF"
URL = DEFAULT_LEMONADE_URL   # "http://localhost:13305/api/v1"

# Used internally by LLMClient when no provider specified
```

***

## Related Topics

* **[Agent SDK](./chat)** - Higher-level chat interface
* **[Configuration](../configuration)** - Setting up API keys and URLs
* **[Agent System](../core/agent-system)** - Using LLMs in agents

***

<small style="color: #666;">
  Copyright(C) 2024-2026 Advanced Micro Devices, Inc. All rights reserved.

  SPDX-License-Identifier: MIT
</small>