LLM Integration

Source Code: src/gaia/llm/

Import: from gaia.llm import LLMClient

Detailed Spec: spec/llm-client Purpose: Unified client for local and cloud LLM providers.

LLM Client

LLMClient is an abstract base class. Use create_client() to get the right provider:

from gaia.llm import create_client

# Local LLM (Lemonade server — default)
llm = create_client()

# Generate response
response = llm.generate(
    prompt="What is AI?",
    model="Qwen3-0.6B-GGUF"
)
print(response)

# Streaming response
for chunk in llm.generate(prompt="Tell me a story", stream=True):
    print(chunk, end="", flush=True)

# Chat completions format
messages = [
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi! How can I help?"},
    {"role": "user", "content": "Tell me about Python"}
]

response = llm.chat(
    messages=messages,
    model="Qwen3-0.6B-GGUF"
)
print(response)

Cloud Providers

Use create_client(use_claude=True) for the Claude provider. Local and on-prem models use create_client("lemonade", base_url=..., model=...).

Gateway migration

The openai and litellm provider adapters have been removed because they silently discarded tool calls. The litellm install extra is also removed. The OpenAI Python package remains a dependency of the Lemonade client. use_openai=True, use_chatgpt=True, and --use-chatgpt are retained only as migration-error shims: they fail before starting inference. They never switch your request to another provider. Remove these options after configuring a model in Lemonade. --use-chatgpt is no longer advertised in CLI help. Configure the upstream endpoint and credentials in your Lemonade server first, then select the model ID exposed by its catalog. base_url below is the Lemonade server URL, not the upstream OpenAI-compatible endpoint:

from gaia.llm import create_client

client = create_client(
    "lemonade",
    base_url="http://localhost:13305/api/v1",
    model="amd.Gemma-4-31B",  # use the ID in your configured server's catalog
)
response = client.chat([{"role": "user", "content": "Hello"}])

In the TUI, select your configured local, Fireworks, or AMD gateway model in Providers. The shared Lemonade adapter preserves native tool calls. A generic OpenAI-compatible server is not a drop-in replacement for Lemonade’s lifecycle and catalog API. Credentials belong in the server/provider configuration, not in prompts or model IDs.

Lemonade Client (AMD-Optimized)

Import: from gaia.llm.lemonade_client import DEFAULT_MODEL_NAME, DEFAULT_LEMONADE_URL

from gaia.llm.lemonade_client import DEFAULT_MODEL_NAME, DEFAULT_LEMONADE_URL

# Default configuration
MODEL = DEFAULT_MODEL_NAME  # "Gemma-4-E4B-it-GGUF"
URL = DEFAULT_LEMONADE_URL   # "http://localhost:13305/api/v1"

# Used internally by LLMClient when no provider specified

Agent SDK - Higher-level chat interface
Configuration - Setting up API keys and URLs
Agent System - Using LLMs in agents

Getting Started

C++ Framework

Python Framework

LLM Client

Cloud Providers

Gateway migration

Lemonade Client (AMD-Optimized)

​LLM Client

​Cloud Providers

​Gateway migration

​Lemonade Client (AMD-Optimized)

​Related Topics

LLM Client

Cloud Providers

Gateway migration

Lemonade Client (AMD-Optimized)

Related Topics