📖 You are viewing: Conceptual Guide - Learn how agents work and build your first agentSee also: API Specification · Quickstart Tutorial
Source Code:
src/gaia/agents/base/agent.pyWhat You’ll Learn
- What an agent is and why it’s more powerful than calling an LLM directly
- How the agent reasoning loop works under the hood
- How to build your first agent from scratch
- How to configure agents for different use cases
- Common pitfalls and how to avoid them
What is an Agent?
An agent is more than just an LLM. Think of the difference like this:
Real-world analogy: An LLM is like a brilliant consultant who can only give advice. An agent is that same consultant, but now they have a phone, a computer, and access to your company’s systems—they can actually do the work.
Why Use Agents?
- Without Agent
- With Agent
How Agents Work: The Reasoning Loop
When you send a message to an agent, it doesn’t just generate a response. It enters a reasoning loop:Key Insight: The loop can repeat! If the LLM decides it needs more information after step 5, it goes back to step 3 and executes another tool. This enables complex multi-step reasoning.
Agent States
During processing, agents transition through different states:
You can check the current state in your tools if you need conditional behavior:
Building Your First Agent
Let’s build an agent step by step, starting simple and adding features progressively.Step 1: The Minimal Agent
Every agent needs three things:- A system prompt - Tells the LLM who it is
- A console - Handles output display
- Tools - What the agent can do
- Agent receives “Hello! What can you help me with?”
- LLM sees: System prompt + User message
- LLM generates a conversational response
- Agent returns the response
Step 2: Adding Your First Tool
Let’s give the agent the ability to tell time:- User asks “What time is it?”
- LLM sees the
get_current_timetool and its description - LLM decides: “This matches ‘What time is it?’ - I should use this tool”
- Agent executes
get_current_time(), gets{"time": "2:30 PM", ...} - LLM receives the result and generates a natural response
Step 3: Adding External Capabilities
Now let’s build something more practical—an agent that can fetch weather data:- Instance variables (
self.api_key) - Store configuration in__init__ - Tool parameters with defaults (
country_code="US") - LLM learns optional params - Error handling - Return error info, don’t raise exceptions
- Timeout handling - External APIs can be slow
- Rich system prompt - Guides LLM behavior beyond just tool selection
Configuration Deep Dive
Agents accept many configuration parameters. Here’s what each one does:Choosing Your LLM Backend
Option 1: Local LLM (Default) - Free, Private, Fast
Option 1: Local LLM (Default) - Free, Private, Fast
Uses AMD-optimized Lemonade Server running on your machine:Pros:
- Free (no API costs)
- Data stays on your machine (privacy)
- Fast inference on AMD hardware (NPU/iGPU acceleration)
- Requires Lemonade Server running
- Smaller models than cloud options
- Local compute resources needed
Option 2: Claude API - Best Reasoning
Option 2: Claude API - Best Reasoning
Uses Anthropic’s Claude models:Pros:
- Excellent reasoning and code understanding
- Large context window
- Most capable for complex tasks
- API costs (pay per token)
- Requires internet connection
- Data sent to Anthropic
Gateway models through Lemonade
Gateway models through Lemonade
The direct OpenAI and LiteLLM adapters have been removed. Configure the upstream
model in Lemonade and pass its server URL and catalog model ID to the agent.
See gateway migration. Legacy
use_chatgpt=True calls raise an actionable error rather than selecting another
provider.Tuning Agent Behavior
Large tool results
Every tool result passes through one character budget before it reaches the model. A result at or below the threshold is sent unchanged. A larger one is archived whole and reaches the model as a chunk index:shown— the parts kept verbatim, as{offset, text}segments.index— one{n, label, offset, length}per part left out, numbered from 1 in document order.read_tool_output(artifact, entry=n)returns exactly that part; the result’sfetchfield says so.artifact,continuation,total_chars,truncated,original_chars,omitted_chars— as before.archived_fieldnames the field the archive holds (content,stdout,matches); every other field is kept, includingcheck_result.
Shown holds the closing part when the result marks it a lead (a run’s summary
line), then the first part, then the other parts marked as leads or named by
the call’s arguments, then the rest in order, while they fit. Text with no structure
at all (a single long line) keeps a
head/tail excerpt, with the middle as
its one index entry.
Local thresholds/targets are 30,000/20,000 characters for NPU and 60,000/40,000
for GPU/CPU. An omitted device follows the configured profile; FLM models retain
the NPU budget. A cloud model reached through Lemonade gets the conservative
30,000/20,000 without reading the local device profile. The Claude path uses its existing larger window. These are
per-result character budgets, not measured remaining tokens.
Evicting stale tool results
On a large repository the context an agent re-sends each step grows to 130–195K tokens, and most of it is old tool results the model has already acted on. Context eviction replaces such results, in the list sent to the model only, with a one-line stub that names where the full text is:read_tool_output; a result that already arrived as a chunk
index keeps its handle and index. The turn’s conversation log — what
transcripts and the Agent UI show — is never touched. It is off by default.
Whether eviction saves money depends on the provider’s cached-versus-uncached
input price: every eviction changes the prompt prefix, so the next call
re-reads everything after the eviction point uncached once. Simulated on real
per-step counts, Kimi K2.7 (cached $0.19/M vs $0.95/M) saves 27%, GLM-5.3
Flash 11%, and DeepSeek V4.1 Flash (0.006 vs 0.30) loses. So the policy fires
rarely and in batches:
context_eviction—off(default),on, orauto.autois on only for a cloud model whose cached/uncached input price ratio, from the table ingaia.llm.cache_pricing, is at least 0.1; a local model or one not in the table is off, said once at info level, never guessed.context_eviction_threshold_tokens(60,000) — nothing is evicted until the last measured prompt (prompt_tokensfrom the previous step) is over this.context_eviction_keep_steps(8) — results from the last this-many steps are never evicted.context_eviction_min_batch_tokens(30,000) — the evictable results must add up to at least this (chars / 3.6) before they go, all in one batch.
GAIA_CONTEXT_EVICTION, GAIA_EVICT_THRESHOLD, GAIA_EVICT_KEEP and
GAIA_EVICT_MIN_BATCH override the four settings; a malformed value is an
error at construction. Eligibility is judged from each result’s own text (a
stub already, or not), never a remembered id — a result the turn log replays
at full size on a later turn (the log always keeps the pre-eviction copy) is
judged fresh, so a long session keeps getting evicted rather than drifting
back to full size after its first turn. read_tool_output pages and
delegate_task results are never evicted, and user and assistant messages
are never changed. Each batch adds evicted_results and evicted_tokens_est
to that step’s stats entry in the conversation log and logs one info line.
When the backend refuses a request as too long, the overflow retry runs once,
eviction on or off. It moves every older tool result out of the request the same
way, archived whole with a stub that names the read_tool_output call that reads
it back. A large result from the latest step is cut to its first 2,000 characters
plus a handle to the rest; a read_tool_output page the model just asked for, a
refusal, and a delegate_task answer stay whole. Assistant text over 800
characters is clipped, and any lesson memory learned this turn is restated on the
last user message. The retry runs only when the shrink frees more than the
request was over by (read from the backend’s error), or, when the error gives no
numbers, at least a tenth of the request. Otherwise nothing is archived and the
turn ends with the message that the request needs more than fits, rather than
repeating a refusal.
Common Pitfalls and Solutions
Pitfall 1: Agent doesn't use my tools
Pitfall 1: Agent doesn't use my tools
Symptom: Agent gives generic responses instead of using your tools.Cause: Tool docstrings don’t clearly explain when to use them.
Pitfall 2: Agent gets stuck in loops
Pitfall 2: Agent gets stuck in loops
Symptom: Agent keeps calling tools repeatedly without making progress.Cause: Usually means
max_steps is too high, or tools return ambiguous results.What the user gets: at max_consecutive_repeats identical calls the model gets one correction in place of the call, and a repeat after that ends the turn. If the calls that ran worked, the agent makes one more model call with no tool calls allowed and answers from their results; if they errored or were refused, the answer names that failure.Two loops that vary their arguments are stopped the same way:- One tool failing with new arguments each time. After
max_consecutive_repeatsrejections of the same tool, with no file changed in between, the next call to it is not run and the model is told to change approach; calling it again ends the turn. A command that ran and exited non-zero (a failing test run) does not count. - New work after the answer. Once the turn has produced an answer, a call that touches nothing the request touched (no file it named, read or changed, and not a test run) is not run, and the model is told to finish and offer the work instead. A second such call ends the turn with a closing answer to the original request.
Pitfall 3: Tool errors crash the agent
Pitfall 3: Tool errors crash the agent
Symptom: Agent stops working when a tool encounters an error.Cause: Raising exceptions in tools instead of returning error information.
Pitfall 4: Context getting too long
Pitfall 4: Context getting too long
Symptom: Agent becomes slow or starts giving inconsistent responses.Cause: Conversation history or tool results exceeding context limits.Solutions:
Practice Challenge
Build a File Explorer Agent
Create an agent that can:
Why this solution works:
- List files in a directory
- Read file contents
- Search for text in files
- Handle errors gracefully (missing files, permission denied)
- Limit file content to prevent context overflow
- Clear tool docstrings that guide the LLM
Hints
Hints
- Use
os.listdir()for listing files - Use
os.path.isfile()to check if path is a file - Catch
FileNotFoundErrorandPermissionError - Return structured dicts with status fields
Solution
Solution
- Security: Validates paths stay within allowed directory
- Error handling: Returns informative error dicts instead of raising
- Context limits: Truncates large results to prevent overflow
- Clear docstrings: LLM knows exactly when to use each tool
- Configurable:
allowed_pathrestricts scope for safety
Deep Dive: Under the Hood
process_query() also returns model_messages: a deep-copied, model-facing trace
of the current turn, including native tool calls and their results. It excludes
prior conversation history and the system prompt. Hosts can persist it and use
complete turns to populate conversation_history on later calls. Overflow retry
trimming does not replace the retained evidence with stubs. Calls interrupted
before a result was recorded receive an explicit incomplete-result message;
this does not claim they executed successfully.
The flagship TUI persists these traces in private SQLite files under
~/.gaia/tui/history/ (GAIA_TUI_HOME/history/ when configured). Each launch
starts a separate session. Set GAIA_HISTORY_FILE=/path/session.sqlite3 to
explicitly resume the same transcript, or pass --history-file to the stdio
agent. A turn is written to the archive when it starts, marked as the running
process’s, and replaced by its full trace when it ends. A turn cut short by the
TUI closing (stdin reaching end of file, or the output pipe breaking) ends with
“This answer was interrupted when GAIA closed.” So does one the TUI had to
kill, so a resumed session knows that turn never finished. A process sharing
the file leaves out another live process’s running turn. A turn that failed
while the TUI was open leaves no entry. This archive contains tool output,
including local file contents.
Delete the file to remove it. A host calling conversation_history.clear()
erases both the visible history and its archived turns.
The next prompt admits complete turns using the active model’s context budget,
reserving space for the system prompt, tool schemas, current question and answer.
Older turns are evicted in blocks from the model view, never cut into invalid
call/result pairs. The archive remains intact across context eviction and model
switches. prepare() and select_history() take an optional on_cut callback
that receives a HistoryCut(turns, tokens) when more turns are hidden than
before. select_history() alone compares with the view minus the newest turn;
pass a CutReporter (as SessionHistory.prepare() does) to compare with the
last reported view, so a switch to a smaller model is reported too. The TUI and
Agent UI use it to tell the user. The terminal event is sent only after the
completed trace is committed; a storage failure is reported as an error rather
than a successful saved turn.
How process_query() Works Internally
How process_query() Works Internally
When you call Key insight: The conversation history grows with each tool call, giving the LLM more context for its next decision.
agent.process_query(user_input), here’s the detailed flow:How Tool Registration Works
How Tool Registration Works
The The function name, type hints, and docstring are what the LLM sees, which is why
they matter so much. See the Tools guide for the authoritative
walkthrough of the registry entry.
@tool decorator does several things:Verification Scope
When the agent can’t confirm its work, the answer says so in one plain line. Without it, “done” reads the same whether the agent ran the test suite or ran nothing at all, and the only rational response is to distrust every completion message.
The tool that ran the check says so.
run_shell_command, run_python, and
execute_python_file put a check_result on every result for a call that ran:
a CheckResult (label, target, kind, passed, summary) when the program
was a test, lint, typecheck, or build runner, and null when it was not. The
note reads that and nothing else. A shell run counts only when the program
itself is the runner (pytest -q, python -m pytest, uv run pytest), so
cat pytest.ini is not a test run; a failing summary line wins over a zero exit
code, so pytest | tail that failed reads as failed.
Results that carry no check_result — MCP tools, hub agents built before the
field existed — are still classified from their text: a tool named run_tests,
run_lint, typecheck, or build, a command argument naming a known runner,
or a run_python / execute_python_file output with a test runner’s summary
(pytest, unittest, jest, vitest, mocha, go test, cargo test). Other
unrecognized runners don’t count as a check.
Before the answer goes out, the loop also checks it against the same record. An
answer that reports a test outcome (111 passed, “all tests pass”) when no test
run passed after the last file edit gets one correction naming the gap. If the
claim remains unsupported, or no correction step remains, the framework replaces
it with an explicit incomplete result. Unsupported claims never become successful
results merely because the correction budget ran out.
A check that ran more than once in the turn is judged by its most recent run. A
test that failed and then passed after a fix is verified and gets no note; one
that passed and then failed after an edit reads pytest didn't pass. Only a
rerun of the same command replaces an earlier run: if pytest tests/ failed and
pytest tests/test_cart.py then passed, the note still reports the failure.
Grounding checks
Four more checks compare the answer, and the files the turn wrote, with the turn’s tool record (gaia.agents.base.grounding, gaia.agents.base.deliverables). They exist because small local models answer
without looking and report work they never did:
Each check except file asks for at most one correction per turn, before the answer is
marked answered, so the lookup it asks for is not refused as off-request work.
An answer that already admits the gap (“I haven’t opened it”, “these figures
are unverified”) is not corrected. If a gap survives its correction, or no step
is left, the answer is kept as written and what is unsupported becomes part of
the same “not confirmed” note, never a second one:
result["verification"]["ungrounded"].
Unlike an unsupported test claim, the answer is not replaced: these checks read
prose and can be wrong, so they flag rather than rewrite.
Saved output and cleanup evidence
File completion is checked separately from the test/lint/build note. A save claim must name an output backed by this turn’s successful write to that path; a transcript or screenshot written elsewhere cannot satisfy it. A save request that names a file, folder or disk is checked even when the answer silently omits the save; a pathless “save it” is checked through the answer’s own save claim. Negated instructions, how-to questions, idioms (“save me some time”) and topics (“write a guide to Node.js”) do not create save obligations, and a reference to a save from an earlier session is not a claim about this turn. “Create”, “make” or “generate” names an output through its direct object (notes.md, not
“a Node.js app”); a file after “in” may be a source, so it is checked only
through the answer’s claim. An answer’s save claim makes the turn
incomplete when the user asked for a save, a file-writing tool ran, or “I” or
“we” is the one saving. Any other save claim gets one correction and then
stands, so “downloads are saved to ~/Downloads” stays an answer.
Requested and claimed text outputs must be read back after their latest
mutation; an ordinary edit the user did not ask to save does not need readback. A read before
a rewrite, a file listing, a binary metadata summary, or only the last page is
insufficient. read_file pages and archived read_tool_output continuations
must cover the complete output. Source-code edits without a save instruction
or claim retain the existing test-verification policy. Binary outputs require
path-specific write evidence but are not forced through a text reader. A script
output larger than 64 KB may instead be checked in part after its latest write:
a non-empty range read, or a later successful run that names the file and
prints something (wc -l, head, a row count). A run that failed, printed
nothing, or rewrote the file is not a check, and smaller outputs are read whole.
For Python and shell executors, the loop compares file metadata before and after
the call for requested and previously tracked targets, plus literal Python path
candidates. These probes respect the read boundary and never read file contents.
A successful executor call alone is not proof of a save; a failed call that
changed an output invalidates earlier evidence. A file last modified during a
successful executor call, such as ls > listing.txt, counts as written once it
is read back; one touched by a failed call does not, and such a file proves only
its own path, never a pathless “saved it to a file” claim. A requested file or
folder that no tool reports writing is checked the same way, so a script that
builds its output paths at run time still fulfils the save; files in the folder
are found only within the read boundary. A bare folder name in the answer
(“saved in reports”) is fulfilled by files written inside any folder of that
name. Cached transcripts are not new writes.
Cleanup claims likewise need concrete paths observed disappearing during the
turn; a general statement that everything was cleaned up is not proof. A claimed
path that is absent and untouched already agrees with the claim, and removal from
an index or list is not a file deletion.
The loop offers one bounded correction for missing artifact evidence. That
correction reopens the turn, so writing the output, running the script that
produces it, and reading it back are not refused as work after the answer;
files inside a folder the request or the correction names are always in scope.
If work or readback remains missing, the emitted answer names the gaps and process_query
returns status: "incomplete" with completion_gaps. When the turn ended on an
error, such as an unreachable server, the error message and failed status are
kept and the gaps are listed beneath it. A blocked save cannot be
reported as success on the last step or after a repeated unsupported answer.
Readback establishes observed output; it does not measure semantic correctness
or recall when extracting every item from a long document.
The statement rides in the emitted answer, so it reaches every surface — the
CLI console, the SSE answer event, and process_query’s return value — and
it is capped at VERIFICATION_SCOPE_MAX_CHARS (200) so it cannot grow the
prompt as conversation history accumulates.
A turn cancelled from the UI returns an empty result on purpose, and carries no
statement — filling it in would make a cancelled turn look like a completed one.
Proactive Lifecycle Hooks
Beyond responding to a user prompt, an agent can act proactively — proposing work on first run or during a steady-state loop. The baseAgent exposes three
methods for this (issue #1484):
All three are no-ops by default — a plain agent never acts on its own, and
process_query() never triggers them. You opt in by overriding the hooks.
Every proposal requires explicit user approval before anything runs.
propose() parks the proposal in pending_approval; nothing executes until the
user accepts it. Proposals also carry a risk tier (low / medium / high
/ critical) — it’s stored as signal to help the approving user prioritize, but
it never gates execution: even risk="low" proposals wait for approval, per the
autonomous agent spec.
Example: a proactive workspace agent
Key Takeaways
Agents = LLM + Tools + Memory
Agents can plan, act, observe, and reason—not just generate text.
Three Required Methods
Every agent needs:
_get_system_prompt(), _create_console(), _register_tools()Docstrings Teach the LLM
The LLM only sees your docstring—make it clear when to use each tool.
Return Errors, Don't Raise
Tools should return error info as data so the agent can recover gracefully.
Next Steps
Tool Decorator
Deep dive into creating effective tools
Console Output
Customize how your agent displays output
User Guides
See guides for the GAIA agent, chat, voice, and skills
Recovering large tool outputs
When a result exceeds the model budget, it carries anartifact handle, a
numbered index of the parts left out, and continuation: read_tool_output.
Call read_tool_output(artifact, entry=n) to get entry n verbatim. The index
names every omitted part, so paging through a whole output is almost never
needed. A part longer than 8,000 characters comes back in pages: the page sets
remaining, and read_tool_output(artifact, offset=next_offset, limit=remaining) continues it. Without an entry, offset and limit (default
2,000) read any span. An entry number outside the index is an error that names
the valid range. Unicode offsets count characters, not bytes.
The archived text is the original string, the field named by
archived_field, or JSON (one record per line) for a structured result. Small
results remain unchanged. Structured excerpts retain whole records.
Shell, Python execution and browser excerpts retain omitted output before
applying their own display limits. File readers accept offset and limit
for bounded text pages, including files too large for a full read. Their normal
path permissions still apply; paging does not authorize a new path.
Output handles belong to the producing agent process, expire after one hour,
and share a 64 MiB archive quota. Unknown, expired, cross-session handles and
quota exhaustion return explicit errors. Restarting an agent invalidates its
output handles; rerun the source tool if the session has ended. A continuation
reads captured output rather than re-executing a potentially state-changing tool.
Exhaustive document extraction
For explicit requests to list every item in a text document, the agent offersextract_document_items(file_path, fields=None). It reads a permission-checked
snapshot and processes 4,000-character pages sequentially, each widened by up to
600 characters of overlap on either side, with an additional omission check per
page. Each call has isolated context and no tools. Requested fields, written in
the request as fields: name, reps, cue or passed as the fields list, must
appear on every extracted entry; missing source facts are marked as not stated.
A fields list that differs from the request’s is rejected rather than merged.
The framework retains entries with exact source quotations and character spans,
checks the source fingerprint again before completion, and appends the accumulated
inventory to the final answer. A final model summary cannot drop retained entries.
Ambiguous evidence keeps both entries and marks the possible repeat rather than
dropping one. Reading every page without extracting an inventory is insufficient.
save_extracted_items(file_path) exports the retained inventory as JSON, CSV, or
readable text, using the existing write permission gate. JSON records carry each
requested field under fields, and CSV gives each field its own column (prefixed
field: if it collides with a built-in column; cells that would open as formulas
are escaped). Before completion the framework reads each saved inventory back in
full and requires an exact match with the retained entries, so the model need
not page through it; hand edits to a file it wrote this turn are refused. Every requested source
must finish; a failed page, cancellation, source change, or exceeded budget yields
an incomplete result. Limits are 256,000 source characters, 512 entries, 40,000
characters per extraction prompt, and 100,000 characters of retained item text and
quotes, with a 30-minute extraction deadline (the enclosing tool timeout adds 60 seconds).
These bounds fail visibly rather than silently dropping data.
This verifies source coverage and preserves the extracted set; it does not prove
semantic recall of every implicit item. Recall must be measured against known
source items. General summaries, advice about extraction, negated enumeration
requests, analysis of code files (“find all bugs in main.py”), binary documents
such as PDFs, and agents without a file permission validator do not activate this
path. An active extraction skill registers the tools, but enforces coverage only
when the request names a source document or asks for every item; a redaction or
rewrite that happens to load the skill is not checked. Sources are existing files
a request reads from. Every file the request names as a deliverable, and every
file the turn wrote, is an output and never a source; a name that is not a file
(“Node.js”) is a topic. Data files (CSV, JSON, logs, YAML, XML) are left to code
tools. When a source still lacks an extraction, the answer is kept and a short
“I couldn’t confirm all of this” list follows it.
Existing memory hooks record extraction actions and compact provenance (source
fingerprint, page coverage and item count), while conversation history retains the
rendered inventory as one bounded record, bypassing the ordinary 4,000-character
turn cap only for extraction answers (maximum 256,000 characters; a longer answer
is not stored, and an error is logged). Follow-ups can retrieve those observations through existing
conversation search. Private mode suppresses persistence. A remembered summary
never substitutes for reading and validating the current source; no separate
memory database or cloud sync is introduced.
The default gaia-voice skill supplies a short routing rule; the detailed
document-extract procedure is loaded for the task. The implementation uses
schema-constrained page responses and an omission check per page. Overlap keeps
source text visible and aligns pages to whole lines, sentences, or (for unpunctuated
captions) words. Quotes are matched ignoring case and spacing (captions use
non-breaking and doubled spaces), but only on whole words, and the stored quote is
always the source’s own text at its exact offsets.
An item’s identity value is a field named for a name, id or title (such
as exercise_name), else the first requested field. When that value is copied
from the quote, its source span locates the item; when the model composes a label
instead (“Big circles (arms)”), the quote locates the item, and a label sharing no
words with its quote is kept but marked “(label not in quote)”. Descriptions may
paraphrase; the verbatim quote is their evidence. An entry that states no field
carries nothing and is ignored.
Merging never fails a run and never drops an item. Two extractions are one item
when they sit at the same place (the same name span, or quotes sharing at least
half their text) and their names match or nest (“march” and “march in place” at
one spot keep the fuller name). Different names are different items, and
differing descriptions of one item are both kept, joined with /. Items from
one reply are never merged. A quote that names its item twice takes an occurrence
no earlier item claimed and is never merged, so its values cannot land on the
wrong item. Evidence too ambiguous to
merge keeps both entries, and the rendered inventory marks the later one
“(may repeat item N)” for review, including when its label shares words with an
overlapping item’s. Only an unusable reply (not JSON, a quote not
in the page, an unfinished page) is retried once and then fails visibly.
Each page call allows 16,384 output tokens. Cloud models are asked for low
reasoning effort, and a page that still spends the whole budget reasoning is
retried once with reasoning off; local models get no anti-repetition penalties,
which work against verbatim copying.
Run the explicit live regression with either backend: