Skip to main content
Source Code: src/gaia/agents/blender/agent.py · src/gaia/mcp/blender_mcp_server.py
GAIA provides a powerful Blender agent that enables natural language interaction with Blender for 3D scene creation and modification. The agent can create objects, apply materials, manipulate transformations, and manage scenes through conversational commands.
First time here? Complete the Setup guide first to install GAIA and its dependencies.

​
Overview

The GAIA Blender agent bridges the gap between natural language and 3D modeling by:
  • Natural Language Processing: Understanding complex 3D scene creation requests
  • Automated Planning: Breaking down complex tasks into manageable steps
  • Real-time Execution: Direct communication with Blender through MCP (Model Context Protocol)
  • Interactive Workflows: Supporting both example-based learning and custom queries

​
Key Features

​
Scene Management

  • Clear scenes and remove objects
  • Get scene information and object listings
  • Manage scene hierarchy and organization

​
Object Creation

  • Primitive Objects: Cubes, spheres, cylinders, cones, and torus objects
  • Positioning: Precise placement using 3D coordinates
  • Scaling: Custom sizing and proportions
  • Multiple Objects: Create and arrange multiple objects in complex scenes

​
Material System

  • Color Assignment: Apply RGBA colors to objects
  • Material Properties: Set material characteristics and appearance
  • Visual Consistency: Maintain material standards across objects

​
Interactive Planning

  • Multi-step Execution: Break complex requests into logical steps
  • Automatic Planning: AI-driven task decomposition
  • Progress Tracking: Monitor execution through each step
  • Error Handling: Graceful handling of invalid operations

​
Installation & Setup

​
Prerequisites

  1. Blender Installation: Blender version 4.3+ recommended
  2. GAIA Installation: Core GAIA system must be installed
  3. Lemonade Server: Must be running for AI processing

​
MCP Server Setup

The Blender agent requires the MCP (Model Context Protocol) server to communicate with Blender:

​
Step-by-Step Setup:

  1. Open Blender (version 4.3 or newer recommended)
  2. Access Add-ons Menu:
    • Go to Edit > Preferences > Add-ons
  3. Install the MCP Server:
    • Click the down arrow button, then Install...
    • Navigate to: src/gaia/mcp/blender_mcp_server.py
    • Select and install the file
  4. Enable the Add-on:
    • Find Simple Blender MCP in the add-ons list
    • Check the box to enable it
  5. Configure the Server:
    • Open the 3D viewport sidebar (press N key if not visible)
    • Find the Blender MCP panel in the sidebar
    • Set port to 9876 (default) or customize as needed
    • Click Start Server
  6. Verify Connection:
    • The server status should show as running
    • GAIA CLI will validate the connection when starting Blender commands

​
Visual Setup Guide

For detailed setup instructions with screenshots, see: workshop/blender.ipynb

​
Command Reference

​
Basic Command Structure

gaia blender [OPTIONS]

​
Available Options

OptionTypeDefaultDescription
--modelstring”Llama-3.2-3B-Instruct-Hybrid”Language model ID for AI processing (not a Blender 3D model)
--exampleint (1-6)NoneRun a specific example, if not specified run interactive mode
--stepsint5Maximum number of steps per query
--output-dirstring”output”Directory to save output files
--streamflagFalseEnable streaming mode for LLM responses
--statsflagTrueEnable statistics collection (stats saved to output files, not displayed)
--querystringNoneCustom query to run instead of examples
--interactiveflagFalseEnable interactive mode for continuous queries
--debug-promptsflagFalseEnable debug prompts for development
--print-resultflagFalseEnable result printing (output depends on query type)
--mcp-portint9876Port for the Blender MCP server

​
Usage Examples

​
Running Built-in Examples

# Run all Blender examples in sequence
gaia blender

# Run a specific example (1-5 available)
gaia blender --example 2

# Run example with custom model
gaia blender --example 3 --model "custom-model"

​
Interactive Mode

# Start interactive Blender mode for custom 3D scene creation
gaia blender --interactive

# Interactive mode with debug information
gaia blender --interactive --debug-prompts --output-dir ./blender_results

​
Custom Queries

# Single custom query to create specific 3D objects
gaia blender --query "Create a red cube and blue sphere arranged in a line"

# Complex scene setup
gaia blender --query "Clear the scene, then create a green cylinder at (0,0,0) and a yellow cone 3 units above it"

# Advanced scene with multiple operations
gaia blender --query "Create a complex 3D scene with multiple colored objects arranged in a circle"

​
Advanced Configuration

# Use different model with streaming enabled
gaia blender --model "custom-model" --stream --query "Create a sunset scene with mountains"

# Custom MCP port and output directory
gaia blender --mcp-port 9877 --output-dir ./my_scenes --query "Create a modern office space"

# Performance monitoring
gaia blender --stats --steps 10 --query "Build a complete house structure"

​
Built-in Examples

The Blender agent includes several built-in examples to demonstrate capabilities:

​
Example 1: Clearing the Scene

Purpose: Remove all objects from the scene Command: Clear the scene to start fresh Learning: Basic scene management and cleanup

​
Example 2: Creating a Basic Cube

Purpose: Create a red cube at the center Command: Create a red cube at the center of the scene with red material Learning: Object creation and material assignment

​
Example 3: Creating a Sphere with Properties

Purpose: Blue sphere with custom position and scale Command: Create a blue sphere at position (3, 0, 0) with scale (2, 2, 2) Learning: Positioning, scaling, and color properties

​
Example 4: Multiple Objects

Purpose: Green cube and red sphere arrangement Command: Create a green cube at (0, 0, 0) and a red sphere 3 units above it Learning: Multi-object scenes and spatial relationships
Positioning Note: Relative positioning (e.g., “3 units above”) may not always position objects as expected. For precise placement, use explicit 3D coordinates like (0, 0, 3) instead.

​
Example 5: Object Modification

Purpose: Create and then modify a blue cylinder Command: Create a blue cylinder, then make it taller and move it up 2 units Learning: Object modification and transformation workflows Note: Examples 1-5 are currently implemented. Example 6 is planned for future releases.

​
Agent Capabilities

​
Scene Management

  • Clear Scenes: Remove all objects to start fresh
  • Object Inventory: List and identify existing objects
  • Scene Information: Get comprehensive scene details
  • Hierarchy Management: Organize complex scenes

​
Object Creation & Manipulation

  • Primitive Objects:
    • Cubes with customizable dimensions
    • Spheres with radius control
    • Cylinders with height and radius settings
    • Cones with base and tip configuration
    • Torus objects with major and minor radius
  • Positioning: Precise 3D coordinate placement
  • Rotation: Object orientation control
  • Scaling: Non-uniform scaling on X, Y, Z axes

​
Material & Appearance

  • Color Assignment: Full RGBA color control
  • Material Properties: Basic material characteristics
  • Visual Consistency: Standardized material application
  • Material Library: Reusable material definitions

​
Advanced Features

  • Multi-step Planning: Complex task breakdown
  • Dependency Resolution: Handle object relationships
  • Error Recovery: Graceful handling of invalid operations
  • Progress Monitoring: Step-by-step execution tracking

​
Interactive Mode

Interactive mode provides a continuous interface for 3D scene creation:

​
Starting Interactive Mode

gaia blender --interactive

​
Interactive Commands

  • Scene Creation: Describe the scene you want to create
  • Object Modification: Request changes to existing objects
  • Scene Queries: Ask questions about the current scene
  • Control Commands:
    • Type exit, quit, or q to exit
    • Use Ctrl+C for immediate termination

​
Example Interactive Session

Enter Blender query: Create a red cube at the origin
[Agent processes and creates cube]

Enter Blender query: Add a blue sphere 2 units above the cube
[Agent adds sphere with proper positioning]

Enter Blender query: Make the cube twice as large
[Agent modifies cube scale]

Enter Blender query: exit
Exiting Blender interactive mode.

​
Requirements & Dependencies

​
System Requirements

  • Blender: Version 4.3+ (4.2 may work but not fully tested)
  • Python: Compatible with GAIA’s Python environment
  • Memory: Sufficient RAM for both GAIA and Blender (8GB+ recommended)
  • Storage: Space for 3D scene files and outputs

​
Software Dependencies

  • GAIA Core: Full GAIA installation required
  • Lemonade Server: Must be running for AI processing
  • Blender MCP Server: Must be installed and running in Blender

​
Network Requirements

  • Local Communication: MCP server runs on localhost
  • Port Availability: Default port 9876 (customizable)
  • Firewall: Allow local connections on MCP port

​
Troubleshooting

Error: Connection errors or “server not accessible” messagesSolution:
lemonade-server serve
Error: “Blender MCP server is not running or not accessible”Solutions:
  1. Verify Blender is open with the MCP add-on installed
  2. Check the MCP server is started in Blender’s sidebar panel
  3. Confirm port 9876 is available (or use custom port with --mcp-port)
  4. Restart Blender if the server appears unresponsive
Error: MCP server add-on not found or won’t installSolutions:
  1. Verify you’re navigating to the correct file: src/gaia/mcp/blender_mcp_server.py
  2. Check Blender version compatibility (4.3+ recommended)
  3. Ensure GAIA is properly installed
  4. Try restarting Blender after installation
Error: Slow response times or timeoutsSolutions:
  1. Reduce --steps parameter for simpler operations
  2. Use smaller models for faster processing
  3. Ensure adequate system resources
  4. Check for memory constraints in both GAIA and Blender
Error: MCP server won’t start due to port conflictsSolutions:
  1. Use --mcp-port to specify an alternative port
  2. Check for other applications using port 9876
  3. Kill conflicting processes: gaia kill --port 9876
Enable debug mode for detailed troubleshooting:
gaia blender --debug-prompts --interactive --logging-level DEBUG
This provides:
  • Detailed prompt information
  • Step-by-step execution logs
  • Error stack traces
  • Communication details between GAIA and Blender
For additional support:
  1. Check the workshop notebook: workshop/blender.ipynb
  2. Review the Development Guide
  3. Consult the FAQ for common solutions
  4. Enable debug mode to gather detailed information

​
Advanced Usage

​
Custom Models

The Blender agent supports different AI models for varying performance characteristics: 
# Use a more powerful model for complex scenes
gaia blender --model "Llama-3.2-8B-Instruct" --query "Create a detailed cityscape"

# Use a smaller model for simple operations
gaia blender --model "Llama-3.2-1B-Instruct" --query "Create a red cube"

​
Batch Operations

Process multiple scenes or operations efficiently: 
# Save results to organized output directory
gaia blender --output-dir ./batch_scenes --query "Create scene 1: office setup"

# Use streaming for real-time feedback
gaia blender --stream --query "Build a complex architectural structure"
Output Directory Behavior: The --output-dir creates the specified directory, but output files depend on the specific operations performed. Some queries may not generate output files even with this flag set.

​
Integration with Other Tools

The Blender agent can be integrated into larger workflows:
  • Automated Content Creation: Script multiple scene generations
  • Educational Tools: Demonstrate 3D concepts through natural language
  • Rapid Prototyping: Quick visualization of 3D ideas
  • AI-Assisted Modeling: Accelerate traditional 3D modeling workflows

​
Best Practices

​
Query Construction

  • Be Specific: Include precise positions, colors, and sizes
  • Use Clear Language: Avoid ambiguous terms
  • Break Complex Requests: Split large scenes into manageable parts
  • Specify Relationships: Clearly describe object positioning relative to others

​
Scene Organization

  • Start Clean: Begin with Clear the scene for predictable results
  • Logical Progression: Build scenes step by step
  • Test Incrementally: Verify each step before adding complexity
  • Use Consistent Naming: Reference objects clearly in follow-up commands

​
Performance Optimization

  • Appropriate Models: Match model size to task complexity
  • Step Limits: Set reasonable --steps values
  • Resource Monitoring: Use --stats to track performance
  • Batch Similar Operations: Group related tasks together

​
Future Enhancements

Planned improvements to the Blender agent include:
  • Extended Object Library: More primitive types and complex shapes
  • Advanced Materials: PBR materials, textures, and lighting
  • Animation Support: Keyframe animation and motion paths
  • Scene Templates: Pre-built scene configurations
  • Import/Export: Integration with external 3D file formats
  • Collaborative Features: Multi-user scene editing capabilities