Source Code:
src/gaia/agents/emr/This is Part 3 of 3. If you haven’t completed the previous parts:
- Time to complete: 15-20 minutes
- What you’ll learn: Database design, processing pipeline, system architecture
- Platform: Designed for AI PCs with Ryzen AI (NPU/iGPU acceleration)
System Architecture
Component Overview
| Component | Technology | Purpose |
|---|---|---|
| FileWatcher | watchdog library | Monitors directory for new intake forms |
| VLM Extraction | Qwen3-VL on NPU | Extracts patient data from images/PDFs |
| SQLite DB | sqlite3 | Local storage for patients, alerts, sessions |
| FastAPI Server | FastAPI + uvicorn | REST API + SSE events for real-time updates |
| React Dashboard | React + Vite | Four views: Dashboard, Patients, Chat, Settings |
| Electron Wrapper | Electron.js | Native desktop app experience (optional) |
Database Schema
The agent creates three tables to manage patient data and processing state.patients
The main table storing extracted patient information.| Column | Type | Description |
|---|---|---|
id | INTEGER | Primary key (auto-increment) |
first_name | TEXT | Patient first name |
last_name | TEXT | Patient last name |
date_of_birth | TEXT | DOB in YYYY-MM-DD format |
phone | TEXT | Contact phone number |
email | TEXT | Email address |
address | TEXT | Street address |
city | TEXT | City |
state | TEXT | State/Province |
zip_code | TEXT | Postal code |
insurance_provider | TEXT | Insurance company name |
insurance_id | TEXT | Policy/member number |
reason_for_visit | TEXT | Chief complaint |
allergies | TEXT | Comma-separated list of allergies |
medications | TEXT | Current medications |
emergency_contact_name | TEXT | Emergency contact person |
emergency_contact_phone | TEXT | Emergency contact phone |
is_new_patient | BOOLEAN | TRUE for first visit, FALSE for returning |
processing_time_seconds | REAL | VLM extraction duration |
source_file | TEXT | Original file path |
source_file_hash | TEXT | SHA-256 hash for deduplication |
file_content | BLOB | Binary image data for display |
file_hash | TEXT | File content hash for duplicate detection |
created_at | TEXT | ISO timestamp of record creation |
updated_at | TEXT | ISO timestamp of last update |
alerts
Critical notifications requiring staff attention.| Column | Type | Description |
|---|---|---|
id | INTEGER | Primary key |
patient_id | INTEGER | Foreign key to patients table |
alert_type | TEXT | Type: allergy, missing_field, consent |
priority | TEXT | Priority: critical, high, medium |
message | TEXT | Human-readable alert description |
acknowledged | BOOLEAN | FALSE until staff acknowledges |
acknowledged_by | TEXT | Staff member who acknowledged |
acknowledged_at | TEXT | ISO timestamp of acknowledgment |
created_at | TEXT | ISO timestamp of alert creation |
| Type | Priority | Trigger |
|---|---|---|
allergy | critical | Non-empty allergies field detected |
missing_field | medium | Required field (phone, DOB) is empty |
intake_sessions
Tracks each processing session for audit and analytics.| Column | Type | Description |
|---|---|---|
id | INTEGER | Primary key |
patient_id | INTEGER | Foreign key to patients |
source_file | TEXT | Original file path |
processing_time_seconds | REAL | Extraction duration |
is_new_patient | BOOLEAN | New or returning patient |
changes_detected | TEXT | JSON of field changes for returning patients |
extraction_confidence | REAL | VLM confidence score (0-1) |
created_at | TEXT | ISO timestamp |
Processing Pipeline
The agent follows a 7-step pipeline from file detection to database storage.Pipeline Steps
1
File Detection
Component: FileWatcherMixin with watchdogThe file watcher monitors the directory for new files:Key behaviors:
- Filters by extension (only images/PDFs)
- Debounces events to avoid processing incomplete files
- Runs in background thread
2
Image Conversion
For PDFs: Convert to PNG using PyMuPDF at 2x resolutionWhy 2x resolution? Higher resolution improves VLM extraction accuracy, especially for handwritten text.
3
VLM Extraction
Model: Qwen3-VL-4B-Instruct running on NPU/iGPUWhy Qwen3-VL? Best performance for document understanding on AMD hardware.
4
JSON Parsing
Robust brace-counting parser handles nested objects and VLM quirksWhy custom parsing? VLMs sometimes add explanatory text before/after JSON.
5
Duplicate Check
Match on name + DOB to identify returning patients
6
Change Detection
Compare 10 fields for returning patients
7
Database Storage
INSERT for new, UPDATE for returning
8
Alert Creation
Critical alerts for allergies and missing fields
9
Console Output
Rich formatted success/warning messages
File Deduplication
The agent uses SHA-256 hashing to prevent processing the same file twice.Time Savings Calculation
The agent calculates per-form time savings based on actual extracted data, not fixed estimates.Per-Field Estimation Formula
Example Calculation
A form with 15 fields and ~300 characters:- Base time: 15 × 10 sec = 150 sec
- Typing: 300 × 0.3 sec = 90 sec
- Subtotal: ~240 sec (4 min)
- With verification (+15%): ~276 sec (4.6 min)
Expected Results
| Patients | Manual Time | AI Time | Time Saved | % Saved |
|---|---|---|---|---|
| 10 | ~46 min | ~5 min | ~41 min | ~89% |
| 50 | ~230 min | ~25 min | ~205 min | ~89% |
| 100 | ~460 min | ~50 min | ~410 min | ~89% |
Agent Composition
The Medical Intake Agent combines three GAIA mixins.Mixin Responsibilities
| Mixin | Provides | Methods |
|---|---|---|
| Agent | LLM reasoning loop | process_query(), tool registration |
| FileWatcherMixin | Directory monitoring | watch_directory(), event callbacks |
| DatabaseMixin | SQLite operations | query(), insert(), update() |
Production Considerations
Security
- Local processing only - No PHI leaves the machine
- SQLite file permissions - Set appropriate OS-level access controls
- Watch directory isolation - Don’t share with untrusted users
Performance
- VLM on NPU - Ryzen AI accelerates inference
- Lazy VLM loading - Model loaded only when first file arrives
- Connection pooling - DatabaseMixin reuses SQLite connections
Reliability
- File debouncing - 2-second wait avoids partial file reads
- JSON parsing fallbacks - Handles VLM output variations
- Graceful cleanup - Context manager ensures proper shutdown
Extending the Agent
Add Custom Extraction Fields
Add Custom Alert Types
Summary
The Medical Intake Agent demonstrates several GAIA patterns:- Mixin composition - Combine FileWatcherMixin + DatabaseMixin + Agent
- Event-driven processing - React to file system events
- VLM integration - Extract structured data from images
- Real-time updates - SSE streaming to web dashboard
- Audit trail - Track all processing in intake_sessions table