Agno Database Architecture - agno_db Design¶

Overview¶

This document describes the architecture decision to use a dedicated agno_db database for all Agno-related collections, including shared user memories and agent-specific collections (sessions, metrics, eval, knowledge).

Decision¶

Use agno_db as the single database for all Agno-related collections, instead of using auth_db or agent-specific databases.

Rationale¶

Separation of Concerns: Keep auth_db focused on authentication and user management
Agno Isolation: All Agno-related data in one dedicated database
Shared Memories: Enable memory sharing across all agents using agno_db.user_memories
Agent Isolation: Maintain agent-specific collections within the same database
Simplified Management: Single database for all Agno operations

Architecture Design¶

Database Structure¶

auth_db/                          # Authentication & User Management (UNCHANGED)
├── users (collection)            # User information (existing)
└── ... (other auth collections)  # Other auth-related collections

agno_db/                          # Agno Database (NEW)
├── user_memories (collection)   # SHARED user memories for all agents/apps
├── {agent_name}_sessions        # Agent-specific sessions
├── {agent_name}_metrics         # Agent-specific metrics
├── {agent_name}_eval            # Agent-specific eval
└── {agent_name}_kb              # Agent-specific knowledge

Key Principle¶

auth_db: Unchanged - handles authentication and user management only
agno_db: All Agno-related data, including:
Shared user memories (user_memories collection)
Agent-specific collections (sessions, metrics, eval, knowledge)
All collections use agent-specific names for isolation

Database Collections¶

agno_db Database¶

agno_db/
├── user_memories                 # SHARED - All agents/apps
│   ├── user_id: "user123"
│   ├── user_id: "user456"
│   └── ...
│
├── support_agent_sessions        # support_agent specific
├── support_agent_metrics        # support_agent specific
├── support_agent_eval           # support_agent specific
├── support_agent_kb             # support_agent specific
│
├── sales_agent_sessions          # sales_agent specific
├── sales_agent_metrics          # sales_agent specific
├── sales_agent_eval             # sales_agent specific
└── sales_agent_kb               # sales_agent specific

auth_db Database (Unchanged)¶

auth_db/
├── users                         # User information (existing)
└── ... (other auth collections)  # Other auth-related collections

Implementation¶

How It Works¶

Agent Creation: All agents use agno_db as their main database
Shared Memories: All agents share agno_db.user_memories collection
Agent Isolation: Each agent has its own collections (sessions, metrics, eval, knowledge)
Collection Naming: Agent-specific collections use {agent_name}_{collection_type} pattern

Code Example¶

from bast_ai_agno.bp.agno_agents import AgnoAgentFactory

factory = AgnoAgentFactory(catname="testcat")

# Create agent with shared memory storage (default behavior)
agent1 = factory.create_openai_chat_agent(
    name="support_agent",
    # db=None means agent gets agno_db for all collections
    # MemoryManager uses agno_db.user_memories (shared)
    config_raw={
        "enable_user_memories": True
    }
)

agent2 = factory.create_openai_chat_agent(
    name="sales_agent",
    # Same setup - agent gets agno_db, memories in agno_db.user_memories
    config_raw={
        "enable_user_memories": True
    }
)

# Agent 1 stores memory in agno_db.user_memories
agent1.print_response("My name is John", user_id="user123")

# Agent 2 retrieves memory from agno_db.user_memories
agent2.print_response("What's my name?", user_id="user123")
# Agent 2 knows: "John" (from shared memory)

Configuration¶

Default Configuration¶

Database Name: agno_db
Memory Collection: user_memories (shared)
Session Collection: {agent_name}_sessions (agent-specific)
Metrics Collection: {agent_name}_metrics (agent-specific)
Eval Collection: {agent_name}_eval (agent-specific)
Knowledge Collection: {agent_name}_kb (agent-specific)

Custom Configuration¶

factory = AgnoAgentFactory(catname="testcat")

# Use default agno_db
agent = factory.create_openai_chat_agent(
    name="my_agent",
    # use_shared_memory_storage=True (default)
    # shared_memory_db_name='agno_db' (default)
    config_raw={
        "enable_user_memories": True
    }
)

# Custom agno database name
agent = factory.create_openai_chat_agent(
    name="my_agent",
    use_shared_memory_storage=True,
    shared_memory_db_name='custom_agno_db',  # Custom database name
    config_raw={
        "enable_user_memories": True
    }
)

PROs¶

1. Clear Separation of Concerns¶

✅ auth_db remains focused on authentication and user management
✅ agno_db handles all Agno-related operations
✅ Clear boundaries between authentication and AI agent functionality

2. Simplified Management¶

✅ Single database for all Agno operations
✅ Easier to backup, restore, and manage Agno data
✅ Centralized location for all agent-related data

3. Shared Memories¶

✅ All agents share memories via agno_db.user_memories
✅ Memory sharing works seamlessly across agents and applications
✅ Consistent memory access for all Agno agents

4. Agent Isolation¶

✅ Each agent has its own collections (sessions, metrics, eval, knowledge)
✅ Agent-specific collections use {agent_name}_{collection_type} pattern
✅ Clear isolation while maintaining shared memories

5. Scalability¶

✅ Easy to add new agents without database changes
✅ All agents use the same database structure
✅ Consistent pattern across all agents

6. Maintainability¶

✅ Single database to monitor and maintain
✅ Easier to debug and troubleshoot Agno-related issues
✅ Clear separation from authentication concerns

7. Compatibility with Agno Architecture¶

✅ Works with Agno's StorageMongoDB single-database design
✅ No custom storage wrappers needed
✅ Uses standard Agno patterns

CONs¶

1. All Collections in One Database¶

⚠️ All Agno collections (memories, sessions, metrics, eval, knowledge) are in agno_db
⚠️ Cannot separate memories into a different database from agent-specific data
⚠️ This is a limitation of Agno's StorageMongoDB architecture

2. Database Size Growth¶

⚠️ agno_db will grow as more agents and sessions are added
⚠️ Need to monitor database size and performance
⚠️ May require sharding or partitioning for very large deployments

3. Single Point of Failure¶

⚠️ All Agno operations depend on agno_db
⚠️ Database downtime affects all agents
⚠️ Need robust backup and disaster recovery strategy

4. No Physical Separation¶

⚠️ Cannot physically separate shared memories from agent-specific data
⚠️ All collections are in the same MongoDB database
⚠️ Isolation is logical (collection names) rather than physical (separate databases)

5. Migration Considerations¶

⚠️ Need to migrate existing data if moving from auth_db to agno_db
⚠️ May require downtime during migration
⚠️ Need to update all existing agents to use agno_db

Reasons for This Decision¶

1. Agno Architecture Limitation¶

Agno's StorageMongoDB uses a single database for all collections. This is a fundamental limitation of Agno's storage architecture. We cannot: - Use different databases for memories vs. sessions/metrics/etc. - Physically separate shared memories from agent-specific data - Use different databases per collection within a single storage instance

Solution: Accept this limitation and use agno_db for all Agno collections, with logical isolation via collection names.

2. Separation from Authentication¶

auth_db should remain focused on authentication and user management
Adding Agno collections to auth_db would mix concerns
agno_db provides clear separation for AI agent functionality

3. Shared Memories Requirement¶

Need to share user memories across all agents and applications
Using agno_db.user_memories enables this sharing
All agents can access the same memory collection

4. Agent Isolation Requirement¶

Each agent needs its own sessions, metrics, eval, and knowledge collections
Agent-specific collection names ({agent_name}_{collection_type}) provide isolation
Logical isolation is sufficient for most use cases

5. Operational Simplicity¶

Single database for all Agno operations simplifies management
Easier to backup, restore, monitor, and maintain
Centralized location for all Agno-related data

6. Future-Proofing¶

Easy to add new agents without database changes
Consistent pattern across all agents
Scalable architecture for growth

Implementation Details¶

Changes Required¶

Update AgnoMongoDBResources:
Change default database from auth_db to agno_db
Update get_shared_user_memory_storage() to use agno_db
Update get_agent_storage() to use agno_db when shared memory is enabled
Update AgnoAgentFactory:
Change default shared_memory_db_name from 'auth_db' to 'agno_db'
Update documentation and comments
Update Configuration:
Update all default configurations to use agno_db
Update documentation and examples
Migration (if needed):
Migrate existing data from auth_db to agno_db (if applicable)
Update all existing agents to use agno_db

Database Creation¶

# agno_db will be created automatically by MongoDB when first used
# No explicit creation needed - MongoDB creates databases on first write

Collection Naming Pattern¶

Shared Collections: user_memories (no prefix)
Agent-Specific Collections: {agent_name}_{collection_type}
{agent_name}_sessions
{agent_name}_metrics
{agent_name}_eval
{agent_name}_kb

Usage Examples¶

Example 1: Basic Agent with Shared Memories¶

from bast_ai_agno.bp.agno_agents import AgnoAgentFactory

factory = AgnoAgentFactory(catname="testcat")

# Create agent with shared memory storage (uses agno_db by default)
agent = factory.create_openai_chat_agent(
    name="support_agent",
    config_raw={
        "enable_user_memories": True
    }
)

# Agent uses:
# - agno_db.user_memories (shared memories)
# - agno_db.support_agent_sessions (agent-specific)
# - agno_db.support_agent_metrics (agent-specific)
# - agno_db.support_agent_eval (agent-specific)
# - agno_db.support_agent_kb (agent-specific)

Example 2: Multiple Agents with Shared Memories¶

from bast_ai_agno.bp.agno_agents import AgnoAgentFactory

factory = AgnoAgentFactory(catname="testcat")

# Create multiple agents
support = factory.create_openai_chat_agent(
    name="support",
    config_raw={"enable_user_memories": True}
)

sales = factory.create_openai_chat_agent(
    name="sales",
    config_raw={"enable_user_memories": True}
)

# Support agent stores memory in agno_db.user_memories
support.print_response("I prefer email", user_id="user123")

# Sales agent retrieves memory from agno_db.user_memories
sales.print_response("How should I contact you?", user_id="user123")
# Sales knows: "email" (from shared memory in agno_db)

Example 3: Custom Database Name¶

from bast_ai_agno.bp.agno_agents import AgnoAgentFactory

factory = AgnoAgentFactory(catname="testcat")

# Use custom agno database name
agent = factory.create_openai_chat_agent(
    name="my_agent",
    use_shared_memory_storage=True,
    shared_memory_db_name='custom_agno_db',  # Custom database name
    config_raw={
        "enable_user_memories": True
    }
)

Migration Guide¶

From auth_db to agno_db¶

If you have existing data in auth_db that needs to be migrated to agno_db:

Backup existing data:

mongodump --db=auth_db --collection=user_memories

Create agno_db (will be created automatically on first write)

Restore data to agno_db:

mongorestore --db=agno_db --collection=user_memories dump/auth_db/user_memories.bson

Update configuration:
Update all agents to use agno_db instead of auth_db
Update default configurations
Verify migration:
Verify data integrity
Test agent operations
Monitor for any issues

Monitoring and Maintenance¶

Database Monitoring¶

Monitor agno_db size and growth
Monitor collection sizes and document counts
Monitor query performance and indexes

Backup Strategy¶

Regular backups of agno_db
Separate backup schedule from auth_db
Test restore procedures regularly

Performance Optimization¶

Create appropriate indexes on collections
Monitor slow queries
Optimize collection sizes and sharding if needed

Conclusion¶

Using agno_db as the single database for all Agno-related collections provides:

Clear separation from authentication concerns
Shared memories across all agents
Agent isolation via collection naming
Simplified management with a single database
Compatibility with Agno's architecture

While this approach has limitations (all collections in one database), it is the best solution given Agno's StorageMongoDB architecture and provides sufficient isolation via collection naming.

References¶

Date¶

2025-01-27