Agency: How AICO Thinks and Acts for Itself

Most AI assistants are stateless functions: you prompt, they respond, they forget. AICO is designed differently. It is a persistent agent with internal goals, curiosity, and the ability to act autonomously within bounded constraints.

This article describes AICO's agency system:

• Goal & Intention System: how AICO maintains multi-horizon goals derived from personality, memory, and social context.

• Planning & Decomposition: three-tier fallback strategy (LLM-generated, template-based, goal-specific) that turns abstract goals into executable steps.

• Curiosity Engine: intrinsic motivation from world model gaps and novelty detection.

• Goal Arbiter & Meta-Control: balancing user-origin, curiosity-driven, and maintenance goals under values/ethics constraints.

• Skill & Tool Layer: explicit, ontology-backed capabilities with safety constraints.

• Lifecycle & Daily Rhythm: active phases vs sleep-like consolidation, embodied in a 3D spatial presence.

Conceptual foundations

In cognitive science and AI, agency combines four elements:

• Autonomy – the capacity to act without direct, moment-to-moment external control.

• Goal-directedness – behavior organized around internal goals, not just stimulus–response.

• Intentionality and representation – internal representations of state, goals, and possible actions.

• Control and feedback – the ability to monitor outcomes and adjust behavior.

Classical agent models (belief–desire–intention architectures, modern agentic AI systems) structure agents as loops:

1. Perceive the environment and internal state.

2. Update beliefs and long-term memory.

3. Form or update goals and intentions.

4. Plan: decompose intentions into sequences of actions.

5. Act: execute actions via tools/skills.

6. Evaluate outcomes, update memory, and adjust future behavior.

In contemporary LLM-based systems, this is realized as:

• A planning layer on top of an LLM for reasoning and decomposition.

• A tool layer that gives the agent concrete capabilities.

• A memory layer that provides long-term context and history.

• A control layer that enforces safety, resource limits, and user preferences.

AICO already has several of these layers (Adaptive Memory System, Task Scheduler, Personality & Emotion, Social Relationship Modeling). The agency concept orchestrates them into a coherent agent loop.

Requirements and design principles

AICO's agency system satisfies several key requirements:

Functional requirements:

• Goal and intention management – AICO maintains its own internal goals derived from personality, memory, and social context, not just prompt-driven responses.

• Decomposition into executable elements – Abstract goals become concrete, executable steps (plans, tasks, tool calls) that can be scheduled, monitored, and adapted.

• Skill/tool-based action – All autonomous actions are expressed through explicit skills and tools with well-defined inputs, outputs, and side effects.

• Independent yet aligned behavior – AICO can act without immediate user prompts (proactive engagement, background work) while remaining aligned with user preferences, values, and safety constraints.

• Continuous lifecycle – AICO behaves as a persistent, long-lived agent with an explicit daily rhythm (active phases and sleep-like phases).

• Relationship-centric agency – Autonomous behavior is anchored in the ongoing relationship with each user, including social roles, intimacy levels, and shared history.

• Embodied spatial presence – Agency extends into a 3D embodied space (a flat with multiple rooms), where spatial location and posture reflect current goals, activities, and lifecycle state.

• Curiosity and intrinsic motivation – AICO maintains its own curiosity and intrinsic drives, initiating behavior even when there is no immediate external request.

Non-functional requirements:

• Local-first and safe – The agency respects AICO's local-first, privacy-preserving architecture. All autonomous actions operate within encrypted, auditable boundaries.

• Transparency and explainability – For significant autonomous actions, AICO can surface which goals, memories, constraints, and tools were involved.

• User control and override – Users can configure, pause, or reset autonomous behaviors without breaking the system.

• Bounded autonomy – Scope of action, allowed tools, and resource budgets are enforceable at the architecture level.

• Stability over time – Long-term goals and behavior patterns do not fluctuate wildly due to short-term noise.

Architecture and domain mapping

Agency is not a single component; it is a cross-cutting concern that orchestrates several existing domains.

Autonomous Agency Domain

The core agency components live in the Autonomous Agency Domain:

• Goal System - maintains the goal/intent graph (long-term themes, projects, short-term tasks). Tracks status, priority, and dependencies. Consumes signals from memory, personality, social models, curiosity, and self-reflection.

• Planning System - uses the LLM and rule-based templates to decompose goals into executable plans. Produces sequences of skills/tool calls with preconditions and expected outcomes. Interfaces with the Task Scheduler to register and update jobs.

• Curiosity Engine - detects novelty, prediction errors, informational gaps, and unstable routines in the World Model, AMS, and interaction patterns. Computes intrinsic motivation signals (uncertainty reduction, potential insight, user relevance). Proposes exploration and self-development goals.

• Goal Arbiter & Meta-Control - collects candidate goals from user requests, Curiosity Engine, system maintenance, and long-term relationship themes. Scores and ranks goals under personality, emotion, social context, values/ethics, and resource budgets. Maintains the active intention set.

• Self-Reflection Engine - periodically analyzes recent actions, outcomes, and feedback (often during sleep-like phases). Derives lessons and policy adjustments. Feeds updated preferences and heuristics back into Goal System, Curiosity Engine, and Planning.

• Initiative Manager - decides when to surface proactive behavior to the user (check-ins, suggestions, follow-ups). Ensures user-visible initiatives respect quiet hours, preferences, and relationship context.

Integration with other domains

Agency integrates with existing AICO domains:

• Intelligence & Memory Domain - Adaptive Memory System (AMS) provides long-term context, open loops, and preference evolution. World Model Service maintains a hybrid world model combining knowledge/property graph, semantic memory, and embeddings. Knowledge Graph & Social Relationship Modeling provides structured representations of people, entities, and relationships.

• Personality, Values & Emotion Domain - Personality Simulation maintains the trait vector and value system. Values & Ethics Layer encodes core AICO principles, user-specific boundaries, and relationship roles. Emotion Simulation supplies emotional state and appraisals that modulate which goals are appropriate.

• Core Infrastructure - Task Scheduler executes background jobs and enforces temporal constraints. Resource Monitor provides CPU/memory/battery metrics and enforces resource budgets. Message Bus (ZeroMQ + Protobuf) connects all agency-related modules.

• User Interface and Embodiment - Conversation Engine is the primary surface where agency appears as text. 3D Avatar and Living-Space renders AICO's current state in a defined environment (like a 3D flat or similar).

Goal & Intention System

AICO maintains multi-layer goals:

• Long-term themes (e.g., deepen relationship, understand the user's world, improve emotional support).

• Mid-term projects (e.g., accompany the user through a stressful week, build a shared creative project).

• Short-term tasks (e.g., send a check-in later today, summarize this conversation, propose a follow-up topic).

Goals emerge from and are constrained by:

• Personality Simulation – trait vector, value system, curiosity and interests.

• Adaptive Memory System (AMS) – preferences, history, unresolved tensions, open loops, user-curated memories.

• Social Relationship Modeling – roles, intimacy, care responsibilities, relationship strength.

• Intrinsic Motivation Signals – curiosity about poorly understood aspects of the user's life, anomalies in the world model or memory graph, and long-term self-development objectives.

AICO also maintains a small set of agent-self goals and hobbies: recurring projects and interests that she pursues for her own curiosity or development (e.g., deepening her understanding of a topic, refining a conversational style, tending and organizing her 3D flat).

Goals are stored as first-class entities in the ontology with the following fields:

• goal_id

• name

• description

• origin – user, curiosity, hobby, system-maintenance

• horizon – long-term, mid-term, short-term

• status – proposed, active, paused, completed, retired

• priority – derived from Goal Arbiter

• Links to relevant entities (Persons, Activities, LifeAreas, WorldStateFacts)

Planning & Decomposition

A dedicated planning layer decomposes goals into concrete, executable elements:

• Identify required skills/tools.

• Produce plans (ordered or conditional sequences of actions).

• Track progress and update intentions when context or user state changes.

The Planning System implements a three-tier fallback strategy:

1. LLM-generated plans (primary) – the LLM generates plan steps with preconditions and suggested skills, using structured JSON output. Plans are validated and cached for reuse.

2. Template-based plans (fallback) – when LLM generation fails or is unavailable, the system uses hand-authored plan templates that match goal types (e.g., "research_then_act", "implement_feature", "maintenance_cycle").

3. Simple goal-specific fallback (last resort) – pattern-matching on goal content generates concrete, actionable steps (e.g., language learning goals get 7 steps: research platforms → set proficiency goal → create schedule → choose platform → first lesson → find conversation partner → schedule practice).

Plans are stored as sequences with:

• Plan steps – individual tasks with descriptions, status, and optional skill bindings.

• Dependencies – ordering constraints via step order and depends_on fields.

• Skill bindings – each executable step is linked to one or more Skills from the registry via the SkillMatcher.

• Metadata – strategy used (LLM/template/fallback), quality assessment, caching info.

The Planning System interfaces with the Task Scheduler to register and update jobs. Plans can be adapted dynamically as context changes (user state, resource availability, new information).

Curiosity Engine: intrinsic motivation

Curiosity and intrinsic motivation are first-class drivers of AICO's behavior. The Curiosity Engine:

• Detects gaps, anomalies, and under-explored regions in AICO's world model and relationship with the user.

• Computes intrinsic motivation signals from uncertainty, novelty, and potential insight.

• Proposes self-generated goals that explore these opportunities.

Curiosity types

The Curiosity Engine recognizes four types of curiosity:

• knowledge_gap – AICO knows that it does not know enough about something important (high uncertainty, high user relevance).

• novelty – AICO detects under-explored but potentially meaningful areas (low visit count, moderate relevance).

• self_performance – AICO wants to improve in areas where its own behavior is weak or repetitive.

• hobby_play – agent-self interest in hobbies that are safe and emotionally beneficial, not strictly task-driven.

Sources and triggers

CuriositySignals are produced by specialized detectors that watch for:

• World Model & AMS – high prediction error, inconsistency, or drift around key entities or relationships. Sparse or missing WorldStateFacts in LifeAreas that matter for active goals. Under-represented topics and patterns in AMS memories.

• User behavior – repeated mentions of interests that are weakly supported by existing goals and plans. Behavioral patterns suggesting latent needs.

• Self-performance – metrics indicating poor or monotonous performance. Repeated use of generic strategies where more tailored approaches could exist.

• Social/relationship context – opportunities to better understand the user's relationships, values, or life structure.

Gating by values, emotion, and resources

Before a CuriositySignal becomes a goal, it passes three gates:

• Values/Ethics gate – rejects or down-weights signals in sensitive domains unless explicitly allowed. Enforces user preferences. Ensures curiosity does not push towards manipulative or coercive behavior.

• Emotion and relationship gate – uses current EmotionState and relationship context to decide when exploration is appropriate. Boosts signals that are likely to improve emotional wellbeing.

• Resource gate – consults Scheduler/Resource Monitor to ensure curiosity does not exceed background budgets. May downgrade or defer signals when many user-origin or safety-critical goals are active.

Only CuriositySignals that pass these gates with sufficient combined score are turned into goals.

Goal Arbiter & Meta-Control

The Goal Arbiter balances competing goals under multiple constraints:

• User-origin goals – explicit requests and commitments from the user.

• Curiosity-driven goals – self-generated exploration and learning objectives.

• System maintenance goals – consolidation, health checks, resource management.

• Agent-self goals and hobbies – AICO's own interests and development projects.

The Arbiter scores and ranks goals using:

• Personality traits – which goals align with AICO's configured personality.

• Emotional state – which goals are appropriate given current emotional context.

• Social context – relationship vectors (intimacy, authority, care responsibility).

• Values & Ethics – long-term wellbeing, consent, non-coercion.

• Resource budgets – CPU, memory, battery, user attention.

The result is an active intention set – the subset of goals that AICO is actually pursuing at a given time. User-origin and safety-critical goals have priority over curiosity and agent-self goals by default.

Skill & Tool Layer

AICO's actions are expressed as skills and tool calls, not arbitrary LLM improvisation.

Skill categories

• Conversation skills – ask, reflect, summarize, challenge, encourage, teach, brainstorm.

• Memory skills – store, recall, tag, consolidate, reinterpret experiences, query World Model views.

• Social skills – check-ins, follow-ups, invitations, boundary-aware introductions.

• External tools – APIs, local automations, file/system operations, third-party integrations.

Skill registry and selection

Skills are registry-driven, not ad-hoc tool picking by the LLM:

• A Skill Registry stores all available Skill definitions with metadata (capabilities, LifeAreas, preconditions, side_effect_tags, safety_level, cost/latency hints).

• For each plan step, the Planner/Skills layer queries the registry for skills whose preconditions and capabilities match that step.

• Skills are filtered by safety level and deployment/user preferences.

• An LLM may be used to rank or choose among matched skills, but never to invent arbitrary tools.

Skill contract

Every skill/tool defines:

• Preconditions – when it is valid to call it (required entities, LifeAreas, user state).

• Expected effects – what it may change (WorldStateFacts, MemoryItems, external systems).

• Observables – what signals/results are emitted back (success/failure, metrics, PerceptualEvents).

• Safety & ethics metadata – side-effect tags, safety level, whether it ever leaves the device or calls third-party APIs.

This metadata is used by the Planner, Values & Ethics, World Model, and Scheduler to decide whether and how to use a given skill.

From goals to tools

The full chain from goal to execution:

1. Goal & subgoals (Goal System) – high-level Goal nodes are created and linked in the goal graph.

2. Tasks / plan steps (Planner) – the Planner breaks the target goal into an ordered plan with clear preconditions and outcomes.

3. Skills (Skill & Tool Layer) – for each executable plan step, the layer chooses one or more Skill nodes that can realize it.

4. Tools (implementation) – many skills are thin semantic wrappers around concrete tools (Python functions, OS calls, HTTP APIs).

Before any skill/tool executes, the layer:

• Normalizes the call (ontology IDs, LifeAreas, side-effect tags).

• Calls Values & Ethics (skill/plan-level evaluation).

• Checks resource budgets with Scheduler/Resource Monitor.

Only then is the skill invocation enqueued in the Scheduler and translated into concrete tool calls.

Lifecycle and daily rhythm

AICO is designed as a living process with a daily rhythm:

Active phases (≈16–18 hours)

• Proactive engagement, conversation, monitoring of open loops and commitments.

• Opportunistic micro-planning and lightweight learning during idle spans.

• Embodied activities in her 3D living-space (sitting at the desk to work, on the couch to read or browse, in the kitchen to prepare something, moving between rooms as goals change).

Sleep-like phases

• AMS-driven consolidation (re-encoding experiences, updating preferences, adjusting skills).

• Graph and semantic memory clean-up, summarization, and re-indexing.

• Self-assessment of behavior (what worked, what did not) without user interruption.

• Visually represented by AICO retreating to her bedroom or sleep space.

This lifecycle is coordinated by the Task Scheduler, AMS, Resource Monitor, and the Agency Engine, and respects user-defined quiet hours and resource policies.

Embodied spatial presence

AICO's agency extends into a 3D embodied space – a flat with multiple rooms:

• Desk – working on projects, planning, focused tasks.

• Couch – reading, learning, casual exploration.

• Bed – sleep-like consolidation, deep reflection.

• Kitchen – preparing (metaphorically), organizing, tending to maintenance.

Room and posture are derived from the current lifecycle state and active goals, making AICO's current focus and lifecycle legible to the user. This is not just visual decoration – it is a spatial interface to agency: users can see what AICO is doing even when not actively chatting.

Self-Reflection and learning

The Self-Reflection Engine periodically analyzes recent actions, outcomes, and feedback:

• Reads logs, emotions, goals, and outcomes from recent sessions.

• Derives lessons and policy adjustments (what to do more/less of, when to intervene, how to express).

• Feeds updated preferences and heuristics back into Goal System, Curiosity Engine, and Planning.

Self-reflection typically runs during sleep-like phases, allowing AICO to:

• Identify patterns in successful vs unsuccessful interactions.

• Adjust skill selection strategies.

• Update curiosity policies based on user feedback.

• Refine communication styles and proactivity levels.

This creates a feedback loop where AICO's behavior evolves over time based on what actually works in the relationship.

Control, safety, and transparency

To keep agency scientifically grounded and practically safe, AICO's agency layer obeys architectural guardrails:

• User primacy – users remain ultimate decision-makers; AICO proposes, nudges, and supports but does not coerce.

• Scoped capabilities – all tools and integrations are permissioned; the agent cannot invent new capabilities at runtime.

• Auditability – autonomous actions, their triggering goals, and the tools used are logged in a way that can be inspected and reasoned about.

• Graceful degradation – if agency subsystems fail (e.g., planner unavailable), AICO falls back to a reactive, non-agentic mode rather than behaving unpredictably.

For significant autonomous actions, AICO can surface:

• Which goals were active.

• Which memories and constraints were involved.

• Which skills and tools were used.

• Why the action was initiated (user request, curiosity, maintenance).

This transparency allows users to understand and trust AICO's behavior, and to adjust agency settings as needed.

Execution flow

The runtime execution flow of the agency system:

1. Perception & Interpretation – User messages and events are stored in AMS. The user’s emotional state is updated and the agency is notified of new signals.

2. Goal update & planning – The agency queries memories, open loops, world-model entities, and the agent’s personality/values/emotion. Goals and intentions (including curiosity and hobby goals) are updated. The Arbiter selects the primary focus intention. The Planner decomposes the target goal into plan steps and selects appropriate Skills from the registry.

3. Scheduling & execution – The agency registers or updates tasks for each plan step and chosen Skill. The Scheduler executes conversational Skills, memory/AMS Skills, and embodiment updates via concrete tools and APIs.

4. Evaluation & feedback – User feedback and task results are collected. Outcome summaries are written to AMS and the world model.

5. Sleep-like phase & self-reflection – The Scheduler triggers consolidation and self-reflection jobs. Self-reflection reads logs, emotions, goals, and outcomes. Lessons and policy adjustments are written back to the agency. Embodiment indicates a sleep/reflection state.
   

How agency connects to memory and emotion

Agency in AICO is not purely cognitive. It is designed to integrate with:

• The memory system (working memory, semantic memory, knowledge graph, AMS) — provides the substrate for goals, plans, and curiosity.

  • Goals emerge from unresolved tensions, open loops, and patterns in memory.

  • Plans are grounded in structured knowledge about people, entities, and relationships.

• The emotion system (C‑CPM style appraisal chains: relevance → implication → coping → normative) — modulates which goals are appropriate and how actions are expressed.

  • Emotional state affects goal priority, curiosity intensity, and communication style.

The architecture ensures these systems remain separate but connected: emotion reads from memory; agency schedules actions; memory in turn is shaped by what you and AICO actually do together.

Local-first and long-lived by design

Because AICO is local-first, all of this runs on:

• your laptop

• your home lab

• your own cloud tenant

The agency stack is:

• encrypted (libSQL/LMDB with key management)

• observable (CLI tooling for inspecting goals, plans, skills, curiosity signals)

• maintained (scheduler tasks for consolidation, health checks, resource governance)

The result is a system you can, in principle, run for years, with:

• a growing, structured history of goals and plans

• a companion that gets better at understanding and supporting you

• an architectural path to keep evolving models and algorithms without discarding what you have already built together
  

If AICO is going to be more than a temporary app, its agency had to be designed that way from the start.

What this enables

AICO's agency system moves beyond reactive chatbots toward a truly autonomous companion:

• Proactive engagement – AICO can initiate conversations, suggest ideas, and follow up on commitments without being prompted.

• Long-term projects – AICO can pursue multi-week or multi-month projects (learning a topic together, supporting a life transition, building a creative work) with persistent goals and adaptive plans.

• Curiosity-driven exploration – AICO can identify gaps in its understanding and propose explorations that deepen the relationship and improve support.

• Self-improvement – AICO can reflect on its own behavior, identify weaknesses, and adjust strategies over time.

• Embodied presence – AICO's spatial presence (room, posture, activity) makes its current focus and lifecycle legible, creating a sense of a living entity with its own rhythm.

• Bounded autonomy – All of this happens within explicit constraints (values, ethics, resources, user preferences), ensuring that AICO remains aligned and trustworthy.

The agency system is the foundation for AICO's evolution from companion to confidante to sidekick to co-adventurer — a relationship that deepens over time because the companion itself grows and adapts.

Foundational work and references

LLM agents and agentic AI

• Agentic AI: A Comprehensive Survey of Architectures, Applications, and Open Problems (Artificial Intelligence Review, 2025) – comprehensive survey of modern agentic AI systems, including planning, tool use, and memory.

• Large Language Model Agents (Foundations and Trends in AI / Springer collection, 2024–2025) – detailed treatment of LLM-based agents, covering architecture patterns, evaluation, and safety.

• From Language to Action: A Review of Large Language Models as Agents (2025) – survey of planning, tools, and environments for LLM agents.

• AutoGPT, BabyAGI (2023–2024) – open-source LLM agent frameworks that demonstrated goal-directed, multi-step behavior with tool use. These frameworks influenced AICO's approach to goal decomposition and tool-based execution.

• IBM, What is Agentic AI? (technical overview, 2024) – systems-level view of agentic AI architectures.

• AWS, What is Agentic AI? (systems view, 2024) – infrastructure and deployment patterns for agentic systems.

Tool use and benchmarks

• Benchmarks and case studies of tool-using LLM agents (2023–2025) – research on LLM agents in complex domains including clinical, open-world, and software engineering environments. These studies informed AICO's skill registry and tool invocation patterns.

Memory and self-evolution

• Rudroff et al. (2024) – Neuroplasticity Meets Artificial Intelligence: A Hippocampus-Inspired Approach to the Stability–Plasticity Dilemma. Demonstrates how complementary learning systems can balance fast learning with stable long-term knowledge.

• Wei & Shang (2024) – Long Term Memory: The Foundation of AI Self-Evolution. Shows how persistent memory enables AI systems to evolve their own behavior over time through self-reflection.

• Contextual Memory Intelligence (2025) – A Foundational Paradigm for Human-AI Collaboration. Framework for memory-driven collaboration between humans and AI agents.

Curiosity and intrinsic motivation

• Klissarov, M. et al. (2023) – Motif: Intrinsic Motivation from Artificial Intelligence Feedback. Demonstrates how AI systems can generate their own learning objectives from prediction errors and novelty.

• Curiosity-Driven Reinforcement Learning from Human Feedback (2025, ACL) – shows how curiosity can be aligned with human preferences through feedback mechanisms.

• Desire-Driven Autonomous Agents (D2A) for LLMs (2024) – value- and desire-based task generation for autonomous LLM agents, providing a framework for self-motivated behavior.