AI Agent Taxonomy: From ReAct to HyperAgents

Defining "Agent" in the HyperAgents Framework

The HyperAgents paper, published by Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina at Meta Research in March 2026 (arXiv:2603.19461), adopts a deliberately broad definition: an agent is any computable program. This definition is intentionally inclusive because it avoids conflating "agent" with specific implementation patterns like LLM-based reasoning or tool use.

Under this definition, a simple Python script that reads a file and writes output is an agent. A neural network that classifies images is an agent. An LLM-based system with ReAct-style reasoning and tool use is an agent. And a self-referential self-modifying system that rewrites its own improvement mechanisms is also an agent. The taxonomy then organizes these agents by their structural properties and capabilities, building from simple to complex.

This broad definition serves an important theoretical purpose: it makes the taxonomy complete. Every AI system that processes input and produces output fits somewhere in the taxonomy, regardless of its implementation. The taxonomy's value lies not in the definition of "agent" but in the structural distinctions between levels.

Level 1: Traditional Agent

The traditional agent is the foundation of the taxonomy. It follows the classic perceive-decide-act cycle from AI textbooks. The agent perceives its environment through sensors or input data, decides on an action using a fixed decision procedure (rules, search algorithms, or learned policies), and acts on the environment to produce output or change state.

Traditional agents include rule-based systems, search algorithms, reinforcement learning agents with fixed policies, and classical planning systems. Their defining characteristic is that their behavior is determined by their initial programming and training. They do not modify their own decision procedures at runtime. A trained RL agent may have learned complex behavior during training, but at deployment time its policy is fixed.

Examples include chess engines using minimax search, thermostat controllers, and trained image classifiers. These systems are well-understood, predictable, and auditable. Their limitations are also well-known: they cannot adapt to situations outside their training distribution without human intervention.

Level 2: Tool-Using LLM Agent

The second level introduces the LLM as the decision-making core and gives it access to external tools. This is the architecture popularized by the ReAct framework (Yao et al., 2022) and Toolformer (Schick et al., 2023), and it forms the basis of most modern AI agent systems.

ReAct Architecture in Detail

ReAct Loop (Yao et al., 2022):

  User Goal
    │
    ▼
┌─────────────────────────────┐
│       Agent Controller       │
│  ┌───────────────────────┐  │
│  │    Planner / LLM      │  │  ◄── Generates Thought + Action
│  │  (reasoning trace)    │  │
│  └───────┬───────────────┘  │
│          │                   │
│          ▼                   │
│  ┌───────────────────────┐  │
│  │     Tool Call          │  │  ◄── Search, Code, API, etc.
│  └───────┬───────────────┘  │
│          │                   │
│          ▼                   │
│  ┌───────────────────────┐  │
│  │    Observation         │  │  ◄── Tool result fed back
│  └───────┬───────────────┘  │
│          │                   │
│          ▼                   │
│  ┌───────────────────────┐  │
│  │    Memory / Context    │  │  ◄── Accumulated history
│  └───────────────────────┘  │
└─────────────┬───────────────┘
              │
              ▼
         Final Output
    (repeat loop until done)

The ReAct loop works as follows. The LLM receives the user's goal along with its accumulated context (previous thoughts, actions, and observations). It generates a "thought," which is an explicit reasoning trace about what to do next. Based on this thought, it generates an "action," which is typically a tool call (web search, code execution, API request, file read, etc.). The tool returns an "observation," which is the result of the action. This observation is appended to the context, and the loop repeats until the LLM determines the task is complete.

The key innovation of ReAct over earlier tool-use approaches is the explicit reasoning trace. By generating thoughts alongside actions, the LLM can plan multi-step solutions, recover from errors, and explain its decision-making process. Toolformer (Schick et al., 2023) took a complementary approach, training LLMs to determine when and how to call tools by embedding tool-use tokens directly in the model's output.

Tool-using LLM agents are powerful and widely deployed. They can browse the web, write and execute code, manage files, interact with APIs, and compose responses from multiple information sources. However, their fundamental limitation is that the agent's reasoning strategy, prompt structure, and tool selection logic are all fixed at design time. The agent can use tools flexibly, but it cannot change how it reasons about using them.

Level 3: Multi-Agent System (MAS / LLM-MAS)

The third level extends from single agents to systems of multiple agents that communicate and collaborate. Multi-agent systems have a long history in AI research, but the combination of LLMs with multi-agent architectures (LLM-MAS) has produced particularly powerful results since 2023.

Multi-Agent Architecture

Multi-Agent System Architecture:

  ┌─────────────────────────────────────┐
  │        Manager / Orchestrator        │
  │  (task decomposition, delegation)    │
  └──────────┬──────────────────────────┘
             │
    ┌────────┼────────┐
    ▼        ▼        ▼
┌────────┐ ┌────────┐ ┌──────────┐
│Research│ │ Coding │ │Critic/QA │   ◄── Worker Agents
│ Agent  │ │ Agent  │ │  Agent   │       (specialized roles)
└───┬────┘ └───┬────┘ └────┬─────┘
    │          │           │
    └──────────┼───────────┘
               ▼
     ┌─────────────────┐
     │   Message Bus    │   ◄── Communication Protocol
     └────────┬────────┘
              ▼
     ┌─────────────────┐
     │Execution Sandbox │   ◄── Shared Environment
     └─────────────────┘

In a multi-agent system, different agents take on specialized roles. A manager or orchestrator agent decomposes tasks and delegates subtasks to worker agents. Worker agents might specialize in research (web search and information gathering), coding (code generation and debugging), critic or QA (reviewing outputs for quality), or other domain-specific roles. Agents communicate through a message bus or shared memory, following defined communication protocols.

Multi-agent systems provide several advantages over single agents: specialization allows each agent to focus on what it does best, parallel execution can speed up complex tasks, and the critic/QA pattern provides built-in quality control. Frameworks like AutoGen (Microsoft, MIT license) and CrewAI (MIT license) make it straightforward to build these systems.

The limitation of Level 3 systems is that the agent roles, communication protocols, and orchestration logic are all predefined by the system designer. A multi-agent system cannot decide that it needs a new type of worker agent, or that the current communication protocol is inefficient and should be redesigned. These structural decisions are fixed at design time.

Level 4: Meta-Agent

The fourth level introduces the concept of a meta-agent: an agent whose primary function is to generate, evaluate, or rewrite other agents' strategies. The meta-agent operates at a higher level of abstraction than task agents. Instead of solving tasks directly, it improves the systems that solve tasks.

The Darwin Godel Machine (DGM), introduced by Jenny Zhang et al. in 2025, is the canonical example of a Level 4 system. DGM uses a meta-mechanism to analyze the task agent's performance, generate improvement instructions, and modify the task agent's code. The task agent improves over successive generations, guided by the meta-agent's instructions.

The key property of Level 4 that distinguishes it from Level 3 is the modification of agent strategies. A multi-agent system orchestrates fixed agents. A meta-agent changes what those agents do. However, the meta-agent itself has a crucial limitation: its own modification procedure is fixed. In DGM, the meta-mechanism that generates improvement instructions is hand-crafted by human engineers and cannot be modified by the system. This limits the meta-agent's ability to improve its own improvement process.

Level 5: HyperAgent

The fifth and highest level in the taxonomy is the hyperagent, the system introduced in the HyperAgents paper. A hyperagent unifies the task agent and meta agent into a single editable program, with open-ended exploration over a population of variants.

DGM-H Architecture

HyperAgent (DGM-H) Architecture:

  ┌──────────────────────────────────────────┐
  │              Open-Ended Archive            │
  │  (population of diverse agent variants)    │
  └──────────┬───────────────────────────────┘
             │  Select Parent
             ▼
  ┌──────────────────────────────────────────┐
  │         Hyperagent Codebase               │
  │  ┌──────────────┐  ┌─────────────────┐   │
  │  │  Task Agent   │  │   Meta Agent    │   │
  │  │  (editable)   │  │   (editable)    │   │
  │  └──────────────┘  └─────────────────┘   │
  └──────────┬───────────────────────────────┘
             │  Metacognitive Self-Modify
             ▼
  ┌──────────────────────────────────────────┐
  │    Modified Hyperagent Codebase           │
  │  (both task AND meta code may change)     │
  └──────────┬───────────────────────────────┘
             │  Validate (sandbox execution)
             ▼
  ┌──────────────────────────────────────────┐
  │              Open-Ended Archive            │
  │  (variant added if novel or improving)     │
  └──────────────────────────────────────────┘

The DGM-H (Darwin Godel Machine with Hyperagents) architecture implements Level 5 through four key properties:

1. Unified codebase — task agent and meta agent exist in a single program, not as separate systems. This eliminates the boundary between "what the agent does" and "how it improves."
2. Full editability — the meta agent can modify any part of the codebase, including the code that implements the meta agent itself. This is the self-referential property.
3. Open-ended archive — a population of agent variants is maintained with diversity pressure, preventing convergence to a single strategy and enabling stepping-stone dynamics.
4. Sandbox validation — all modifications are tested in isolated environments before being accepted into the archive, providing a safety layer.

The Key Differentiator: Self-Referential Modification

The property that separates Level 5 from Level 4 is clear: whether the self-modification mechanism is itself modifiable. In a meta-agent (Level 4), the meta-mechanism is fixed. It can improve the task agent's code, but it cannot improve its own improvement process. In a hyperagent (Level 5), the meta-mechanism is part of the editable codebase. The system can improve how it improves.

This self-referential property is what enables cross-domain transfer. When the meta-mechanism is fixed, it encodes domain-specific improvement strategies. When it is editable, it can evolve domain-general strategies that transfer to unseen domains. The experimental evidence from the HyperAgents paper confirms this: the transferred DGM-H meta achieves imp@50 = 0.630 while the fixed DGM meta achieves imp@50 = 0.0 on the same unseen domain.

Taxonomy Comparison Table

Framework Landscape

Where do the popular AI agent frameworks fit in this taxonomy? The answer reveals a significant gap between what frameworks currently offer and what hyperagent-level systems require.

None of the mainstream frameworks (AutoGen, LangChain, LangGraph, CrewAI, LlamaIndex) natively provide capabilities above Level 3. They are designed for orchestration, tool use, and multi-agent collaboration within predefined structures. Building Level 4 or Level 5 capabilities on top of these frameworks would require substantial additional engineering: self-modification pipelines, code generation and validation infrastructure, sandbox execution environments, and archive management systems.

This is not a criticism of these frameworks. Levels 2 and 3 address the vast majority of current AI agent use cases. But it highlights the HyperAgents framework's unique position as the only open-source implementation that natively provides Level 5 hyperagent capabilities.

The Three-Loop Model

To understand how hyperagents differ operationally from other agent types, the HyperAgents paper describes a three-loop model of nested execution:

1. The Task Loop — the innermost loop, where the agent executes on a specific task instance. In a code patching task, this is one attempt to fix a specific bug. In paper review, this is one evaluation of one paper. All agents, from Level 1 through Level 5, have a task loop.
2. The Evaluation Loop — the middle loop, where the agent's performance is assessed across multiple task instances (validation examples). This loop produces the metrics (accuracy, pass rate, success rate) that quantify the agent's current competence. Levels 4 and 5 use this loop to drive improvement decisions.
3. The Self-Improvement Loop — the outermost loop, where the meta-agent analyzes evaluation results, generates improvement instructions, modifies the agent's code, and triggers re-evaluation. In Level 4 systems (DGM), this loop can modify the code in the task loop and evaluation loop but not its own code. In Level 5 systems (HyperAgents), this loop can modify the code in all three loops, including itself.

The self-referential property of hyperagents means that the self-improvement loop has no fixed boundary. It can change the criteria for evaluation, the strategy for generating modifications, the mechanism for selecting which modifications to try, and even the structure of the improvement loop itself. This is what makes the hyperagent category fundamentally open-ended: there is no predetermined ceiling on the sophistication of its improvement strategies.

Connection to AI Safety Taxonomy

The five-level taxonomy has direct implications for AI safety. Higher levels in the taxonomy correspond to higher autonomy and, consequently, higher potential risk.

The HyperAgents paper acknowledges these risks and proposes mitigations including sandbox execution, human oversight checkpoints, and transparent archives. However, the researchers also note that the fundamental challenge of Level 5 safety, ensuring safe behavior when the behavior space is essentially unbounded, remains an open research problem that the AI safety community must address.

The taxonomy itself contributes to safety by providing a clear classification framework. Regulators, auditors, and safety researchers can use the taxonomy to determine what level of oversight a given system requires. A Level 2 tool-using agent needs different safety measures than a Level 5 self-referential system, and having a shared vocabulary for these distinctions enables more precise safety discussions.

Frequently Asked Questions