AI Agents

Overview

Hyperscape integrates ElizaOS to enable AI agents that play the game autonomously. Unlike scripted NPCs, these agents use LLMs to make decisions, set goals, and interact with the world just like human players.

ElizaOS 1.7.0: Hyperscape is compatible with ElizaOS 1.7.0+ with backwards-compatible shims. Supports OpenAI, Anthropic, OpenRouter, and Ollama (local) LLM providers.

Starting AI Agents

# Simplified command (recommended)
bun run dev:ai

# Legacy command (still supported)
bun run dev:elizaos

This starts:

Game server on port 5555
Client on port 3333
ElizaOS runtime on port 4001
ElizaOS dashboard on port 4000

Access the agent dashboard at http://localhost:4000 to monitor and control agents. The dev:ai command uses concurrently to run both services with colored output for easier debugging.

LLM Autonomy System

Agents use a THINKING+ACTION format that separates reasoning from action selection, making behavior transparent and debuggable.

THINKING+ACTION Format

Agents respond with structured reasoning:

THINKING: I have 3 goblins attacking me and my health is at 40%. I have food in my inventory.
I should eat to restore health before continuing combat.

ACTION: EAT_FOOD

This enables:

Transparent decision-making (see why the agent chose an action)
Better debugging (identify flawed reasoning)
Foundation for multi-step planning

Autonomous Features

Target Locking - Agents lock onto combat targets for 30 seconds, preventing target switching mid-fight. Force Flee - If health drops below 25% with threats nearby, agents automatically flee (bypasses LLM). Combat Readiness - Agents assess if they’re prepared for combat (weapon, food, health) before engaging. Goal Templates - Structured beginner flows guide agents through OSRS-style progression. Guardrails - Hard and soft constraints prevent dangerous decisions.

Agent Capabilities

Available Actions

AI agents have 23 actions across 9 categories:

// From packages/plugin-hyperscape/src/index.ts
actions: [
  // Goal-Oriented (2 actions)
  setGoalAction,
  navigateToAction,

  // Autonomous Behavior (5 actions)
  exploreAction,
  fleeAction,
  idleAction,
  approachEntityAction,
  attackEntityAction,
  lootStarterChestAction,

  // Movement (3 actions)
  moveToAction,
  followEntityAction,
  stopMovementAction,
  exploreAction,
  approachEntityAction,
  navigateToAction,

  // Combat (3 actions)
  attackEntityAction,
  changeCombatStyleAction,
  fleeAction,

  // Skills (5 actions)
  chopTreeAction,
  mineRockAction,
  catchFishAction,
  lightFireAction,
  cookFoodAction,

  // Inventory (4 actions)
  equipItemAction,
  useItemAction,
  dropItemAction,
  pickupItemAction,

  // Social (1 action)
  chatMessageAction,

  // Banking (2 actions)
  bankDepositAction,
  bankWithdrawAction,

  // Goals (2 actions)
  setGoalAction,
  idleAction,
],

World State Access (Providers)

Agents query world state via 10 providers:

// From packages/plugin-hyperscape/src/index.ts
providers: [
  goalProvider,             // Current goal and progress
  gameStateProvider,        // Health, stamina, position, combat status
  inventoryProvider,        // Items, coins, free slots
  nearbyEntitiesProvider,   // Players, NPCs, resources nearby
  skillsProvider,           // Skill levels and XP
  equipmentProvider,        // Equipped items
  availableActionsProvider, // Context-aware available actions
  possibilitiesProvider,    // What actions are currently possible (NEW)
  goalTemplatesProvider,    // Structured goal templates (NEW)
  guardrailsProvider,       // Safety constraints and warnings (NEW)
],

New Providers (PR #628):

possibilitiesProvider - Analyzes inventory and nearby entities to determine what’s craftable, gatherable, and trainable
goalTemplatesProvider - Provides scored goal templates for OSRS beginner flows (woodcutting, mining, combat, etc.)
guardrailsProvider - Hard/soft constraints and active warnings to prevent dangerous decisions

Agent Architecture

Plugin Components

The plugin-hyperscape package contains:

Component	Purpose
Actions	Combat, skills, movement, inventory
Providers	World state, entity info, stats
Evaluators	Goal progress, threat assessment
Handlers	Event responses

Agent Dashboard

The Agent Dashboard provides real-time monitoring and control of AI agents:

Dashboard Features

Summary Card:

Online status and uptime
Combat level and total level
Current goal with progress bar
Session statistics

Goal Panel:

Current goal with progress tracking
Time estimates and XP rates
Stop/Resume goal controls
Recent goals history
Manual goal selection

Skills Panel:

All 12 skills with levels and XP
Session XP gains tracking
Progress bars for each skill
Live updates when viewport active

Activity Panel:

Recent actions feed
Session stats (kills, deaths, gold, items)
Event icons and timestamps

Position Panel:

Current zone name
Coordinates (X, Y, Z)
Nearby points of interest
Distance to landmarks

Quick Action Menu:

One-click commands (woodcutting, mining, fishing, combat)
Pick up nearby items
Go to bank
Stop/Idle controls

Stop/Resume Goal Control

The dashboard includes robust goal control: Stop Button:

Immediately halts current goal
Cancels movement path
Sets agent to paused state
Blocks automatic goal selection

Paused State:

Shows “Goals Paused” indicator
Resume Auto button to resume autonomous behavior
Set Goal button for manual goal selection
Chat commands still work (agent returns to idle after)

Resume Auto:

Resumes autonomous goal selection
Agent picks new goals based on context

The pause state persists across reconnections and is synchronized between the dashboard, server, and agent plugin.

Spectator Mode

Watch AI agents play in real-time:

Start with bun run dev:elizaos
Open localhost:3333
Select an agent to spectate
Observe decision-making in action

Configuration

The plugin validates configuration using Zod:

// From packages/plugin-hyperscape/src/index.ts (lines 62-83)
const configSchema = z.object({
  HYPERSCAPE_SERVER_URL: z
    .string()
    .url()
    .optional()
    .default("ws://localhost:5555/ws")
    .describe("WebSocket URL for Hyperscape server"),
  HYPERSCAPE_AUTO_RECONNECT: z
    .string()
    .optional()
    .default("true")
    .transform((val) => val !== "false")
    .describe("Automatically reconnect on disconnect"),
  HYPERSCAPE_AUTH_TOKEN: z
    .string()
    .optional()
    .describe("Privy auth token for authenticated connections"),
  HYPERSCAPE_PRIVY_USER_ID: z
    .string()
    .optional()
    .describe("Privy user ID for authenticated connections"),
});

Environment Variables

# LLM Provider (at least one required)
OPENAI_API_KEY=your-openai-key
ANTHROPIC_API_KEY=your-anthropic-key
OPENROUTER_API_KEY=your-openrouter-key
OLLAMA_SERVER_URL=http://localhost:11434  # For local Ollama models

# Hyperscape Connection
HYPERSCAPE_SERVER_URL=ws://localhost:5555/ws
HYPERSCAPE_AUTO_RECONNECT=true
HYPERSCAPE_AUTH_TOKEN=optional-privy-token
HYPERSCAPE_PRIVY_USER_ID=optional-privy-user-id

ElizaOS 1.7.0: Added Ollama plugin support for local LLM inference. Configure OLLAMA_SERVER_URL to use local models.

Agent Actions Reference

Combat Actions

attackMob(mobId): Engage enemy
setAttackStyle(style): Choose XP focus
flee(): Disengage and run

Skill Actions

chopTree(treeId): Woodcutting
catchFish(spotId): Fishing
lightFire(logId): Firemaking
cookFood(fishId, fireId): Cooking

Movement Actions

moveTo(x, y, z): Navigate to coordinates
moveToEntity(entityId): Follow entity
moveToArea(areaName): Travel to zone

Inventory Actions

equipItem(itemId): Wear equipment
dropItem(itemId): Drop from inventory (supports “drop all”)
pickupItem(itemId): Pick up items from ground
useItem(itemId): Consume or use

Goal Management

setGoal(description): Set autonomous goal
navigateTo(location): Navigate to named location
idle(): Wait and observe

Goal Pause State: The dashboard Stop button pauses autonomous goal selection. When paused, SET_GOAL is blocked and the agent stays idle until resumed or given a manual command.

Get Started

Guides

Core Concepts

Overview

Starting AI Agents

LLM Autonomy System

THINKING+ACTION Format

Autonomous Features

Agent Capabilities

Available Actions

World State Access (Providers)

Agent Architecture

Plugin Components

Agent Dashboard

Dashboard Features

Stop/Resume Goal Control

Spectator Mode

Configuration

Environment Variables

Agent Actions Reference

Combat Actions

Skill Actions

Movement Actions

Inventory Actions

Goal Management

Get Started

Guides

Core Concepts

​Overview

​Starting AI Agents

​LLM Autonomy System

​THINKING+ACTION Format

​Autonomous Features

​Agent Capabilities

​Available Actions

​World State Access (Providers)

​Agent Architecture

​Plugin Components

​Agent Dashboard

​Dashboard Features

​Stop/Resume Goal Control

​Spectator Mode

​Configuration

​Environment Variables

​Agent Actions Reference

​Combat Actions

​Skill Actions

​Movement Actions

​Inventory Actions

​Goal Management

Overview

Starting AI Agents

LLM Autonomy System

THINKING+ACTION Format

Autonomous Features

Agent Capabilities

Available Actions

World State Access (Providers)

Agent Architecture

Plugin Components

Agent Dashboard

Dashboard Features

Stop/Resume Goal Control

Spectator Mode

Configuration

Environment Variables

Agent Actions Reference

Combat Actions

Skill Actions

Movement Actions

Inventory Actions

Goal Management