Client Application

The Hyperscape client is a React-based web application with Three.js WebGPU rendering. It provides a modern MMORPG experience with VRM avatars, responsive UI panels, and real-time multiplayer.

Client code lives in packages/client/src/. The rendering systems are in packages/shared/src/systems/client/.

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                       React Application                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ Auth Flow   │  │  Screens    │  │    Game Panels          │  │
│  │ PrivyAuth   │  │ - Login     │  │ - Inventory  - Skills   │  │
│  │ Wallet Auth │  │ - CharSel   │  │ - Equipment  - Bank     │  │
│  │             │  │ - GameClient│  │ - Combat     - Chat     │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                    Three.js WebGPU Renderer                     │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │ Scene Graph │  │  Cameras    │  │    Post-Processing      │  │
│  │ Terrain     │  │ Third Person│  │ - Bloom                 │  │
│  │ Entities    │  │ First Person│  │ - Tone Mapping          │  │
│  │ VRM Avatars │  │ RTS Mode    │  │ - Color Grading         │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                      Client World                               │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
│  │  Network    │  │    Input    │  │     Systems             │  │
│  │  WebSocket  │  │  Keyboard   │  │ - Graphics  - Camera    │  │
│  │  Reconnect  │  │  Mouse      │  │ - Audio     - Loader    │  │
│  │             │  │  Touch      │  │ - Health    - XP Drops  │  │
│  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Entry Point

The client entry point is packages/client/src/index.tsx:

// Main entry point for the Hyperscape browser client
import { World, installThreeJSExtensions } from "@hyperscape/shared";
import React from "react";
import ReactDOM from "react-dom/client";

// Screens
import { LoginScreen } from "./screens/LoginScreen";
import { CharacterSelectScreen } from "./screens/CharacterSelectScreen";
import { GameClient } from "./screens/GameClient";

// Authentication (Privy + Wallet)
import { PrivyAuthProvider } from "./auth/PrivyAuthProvider";

// Embedded mode support for spectator views
import { EmbeddedGameClient } from "./components/EmbeddedGameClient";
import { isEmbeddedMode } from "./types/embeddedConfig";

Authentication Flow

Authentication uses Privy for wallet-based login:

Login Screen → Privy Modal → Wallet Signature → Character Select → Game

// auth/PrivyAuthProvider.tsx
export function PrivyAuthProvider({ children }: { children: React.ReactNode }) {
  return (
    <PrivyProvider
      appId={PRIVY_APP_ID}
      config={{
        loginMethods: ["wallet", "email", "google"],
        appearance: { theme: "dark" },
      }}
    >
      {children}
    </PrivyProvider>
  );
}

Screens

Wallet connection via Privy
Social login (email, Google)
Session persistence

Character Select Screen

List existing characters
Create new character
Character preview with VRM avatar

Game Client

Main game loop
World initialization
UI overlay rendering

// screens/GameClient.tsx
export function GameClient({ characterId }: { characterId: string }) {
  const worldRef = useRef<World | null>(null);
  
  useEffect(() => {
    const world = new World({
      isServer: false,
      systems: clientSystems,
    });
    
    world.connect({ characterId });
    worldRef.current = world;
    
    return () => world.destroy();
  }, [characterId]);
  
  return <CoreUI world={worldRef.current} />;
}

UI Architecture

The client uses a clear separation between reusable UI primitives and game-specific code:

UI Design System (`src/ui/`)

Pure, reusable UI components and systems:

// From packages/client/src/ui/
components/               // Base UI primitives
  Window.tsx             // Draggable, resizable windows with anchor-based positioning
  TabBar.tsx             // Tab navigation with drag-to-combine
  DragOverlay.tsx        // Drag preview overlay
  Portal.tsx             // React portal for overlays
  MenuButton.tsx         // Menu buttons
  ItemSlot.tsx           // Item slot component
  // ... and more

core/                    // Core UI systems
  drag/                  // Drag-and-drop (@dnd-kit integration)
    DragContext.tsx      // Global drag state
    useDrag.ts           // Drag hook
    useDrop.ts           // Drop hook with race condition fixes
  edit/                  // Edit mode (L key to unlock)
  notifications/         // Notification system
  presets/               // Layout presets and cloud sync
  responsive/            // Breakpoint and mobile detection
  tabs/                  // Tab management and combining
  window/                // Window management with anchors

stores/                  # UI state management (Zustand)
  windowStore.ts         # Window positions with anchor-based positioning
  dragStore.ts           # Drag-and-drop state
  editStore.ts           # Edit mode state
  anchorUtils.ts         # Anchor positioning utilities
  // ... and more

Anchor-Based Positioning: Windows use viewport anchors (Unity/Unreal-style) for responsive scaling:

// From src/ui/stores/anchorUtils.ts
export type WindowAnchor =
  | "top-left" | "top-center" | "top-right"
  | "left-center" | "center" | "right-center"
  | "bottom-left" | "bottom-center" | "bottom-right";

// Key functions
getAnchorPosition(anchor, viewport)      // Get anchor coordinates
calculateOffsetFromAnchor(position, anchor, viewport)  // Calculate offset
repositionWindowForViewport(window, oldViewport, newViewport)  // Resize handler

Game-Specific Code (`src/game/`)

All game logic and components:

// From packages/client/src/game/
interface/               # Window management
  InterfaceManager.tsx   # Main UI orchestrator with drag-drop
  DefaultLayoutFactory.ts # Default window layouts with anchors
  useViewportResize.ts   # Responsive window positioning
  
panels/                  # Game panels
  InventoryPanel.tsx     # 28-slot inventory with drag & drop
  EquipmentPanel.tsx     # Equipment slots with stat display
  SkillsPanel.tsx        # All skills with XP progress bars
  CombatPanel.tsx        # Combat style selection with draggable styles
  BankPanel/             # Modular bank system
  TradePanel/            # Two-screen trade confirmation
  ActionBarPanel/        # Action bar with drag-drop support
  // ... and more

hud/                     # HUD elements
  StatusBars.tsx         # Health, prayer, run energy bars
  XPProgressOrb.tsx      # XP tracking orb with skill icons
  ActionProgressBar.tsx  # Skilling action progress
  Minimap.tsx            # Top-down map view
  MinimapCompass.tsx     # Compass overlay
  MinimapStaminaBar.tsx  # Stamina display
  level-up/              # Level-up notifications
  xp-orb/                # XP orb components

components/              # Game components
  chat/                  # Chat system (ChatBox, ChatInput, ChatMessage, ChatTabs)
  currency/              # Currency display and exchange
  dialog/                # Dialogue system
  equipment/             # Equipment components
  map/                   # World map
  quest/                 # Quest system
  settings/              # Settings panels
  skilltree/             # Skill tree

Core UI (CoreUI.tsx)

The main UI wrapper that renders HUD elements and game panels:

export function CoreUI({ world }: { world: ClientWorld }) {
  const [ready, setReady] = useState(false);
  const [deathScreen, setDeathScreen] = useState(null);
  
  // Event handlers for loading, death, disconnect
  useEffect(() => {
    world.on(EventType.READY, () => setReady(true));
    world.on(EventType.PLAYER_DIED, handleDeath);
    world.on(EventType.DISCONNECTED, handleDisconnect);
  }, [world]);
  
  if (!ready) return <LoadingScreen />;
  
  return (
    <>
      <InterfaceManager world={world} />
      <StatusBars world={world} />
      <ActionProgressBar world={world} />
      <XPProgressOrb world={world} />
      {deathScreen && <DeathScreen {...deathScreen} />}
    </>
  );
}

3D Graphics System

Renderer (WebGPU with WebGL Fallback)

The graphics system uses Three.js with WebGPU for high-performance rendering, with automatic WebGL fallback for environments that don’t support WebGPU (e.g., WKWebView in Tauri, older browsers):

// systems/client/ClientGraphics.ts
export class ClientGraphics extends SystemBase {
  private renderer: UniversalRenderer;
  private postProcessing: PostProcessingComposer;
  isWebGPU: boolean = true;
  
  async init() {
    // Create renderer (WebGPU preferred, WebGL fallback)
    this.renderer = await createRenderer({
      powerPreference: "high-performance",
      antialias: true,
    });
    
    this.isWebGPU = isWebGPURenderer(this.renderer);
    
    // Log backend capabilities
    if (this.isWebGPU) {
      logWebGPUInfo(this.renderer);
      const caps = getWebGPUCapabilities(this.renderer);
      console.log("[ClientGraphics] WebGPU features:", caps.features.length);
    } else {
      console.warn("[ClientGraphics] WebGPU unavailable (falling back to WebGL renderer)");
    }
    
    // Configure shadows
    // WebGPU: Cascaded Shadow Maps (CSM)
    // WebGL: Single directional light shadow
    configureShadowMaps(this.renderer, {
      cascades: 3,
      shadowMapSize: 2048,
    });
    
    // Post-processing (TSL-based, WebGPU only)
    // WebGL fallback disables post-processing
    this.usePostprocessing = (this.world.prefs?.postprocessing ?? true) && this.isWebGPU;
    
    if (this.usePostprocessing && isWebGPURenderer(this.renderer)) {
      this.postProcessing = createPostProcessing(this.renderer, {
        bloom: true,
        toneMapping: true,
        colorGrading: true,
      });
    }
  }
}

Renderer Backend Detection

// utils/rendering/RendererFactory.ts
export type RendererBackend = "webgpu" | "webgl";
export type UniversalRenderer = InstanceType<typeof THREE.WebGPURenderer>;

// Check if WebGPU is available
export async function isWebGPUAvailable(): Promise<boolean> {
  if (typeof navigator === "undefined") return false;
  
  const gpuApi = (navigator as NavigatorWithGpu).gpu;
  if (!gpuApi) return false;
  
  try {
    const adapter = await gpuApi.requestAdapter();
    return adapter !== null;
  } catch {
    return false;
  }
}

// Check if WebGL is available
export function isWebGLAvailable(): boolean {
  if (typeof document === "undefined") return false;
  
  const canvas = document.createElement("canvas");
  const gl = canvas.getContext("webgl2") || canvas.getContext("webgl");
  return gl !== null;
}

// Detect rendering capabilities
export async function detectRenderingCapabilities(): Promise<RenderingCapabilities> {
  const supportsWebGPU = await isWebGPUAvailable();
  if (supportsWebGPU) {
    return { supportsWebGPU: true, supportsWebGL: true, backend: "webgpu" };
  }
  
  const supportsWebGL = isWebGLAvailable();
  if (supportsWebGL) {
    return { supportsWebGPU: false, supportsWebGL: true, backend: "webgl" };
  }
  
  throw new Error(
    "Neither WebGPU nor WebGL is supported in this environment. " +
    "Please use a modern browser or a WebView with GPU acceleration enabled."
  );
}

WebGL Fallback Features

When running on WebGL backend:

No TSL Post-Processing: Bloom, tone mapping, and color grading are disabled
Simplified Shadows: Single directional light instead of Cascaded Shadow Maps (CSM)
Auto Exposure: Still works (tone mapping exposure is renderer-agnostic)
Settings Panel: Displays “WebGL” instead of “WebGPU”

// Environment.ts - WebGL shadow fallback
if (!useWebGPU) {
  this.csmShadowNode = null;
  this.csmNeedsAttach = false;
  this.needsFrustumUpdate = false;
  
  scene.add(this.sunLight);
  scene.add(this.sunLight.target);
  
  console.log(
    `[Environment] WebGL shadow map enabled (no CSM): mapSize=${csmConfig.shadowMapSize}, frustum=${baseFrustumSize * 2}`
  );
  return;
}

The WebGL fallback uses THREE.WebGPURenderer with forceWebGL: true instead of switching to THREE.WebGLRenderer. This keeps the codebase unified while supporting both backends.

Rendering Pipeline

Pre-render: Update matrices, frustum culling
Shadow Pass: Render cascaded shadow maps
Main Pass: Render scene with deferred lighting
Post-Processing: Bloom, tone mapping, effects (TSL-based)
UI Overlay: Render 2D React UI on top

Model Loading & Transform Baking

The ModelCache system handles GLTF model loading with transform baking to prevent rendering issues:

// From ModelCache.ts
private bakeTransformsToGeometry(scene: THREE.Object3D): void {
  // Ensure all matrices are up to date
  scene.updateMatrixWorld(true);

  // Apply transforms to each mesh's geometry
  scene.traverse((child) => {
    if (child instanceof THREE.Mesh && child.geometry) {
      // Clone geometry to avoid modifying shared geometry
      child.geometry = child.geometry.clone();

      // Apply world matrix to geometry (Three.js built-in method)
      child.geometry.applyMatrix4(child.matrixWorld);

      // Reset transform to identity
      child.position.set(0, 0, 0);
      child.rotation.set(0, 0, 0);
      child.scale.set(1, 1, 1);
      child.updateMatrix();
    }
  });
}

Why Transform Baking? GLTF files can have transforms stored in various ways:

Position/rotation/scale properties
Baked into matrices
Non-decomposable transforms (shear)

Baking all transforms into vertex positions guarantees correct rendering regardless of how the GLTF was exported from Blender or other 3D tools. Quaternion Normalization: Entity rotations use quaternions with all four components (x, y, z, w):

// From Entity.ts
quaternion: config.rotation
  ? [
      config.rotation.x,
      config.rotation.y,
      config.rotation.z,
      config.rotation.w,  // Uses actual w value, not hardcoded 1
    ]
  : undefined,

This prevents “squished” or incorrectly rotated models that can occur when quaternion components are not properly normalized.

Camera System

Supports multiple camera modes:

// systems/client/ClientCameraSystem.ts
export class ClientCameraSystem extends SystemBase {
  private settings = {
    minDistance: 2.0,        // Min zoom
    maxDistance: 15.0,       // Max zoom
    minPolarAngle: Math.PI * 0.35,  // Pitch limits
    maxPolarAngle: Math.PI * 0.48,
    rotateSpeed: 0.9,        // RS3-like feel
    zoomSpeed: 1.2,
    shoulderOffsetMax: 0.15, // Over-the-shoulder offset
  };
}

Mode	Controls
Third Person	Right-drag to rotate, scroll to zoom, click-to-move
First Person	Pointer lock, WASD movement
Top-down/RTS	Pan, zoom, click-to-move

VRM Avatar System

Characters use VRM format avatars with humanoid bone mapping:

// components/CharacterPreview.tsx
import { VRM, VRMLoaderPlugin, VRMUtils } from "@pixiv/three-vrm";
import { retargetAnimationToVRM } from "../utils/vrmAnimationRetarget";

export const CharacterPreview: React.FC<{ vrmUrl: string }> = ({ vrmUrl }) => {
  const vrmRef = useRef<VRM | null>(null);
  const mixerRef = useRef<THREE.AnimationMixer | null>(null);
  
  useEffect(() => {
    const loader = new GLTFLoader();
    loader.register((parser) => new VRMLoaderPlugin(parser));
    
    loader.load(vrmUrl, async (gltf) => {
      const vrm = gltf.userData.vrm as VRM;
      VRMUtils.rotateVRM0(vrm);  // Fix rotation for VRM 0.x
      
      // Retarget animations to VRM bones
      const mixer = new THREE.AnimationMixer(vrm.scene);
      const idleClip = await loadAnimation("idle.glb");
      const retargeted = retargetAnimationToVRM(idleClip, vrm);
      mixer.clipAction(retargeted).play();
      
      vrmRef.current = vrm;
      mixerRef.current = mixer;
    });
  }, [vrmUrl]);
};

VRM Bone Mapping

type VRMHumanBoneName =
  | "hips" | "spine" | "chest" | "upperChest" | "neck" | "head"
  | "leftShoulder" | "leftUpperArm" | "leftLowerArm" | "leftHand"
  | "rightShoulder" | "rightUpperArm" | "rightLowerArm" | "rightHand"
  | "leftUpperLeg" | "leftLowerLeg" | "leftFoot" | "leftToes"
  | "rightUpperLeg" | "rightLowerLeg" | "rightFoot" | "rightToes";

Client Systems

Located in packages/shared/src/systems/client/:

System	Description
`ClientGraphics.ts`	WebGPU rendering, shadows, post-processing
`ClientCameraSystem.ts`	Camera controls and collision
`ClientNetwork.ts`	WebSocket connection, reconnection
`ClientInput.ts`	Keyboard, mouse, touch handling
`ClientAudio.ts`	3D positional audio, music
`ClientLoader.ts`	Asset loading with progress
`HealthBars.ts`	Floating health bars over entities
`Nametags.ts`	Entity name labels
`DamageSplatSystem.ts`	Floating damage numbers
`XPDropSystem.ts`	XP gain notifications
`EquipmentVisualSystem.ts`	Equipment rendering on avatars
`TileInterpolator.ts`	Smooth tile-based movement

Embedded Mode

For stream overlays and spectator views:

// URL params for embedded mode
?embedded=true
&mode=spectator|free
&agentId=AGENT_ID
&followEntity=ENTITY_ID
&quality=low|medium|high
&hiddenUI=inventory,skills,chat

// components/EmbeddedGameClient.tsx
export function EmbeddedGameClient() {
  const config = window.__HYPERSCAPE_CONFIG__;
  
  return (
    <World
      mode={config.mode}
      followEntity={config.followEntity}
      hiddenUI={config.hiddenUI}
    />
  );
}

Overview

Engine Architecture

Client Application

Game Systems

AI Agents

Data Manifests

Technical Reference

Client Application

Client Application

Architecture Overview

Entry Point

Authentication Flow

Screens

Character Select Screen

Game Client

UI Architecture

UI Design System (`src/ui/`)

Game-Specific Code (`src/game/`)

Core UI (CoreUI.tsx)

3D Graphics System

Renderer (WebGPU with WebGL Fallback)

Renderer Backend Detection

WebGL Fallback Features

Rendering Pipeline

Model Loading & Transform Baking

Camera System

VRM Avatar System

VRM Bone Mapping

Client Systems

Embedded Mode

Overview

Engine Architecture

Client Application

Game Systems

AI Agents

Data Manifests

Technical Reference

​Client Application

​Architecture Overview

​Entry Point

​Authentication Flow

​Screens

​Login Screen

​Character Select Screen

​Game Client

​UI Architecture

​UI Design System (src/ui/)

​Game-Specific Code (src/game/)

​Core UI (CoreUI.tsx)

​3D Graphics System

​Renderer (WebGPU with WebGL Fallback)

​Renderer Backend Detection

​WebGL Fallback Features

​Rendering Pipeline

​Model Loading & Transform Baking

​Camera System

​VRM Avatar System

​VRM Bone Mapping

​Client Systems

​Embedded Mode

​Related Documentation

Client Application

Architecture Overview

Entry Point

Authentication Flow

Screens

Login Screen

Character Select Screen

Game Client

UI Architecture

UI Design System (`src/ui/`)

Game-Specific Code (`src/game/`)

Core UI (CoreUI.tsx)

3D Graphics System

Renderer (WebGPU with WebGL Fallback)

Renderer Backend Detection

WebGL Fallback Features

Rendering Pipeline

Model Loading & Transform Baking

Camera System

VRM Avatar System

VRM Bone Mapping

Client Systems

Embedded Mode

Related Documentation