CamPlay is five very different games — an endless runner, a party game, a dodge-’em-up, a freeze-tag racer and a downhill skier — driven by one webcam input engine, running alongside real-time 3D rendering and live multiplayer, all in a browser tab. This article is the architecture tour: the boundaries that keep it maintainable and the budgets that keep it fast. It is written for people building their own motion-controlled things; steal freely.

Two stacks, one bridge

The codebase enforces a single load-bearing rule: input and game never touch. The pose side (camera, model, signal extraction) knows nothing about games; the game side (physics, scoring, rendering) knows nothing about cameras. They meet at exactly one interface — a per-frame snapshot function each game polls:

flowchart TD
  subgraph INPUT["Input stack — knows nothing about games"]
    CAM["Camera"] --> ML["PoseLandmarker<br/>WASM · GPU · ~22 Hz"] --> ENG["Pose engine<br/>landmarks → signals"]
  end
  ENG -- "poll(): one snapshot per tick" --> GAME
  KB["Keyboard fallback"] -. "merges on top" .-> GAME
  subgraph GAMES["Game stack — never sees the camera"]
    GAME["Game loop<br/>physics · scoring · calories"] --> R3["three.js renderer<br/>60 fps budget"]
  end
  GAME -- "CustomEvents (camplay:*)" --> PLAT["Platform (React)<br/>auth · rooms · leaderboards"]
Two stacks, one synchronous bridge — and a second event bridge out to the React platform.
// the entire input⇄game contract
const snap = controls.poll(now);
// {
//   usingPose: bool     — pose or keyboard fallback?
//   tracking: bool      — is the body confidently visible?
//   speed: 0..1         — continuous jog intensity
//   jump: bool          — one-shot edge, true for one poll
//   ducking: bool       — held state
//   lean: -1..1         — smoothed analog
//   lane: 0|1|2         — which third of the frame
// }

A synchronous snapshot beats an event stream for game input: the game samples state exactly once per simulation tick, so there is no event-queue skew between what the player did and what this frame simulates. One-shot actions (jump) are edge flags inside the snapshot; held actions (duck, lean) are levels. A staleness guard rides along — if the pose stream stalls for 350 ms, the snapshot reports tracking lost rather than replaying the last known speed forever, which is the difference between “the game paused politely” and “the character sprinted into a wall while the model hiccuped”.

The payoff for the strict boundary: adding a game to the platform costs a registry entry, a folder of game code and one mount registration — the input engine, tutorial scaffolding, camera plumbing and frame coach come for free. It also means the keyboard fallback (arrow keys when no camera is available) merges on top of the same snapshot, so every game supports both input modes without a line of per-game code. And a nice side effect covered in the on-device tracking article: the only code that touches the camera has no network pathway at all.

Budgeting the main thread: ML and a 3D renderer share one laptop

The performance problem in a motion game is that the two expensive things compete: pose inference wants the GPU and main thread, and so does the three.js renderer chasing 60 fps. CamPlay’s budget looks like this:

  • Inference runs at ~22 Hz, not per-frame. Detection is gated to one run per 45 ms — movement signals are smoothed over multiple frames anyway, so 60 Hz inference would burn battery for zero gameplay difference.
  • The model is the lite variant of the pose landmarker, on the GPU delegate, at 640×480 camera resolution. Bigger models and frames measurably improve landmark stability and measurably ruin frame rate; this corner of the trade-off space is where the games stay playable on ordinary laptops.
  • The render side keeps its own discipline: flat-shaded materials, roughly 300–400 draw calls, device-pixel-ratio capped at 1.5. A stylized look that holds 60 fps beats a realistic one that stutters — in a motion game, dropped frames are dropped inputs.
  • On content pages, ads load lazily; on game routes there are no ads and no ad scripts at all, partly as a product decision and partly because a third-party script competing with inference for the main thread is exactly the jank you cannot afford.

Multiplayer: sync the seed, not the world

An eight-player race sounds like it needs a server simulating the world. It doesn’t — it needs everyone to experience the same world, which is a much cheaper problem. When a room starts, all players receive a shared random seed and a synchronized start time. Every machine then generates the identical course — same obstacles, same gates, same trees — deterministically from that seed, and simulates only its own player locally.

flowchart TD
  SEED["Shared seed + synced start time"] --> A["Player A machine<br/>generates the identical course"]
  SEED --> B["Player B machine<br/>generates the identical course"]
  SEED --> C["… up to 8 players"]
  A <-- "progress, a few times/s" --> RT[("Realtime<br/>Database")]
  B <-- "progress, a few times/s" --> RT
  C <-- "progress, a few times/s" --> RT
Sync the seed, not the world: every machine builds the same course; only tiny progress updates cross the network.

What actually crosses the network is small: each player publishes a progress update a few times per second (position, alive/dead), and everyone renders everyone else as ghosts from those updates. Ghosts are presentational — your collisions are yours alone — so latency cannot kill you; it can only make a ghost look slightly stale. In Red Light, Green Light, even the lights are just a deterministic function of the seed and the shared wall clock, so every player’s red light lands at the same instant with no light-switch packet ever sent.

Two lobby flavors sit on this base. Race rooms (Rune Run, Red Light, Ski Slalom) rank by first-to-goal, then distance. Party rooms (Simon Says, Dino Dodge) run elimination: each client judges itself against the seeded command or obstacle stream and self-reports, the host arbitrates results — and if the host disconnects, the host seat migrates automatically to the longest-tenured player, so a game never dies with its creator. Fairness in both flavors leans on the same referee-visibility rule the input engine provides: a player untrackable for more than 1.2 seconds is out.

The game⇄platform bridge: custom events

There is a second boundary, less obvious than input/game: the games are mostly framework-free JavaScript (a canvas, a loop, module-level state), while the platform around them — auth, rooms, leaderboards — is React. They communicate through namespaced browser CustomEvents: the game dispatches “run finished” with the result payload; the platform listens and persists it. Room setup, rosters, countdowns and standings flow the other way as events into the game. Neither side imports the other, which keeps game code portable and lets the platform wrap five differently-built games with one set of components.

What we would tell someone building one

  • Put a hard wall between perception and simulation on day one. Every motion-game codebase we have seen rot, rotted at that seam.
  • Design detection for the room you expect players to have, not an idealized studio — CamPlay’s upper-body-only constraint (playable one metre from the screen) shaped its entire signal design, detailed in the detection article.
  • Feel is a budget line, not a polish task: cap inference rate, cap render cost, and treat any third-party script on a game route as a frame-rate bug.
  • For multiplayer, determinism-from-a-seed converts a hard distributed-systems problem into an easy broadcast problem. Sync inputs to the world, not the world itself.