Featured project

Gestalt Gallery

Gesture-based gallery for exploring and restoring masterpieces.

An interactive web experience where you browse paintings and rebuild them as puzzles — using nothing but your hand.

New product

Product Design · Interaction Design · AI-assisted build

View live project

Desktop + webcam

Gallery wall — hero still

How it works

Natural interactions, in real time.

  • An open hand sweeping sideways to move along the gallery wall

    Sweep to browse

    Sweep your open hand right to left to move along the wall.

  • A hand holding a thumbs-up to select the painting in view

    Hold a thumbs-up to select

    Hold it half a second to open the painting in front of you.

  • Thumb and forefinger pinching to lift a puzzle piece

    Pinch to restore

    Pinch to lift a piece and settle it back into the painting.

1

camera

1

tracker

4

gestures

0

backend

Real-time hand tracking, entirely on-device. A fourth gesture — fingers held up — picks the difficulty.

Live demo

See it in action

Gestalt Gallery running in real time. No mouse, no touchscreen — just your hand.

Onboarding

The tutorial only moves when you do

Each step unlocks by actually performing the gesture.

Onboarding step — prompted vs. unlocked

My approach

  1. 01

    Designed the system, not the screens

    Built the design system from the shipped code, and caught that the project docs described an architecture that no longer existed.

  2. 02

    Directed the agent with structured prompts

    Every iteration carried an explicit do-not-touch block protecting the fragile core: hand tracking, pinch hysteresis, snap logic.

    Task: add a difficulty picker to the puzzle overlay.Match the existing overlay spacing and type scale.DO NOT TOUCH:  - hand tracking loop or landmark smoothing  - pinch hysteresis thresholds (0.4 engage / 0.6 release)  - piece snap logic and its toleranceShip one change at a time. Stop if a threshold needs to move.
    Prompt excerpt
  3. 03

    Tuned the interaction by feel

    Pinch engages at 0.4 and releases at 0.6; a thumbs-up confirms after 500ms; a sweep waits 600ms before it will fire again. Every number was adjusted until the gesture felt calm and unmistakable.

  4. 04

    Made the onboarding teach itself

    Three gestures, taught one at a time, each gated by real detection.

Technologies

  • MediaPipe

    Real-time hand tracking, on-device

  • Claude Code

    Agent-directed implementation from structured prompts

  • Next.js

    App Router, no backend

  • Tailwind

    Token-based styling

“The hardest part was making the interactions feel natural and joyful — not just technically recognizable.”

Every threshold was tuned against my own hand, hundreds of times. Detection that scored well but felt wrong went back to the numbers.

Gestalt Gallery is an experiment in natural interfaces and playful exploration.

Try it yourself

What I'd do differently

  • Calibrate thresholds on recorded hands, not only my own.
  • Test a low-light webcam before tuning anything.
  • Write the do-not-touch block before the first prompt.