how to play

gesture.live is an instrument you play with your hands in front of a webcam. There is nothing to install and no account to make: you open a tab, allow the camera, and your hands are the controls.

Your left hand plays chords and is always live. Your right hand picks a mode, which decides whether it is playing drums, bass, melody, effects, or reshaping the chord sound. You do not need to read music, and everything the chord hand can reach is already in key.

what you need

a camera: the webcam in a laptop is plenty. A phone works too, held sideways, and so does a Chromebook.
a browser: Chrome is the one to use. Recent Edge, Safari and Firefox work as well.
sound: speakers or headphones. Headphones if the room has a microphone in it.
room to move: far enough back from the camera that both hands and your elbows fit in the picture.

Nothing is installed and nothing is uploaded. The hand tracking runs in WebAssembly on your own machine and there is no server for a frame to be sent to, so nothing leaves your device. If you record a take, it is made in the browser and handed straight to you.

left hand · the chords

The chord hand is always live. It does not need a mode and it never has to be summoned: count a number on your fingers and that chord of the key sounds.

one finger: the first chord of the key.
two fingers: the second.
three fingers: the third.
four fingers: the fourth.
open hand, thumb out: the fifth.
devil horns: the sixth.
horns and thumb: the seventh.
turn the hand over: the same count flips to the parallel minor, so every pose has a second chord behind it.
fingers pointing down: the chord drops a semitone flat.
drop the pose: the chord releases, and the hand is re-armed. Making the same pose again sounds it again.

counting the chord

left hand · height and roll

Where you hold the pose matters as much as which pose it is. Height is spice: the higher the hand, the more notes are stacked on the chord. It changes while the chord is still sounding, so lifting your hand thickens a chord you are already holding, and lowering it thins it back out.

hold low: a plain triad. Nothing added.
lift: the same chord grows a seventh.
higher: a ninth.
top of the band: a thirteenth, wide open.
roll the wrist: sweeps the pad filter and the note length. It works while a chord is sounding, so a chord you are already holding keeps changing colour.

height is spice

right hand · the mode arc

The right hand does one of five jobs at a time, and you choose which with the arc.

fist, back of hand: make a fist with the back of your hand to the camera. The arc appears.
move: the nearest cell of the arc highlights.
open the hand: that mode is now live.
palm back instead: cancels. Nothing changes, and the mode you were in stays.

summoning the mode arc

the five modes

drums: one pinch finger per family of the kit: kick, snare, hats and toms. Pinch a finger and roll the wrist to say how present that family is, from silence through ghost notes to fills. Release and it keeps playing. Roll back to the floor to switch it off.
bass: four performed aspects, one per pinch finger: density, chord tones, octave jumps and feel. Height is the filter resonance, and reach is how hard the kick ducks the line.
melody: your height picks the note and a pinch sounds it for as long as you hold it. Drag the pinch and the line follows your hand. Each of the four fingers is its own lead voice.
fx / dj: the same four pinches hold effects instead of notes. Every effect is momentary, so it lasts exactly as long as the pinch, and the wrist roll rides whichever one you are holding.
sculpt: reshapes the chord hand instrument rather than adding a part: arp rate, envelope, glide, tremolo, grit and how wide the voicing spreads. Everything latches, so a sound you build survives a trip to the drums and back.

a pinch per part of the kit

the wrist roll

Rolling a wrist sweeps one control, and which control depends on what that hand is doing. On the chord hand it is the pad filter and the note length. In drums it is how present the part you are pinching is, or the cutoff when you are pinching nothing. In melody it is the lead filter. In fx it is the stutter rate and the crush depth. Both wrists keep their value printed on the plate, so a setting survives the hand leaving the frame.

one wrist, one sweep

recording a take

A take records in the browser, with the picture and the music together. You can record the wide frame, a 720x1280 vertical sheet made for a phone, or both at once. The vertical one is not a crop: it is drawn again for the taller frame, with the camera full width and its own panels. When you stop, the file is handed to you. It is never sent anywhere.

if something is not working

camera blocked: once a browser has blocked the camera it stops asking, so pressing start again cannot work. Use the camera icon in the address bar to allow it, then press start again.
opened from an app: the in-app browsers inside Instagram, TikTok and similar cannot ask for the camera at all. Open gesture.live in your normal browser instead.
no sound: browsers refuse to start audio until you have interacted with the page. Press start, and check the tab is not muted.
hands not picked up: get more light on your hands, and put the light in front of you rather than behind you. A bright window at your back turns your hands into silhouettes.
only one hand works: both hands need to be inside the frame. Step back until you can see your elbows in the picture.