position
The center of movement becomes stereo position and register. Moving through the frame changes where the sound lives.
signal / live study
WORLDINSTRUMENT
What if looking were already a form of playing?
Point the camera at a room, a hand, a street, a plant, or light moving across a wall. The browser reduces each moment to visual relationships, then turns those relationships into pitch, harmony, filtering, spatial position, and resonance. The camera is not the artwork. The world becomes the control surface.
move the camera or move through the frame · space toggles the instrument when this field is focused
camera permission has not been requested
motion00%
structure00%
brightness00%
space00%
pitch—
pancenter
Camera access is requested only after you choose to play. Frames are sampled into a tiny in-memory canvas, measured locally, and discarded. WORLD INSTRUMENT does not record video, create image files, upload frames, or send camera data to a Warp the Surface service. Starting the instrument pauses the persistent site music so the two playback systems do not compete.
under the surface
The image disappears into relationships you can hear.
structure
Horizontal and vertical edge energy divide the fundamental from its overtones, making architecture behave differently from soft fields.
light
Average brightness opens and closes the filter. Light literally changes how much high-frequency material can pass.
space
Variation across the frame controls resonance and wetness, turning flat visual fields and spatially busy scenes into different acoustic rooms.
system
Vision supplies control. Synthesis supplies the voice.
- input
- camera / explicit browser permission
- vision
- low-resolution luminance · frame difference · edge orientation · spatial variance
- synthesis
- Web Audio oscillators · filters · stereo panning · convolution
- mapping
- motion → energy · structure → harmony · light → filter · space → resonance
- recording
- none
- server inference
- none
- status
- experimental / mobile-first v0
This first build deliberately uses browser-native vision rather than adding a model by default. It tests whether the physical world is already rich enough to become a playable interface at camera-frame latency. A local depth or recognition model should be added only when it creates a perceptibly different instrument, not merely a more complicated implementation.