Karsten van VoorenBeveren, BE
← Back to work

Beweeg Baai

An interactive browser installation where your hands play piano and violin, and every movement shapes a living wave on screen.

Role
Concept, design & code (solo)
Year
2025
Type
School project, Creative Coding (AP Hogeschool)
Tools
JavaScript, MediaPipe Hands, Tone.js, HTML Canvas, AI assistant for coding
Code on GitHub
Beweeg Baai on two screens: the start screen and the piano and violin view

Challenge

Make an interactive installation that turns body movement into sound and visuals. The assignment involved music students from the conservatory, which is where piano and violin come from.

Solution

The webcam tracks your hands. The left side of the screen plays piano, the right side plays violin, and the height of your hand sets the pitch. Your movement drives the colour, rhythm and amplitude of a wave.

Result

A working installation that runs in the browser.

Context

This is an experiment in hand tracking and sound rather than a product for a specific audience.

The project started as "AdemBaai", driven by breathing. Without a microphone good enough to pick up breathing reliably, I switched to hand tracking, which turned out to be more direct and more fun to play with.

Beweeg Baai start screen: "Welkom bij BeweegBaai" with a button to turn on the camera
Start screen

Process

I worked by experimenting directly in code: try, fail, adjust. It was my first project with hand tracking.

  1. 01

    Breathing idea

    The first concept: an installation driven by your breath.

  2. 02

    Microphone problem

    No microphone picked up breathing reliably enough to play with.

  3. 03

    Switch to hand tracking

    Hands turned out to be more direct, and more fun to play with.

  4. 04

    Tuning the instruments and the wave

    Adjusting samples, pitch and the wave until playing felt natural.

Final design

Watch the demo on Vimeo
Playing piano on the left side of the screen

01

Two instruments, one screen

Left is piano, right is violin, using real piano and violin samples.

A raised hand playing a higher note

02

Your hand is the pitch

Raise or lower your hand to play higher or lower notes.

The wave changing colour and amplitude with movement

03

A wave that responds

Colour, rhythm and amplitude change with your movement, so you see what you hear.

Style guide

Colours

  • Night

    #0B1020

    Background

  • Deep Navy

    #12183A

  • Text

    #E9EEF9

  • Muted

    #CFD8F6

  • Cyan

    #7BDFF2

    Accent, buttons

  • Mint

    #B2F7EF

    Highlight

Typography

TitlesPoppins 600
Beweeg Baai
TextPoppins 400
Raise your hand to play higher notes.

Components

Floating info button
Camera button
Wave preview (in the installation it changes hue with your movement)

Visual language

The wave itself changes hue with your movement.

Build

Stack

  • JavaScript
  • MediaPipe Hands
  • Tone.js
  • HTML Canvas
  • MediaPipe Hands for real-time hand tracking via the webcam.
  • Tone.js for sound, with my own piano and violin samples.
  • HTML Canvas for the wave visual.
  • Built with the help of an AI assistant; concept, interaction and tuning are my own.

Next project

KSK Beveren Companion App