Vansh Gadhia

← Back

Maestro

Feb 2025 · Python, TypeScript, MediaPipe, OpenCV, WebSockets, Qwen2.5-VL, Suno API, iOS

Maestro turns a broom into a playable guitar. A camera tracks your hands, magenta tape on the pole marks out the fretboard, and a strum registers when your strumming hand crosses the neck with enough velocity — so pitch and rhythm come from how you actually move. An iPhone companion detects pressed "strings", letting you form chords.

The premise is that everyone has already done this. Desks, bottles, brooms — people reach for whatever is nearby to make rhythm. Maestro makes that real, with no instrument to buy and no training required.

It also coaches. When you stop playing, two models run in parallel: Qwen2.5-VL reads posture and technique from video while NVIDIA Music Flamingo reads timing and style from audio. Their output is merged into short spoken feedback. In generation mode, what you played becomes the seed for a fully produced track.

The whole system runs in real time — a Python WebSocket server streaming webcam frames to a browser front end, with the models on an ASUS GX10. The hard constraint was latency: for a strum to feel musical it has to be instant, which mattered far more than model size.


More projects