An instrument you play in the air, built entirely in hardware.
A camera watches your hands. An FPGA finds them, turns their position into notes, and synthesizes the sound in real time, with no processor anywhere in the signal path. I own the synthesis engine and the audio output.
See the gesture
A camera pipeline locates the player's hands every frame and reduces them to a few control values. A teammate leads this stage.
Make the sound
A polyphonic additive synthesizer: several voices, each built from harmonically related partials with their own envelopes, for a piano-like timbre. One shared sine table and one multiplier are time-multiplexed across every partial of every voice within each audio sample, and a saturating mixer sums them without wrap-around.
Get it out
Audio leaves the board through an analog output that I'm building, and as a digital network stream that a teammate leads.
The interesting constraint is time: at audio rates the FPGA has well over a thousand clock cycles per sample, so a design that looks like dozens of parallel oscillators can be one small datapath visited in turn. Getting the schedule, the fixed-point precision and the envelope arithmetic right is the project. Results and a demo will be posted here when it is done.