Mass-Spring Ball Music Visualization and DFT-based note and beat detection.

I made this project my first semester as a PhD student, along with working on the Assistive getup project with Professor Liu.

The primary goal of this project was to synthesize dynamic motion in a physically-simulated (mass-spring) ball based on musical content (beat and frequency content) in a "dance"-like manner. To facilitate this, I wrote frequency analysis tools for using both standard FFT and frequency-specific DFT transforms (which were much more accurate, all other things being equal, using the piano keys as center freqs), which I calculated myself using multiple threads for speed.

The mass-spring ball is excited by scaled energy content in tiled patterns over the ball's surface that form zones, with the zones being larger or smaller based on the frequency content. Detected "beats" (cyclic increases in acoustic energy) in various frequency bands were used to excite the ball.

Mass-spring ball visualization

Demonstration of FFT and DFT (switches halfway through) analysis, showing DFT benefits, particularly in the lower frequencies.

Piano roll functionality