Can your brain “see” music? Composer, pianist, and software engineer Stephen Malinowski has been trying to answer that question for the past forty years. His “Music Animation Machine” (which began on paper in the form of a 20-foot scroll) now produces wildly popular YouTube videos that visualize symphonic scores.
One video in particular paints in vivid colors Igor Stravinsky’s The Rite of Spring, a ballet that still thwarts audience expectations with primal stabs of sound 100 years after it first debuted at the Théâtre des Champs-Élysées in Paris.
In an interview with NPR’s Anastasia Tsioulcas, Malinowski commented on the neuroscience behind his project:
People usually respond to sound in a unitary way. It’s the reason why you can’t follow more than one conversation at a time at a party, for example. But with vision, your brain is trained to comprehend multiple things at once: you can take in many more elements simultaneously. In music, there’s often much more going on than you can grasp in that moment of hearing. When you have a visualization, your eyes lead your ears through the music. You take advantage of your brain’s ability to process multiple pieces of visual information simultaneously.