Last summer (May 2017) Magenta, the creative wing of Google’s TensorFlow machine learning project, released a new neural-network-based model, NSynth (Neural Audio Synthesis), capable of building brand new sounds from input audio samples. While mixing instruments sounds has been around for a long while, NSynth is able essentially build an entirely new instrument sound from the ground-up based on the input instrument sounds. So rather than the new instrument being a simply adding two audio waves together as past techniques have done, NSynth actually learns and predicts an entirely new, original audio wave generator that marries elements input instruments. This allows for extensive control of the timbre and dynamics that can’t be accomplished by naively adding instrument sounds together. The result is a truly expressive and innovative way for artists to create new sounds. NSynth Super is the Magenta’s first step towards putting their cutting-edge algorithms into the hands of actual artists. With the help of the Google Creative Lab, Magenta developed a hardware interface that allows music creators to directly interact and play with the NSynth model. While the underlying algorithm is rather complicated, the hardware interface is quite elegantly intuitive and simple. The current product iteration allows the user to generate a new instrument sound from up to four different input sounds, affording an impressive but not overwhelming amount of expressivity – a thoughtful choice. The demo hardware enclosure itself is rather minimal – a simple aluminum enclosure sporting cutouts for a MIDI inputs and a few physical buttons for selecting and refining sound samples. All user input occurs on the top surface of the device which is dominated by a large OLED touchscreen. The user is able to interact with the complex NSynth algorithm by simply dragging their fingers across a screen to modulate the sound. By abstracting away the complexities of the underlying algorithm, the NSynth Super becomes a much more accessible device for musicians to integrate into their current production setup.  Doubling down on their belief in accessibility, the Creative Lab open-sourced their prototype, including all necessary software, firmware, schematics and design templates such that users can completely fabricate a functional NSynth Super with the open-source information. While a novel, somewhat niche endeavor, the NSynth Super marks an important first step in applying modern machine learning techniques to music creation in an accessible, functional form.
