Google’s Soundstorm

SoundStorm Overview

SoundStorm, developed by Google Research, is an innovative model for efficient, non-autoregressive audio generation. It uses the semantic tokens of AudioLM as input, and leverages bidirectional attention and confidence-based parallel decoding to generate the tokens of a neural audio codec. SoundStorm is capable of producing high-quality audio with greater consistency in voice and acoustic conditions, and it does so at a speed that is two orders of magnitude faster than the autoregressive generation approach of AudioLM.

SoundStorm Highlights

• SoundStorm can generate 30 seconds of audio in just 0.5 seconds on a TPU-v4, making it significantly faster than other models.
• It maintains high audio quality and consistency in voice and acoustic conditions, ensuring a superior user experience.
• SoundStorm can scale audio generation to longer sequences, demonstrated by its ability to synthesize high-quality, natural dialogue segments from annotated transcripts.