Fugatto Nvidia

Fugatto is a foundational generative audio transformer model developed by NVIDIA Research. It is a versatile audio synthesis and transformation model capable of following free-form text instructions with optional audio inputs. The model introduces a specialized dataset generation approach optimized for producing a wide range of audio generation and transformation tasks, ensuring meaningful relationships between audio and language. To achieve compositional abilities—such as combining, interpolating, or negating instructions—Fugatto leverages ComposableART, an inference-time technique that extends classifier-free guidance to compositional guidance. This enables seamless and flexible composition of instructions, leading to highly customizable audio outputs outside the training distribution. Evaluations across diverse tasks demonstrate that Fugatto performs competitively with specialized models while ComposableART enhances its sonic palette and control over synthesis. Notably, the framework can synthesize emergent sounds—sonic phenomena that transcend conventional audio generation—unlocking new creative possibilities.