MusicLM vs VALL-E

Google introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff".

MusicLM casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task, and it generates music at 24 kHz that remains consistent over several minutes.

VALL-E has developed a context-aware learning function that can be used to synthesize high-quality personalized speech by simply recording an invisible speaker for 3 seconds as a voice prompt. Experimental results show that VALL-E significantly outperforms state-of-the-art zero-shot TTS systems in terms of speech naturalness and speaker similarity. Furthermore, we found that VALL-E can preserve the speaker's emotions and the acoustic environment of the acoustic prompts during synthesis.