T2R2D2

Timbre Transfer for R2D2-alike Robot voice turning into instrument using Diffusion Model

Dataset and Preprocessing

Model Architecture

Audio to Spectrogram

• wav2spec (SoundStream) converts audio to spectrogram
• The input audio is converted to a spectrogram using the wav2spec function in the SoundStream class.

Encoder

Decoder

Diffusion Process