- Associate each modality with an encoder \(E\) that maps data from that modality's data space into a shared latent space, called diffusion space, denoted by \(\mathbb{Z}\) .
- use an aggregate function that mixes noise, timestep and encoded latent representation as the forward function