LMS

LMS — sampling procedure for artificial intelligence generative models

Here are some key details about the LMS sampler for diffusion models:

LMS stands for Latent MCMC Sampler.
It is a sampling procedure for generative models like DDPMs (Denoising Diffusion Probabilistic Models).
Developed by researchers at Anthropic as an alternative to Langevin dynamics sampling.
The key idea is to introduce an auxiliary latent variable at each timestep during reverse diffusion.
These latent variables represent possible ways to reverse the noising process.
An MCMC sampler explores this latent space to find high probability paths back to the original data.
This avoids issues like sample quality deterioration seen in Langevin samplers.
Allows leveraging parallel chains and MCMC techniques like annealing and distillation.
Dramatically fewer sampling steps are needed compared to other methods.
Produces samples with significantly improved coherence and fidelity.
Does not need classifier guidance or denoising networks. Simpler to train end-to-end.
Enables generating 1024x1024 images with Anthropic's CLARA diffusion model.

In summary, the LMS sampler formulates diffusion model sampling as an MCMC problem with latent variables for each timestep. This results in very efficient and high-quality sampling.