The Diffusion Duality, Chapter II: $\Psi$-Samplers and Efficient Curriculum
Justin Deschenaux, Caglar Gulcehre, Subham Sekhar Sahoo
https://arxiv.org/abs/2602.21185 https://arxiv.org/pdf/2602.21185 https://arxiv.org/html/2602.21185
arXiv:2602.21185v1 Announce Type: new
Abstract: Uniform-state discrete diffusion models excel at few-step generation and guidance due to their ability to self-correct, making them preferred over autoregressive or Masked diffusion models in these settings. However, their sampling quality plateaus with ancestral samplers as the number of steps increases. We introduce a family of Predictor-Corrector (PC) samplers for discrete diffusion that generalize prior methods and apply to arbitrary noise processes. When paired with uniform-state diffusion, our samplers outperform ancestral sampling on both language and image modeling, achieving lower generative perplexity at matched unigram entropy on OpenWebText and better FID/IS scores on CIFAR10. Crucially, unlike conventional samplers, our PC methods continue to improve with more sampling steps. Taken together, these findings call into question the assumption that Masked diffusion is the inevitable future of diffusion-based language modeling. Beyond sampling, we develop a memory-efficient curriculum for the Gaussian relaxation training phase, reducing training time by 25% and memory by 33% compared to Duo while maintaining comparable perplexity on OpenWebText and LM1B and strong downstream performance. We release code, checkpoints, and a video-tutorial on: https://s-sahoo.com/duo-ch2
toXiv_bot_toot
SUPERVIKTIGT - SKRIV PÅ FÖR ATT STOPPA ATT PALANTIR FÅR ANSIKTSÖVERVAKA SVENSKAR.
#integritet #palantir #sverige
🇺🇦 Auf radioeins läuft...
Tracy Chapman:
🎵 Crossroads
#NowPlaying #TracyChapman
https://boddhisatvaofficial.bandcamp.com/track/tracy-chapman-crossroads-boddhi-satva-ancestral-soul-remix
https://open.spotify.com/track/4rxamav7i2lJexbuAxcvfe
This will be my first St Patrick's Day since finding details of my Irish ancestors. If you can call them that. They were Norse invaders, descended from Ivar the Boneless. By 1050 they had moved to Wales, where they stayed 600 years before coming to America.
"Climate anxiety: Misconceptions and advice"
#Climate #ClimateChange
https://phys.org/…
KI in der Verwaltung: Bürgermeister wird in Büttelborn zum Avatar, mal schnell Chatbot auf Webseite einbauen. Marcus Schwarze in der #FAZ (€)
Das ist nicht, wo KI in der Verwaltung wirklichen Nutzen bringen muss. Im Backend bei den Fachverfahren spielt die Musik angesichts immer weniger Personal, siehe unser Gespräch bei
Wegen Körperverletzung - Patientin schrie „Stopp“: Frauenarzt angeklagt #News #Nachrichten