Luca Eyring
@LucaEyring
Followers
441
Following
5K
Media
11
Statuses
68
@ELLISforEurope PhD student @ExplainableML, Research Intern @InceptiveCom
Munich, Germany
Joined October 2022
Reward hacking is challenging when fine-tuning few-step Diffusion models. Direct fine-tuning on rewards can create artifacts that game metrics while degrading visual quality. We propose Noise Hypernetworks as a theoretically grounded solution, inspired by test-time optimization.
8
51
344
RT @zeynepakata: To integrate test-time scaling knowledge into a model during post-training in diffusion models, we replace reward guided t….
0
9
0
RT @LucaEyring: Reward hacking is challenging when fine-tuning few-step Diffusion models. Direct fine-tuning on rewards can create artifact….
0
51
0
@ShyamgopalKart1 @natanielruizg @zeynepakata @ajwagenmaker @mitsuhiko_nm @yunchuzh @svlevine And @siddarthv66, @mh_steps, @FelineAutomaton propose Outsourced Diffusion Sampling, which uses a GFlowNet-based trajectory balance objective on a black-box reward and works for arbitrary generators:
Is there a universal strategy to turn any generative model—GANs, VAEs, diffusion models, or flows—into a conditional sampler, or finetuned to optimize a reward function?.Yes! Outsourced Diffusion Sampling (ODS) accepted to @icmlconf , does exactly that!
0
0
3
@ShyamgopalKart1 @natanielruizg @zeynepakata Beyond HyperNoise, there's been some really cool concurrent work on noise-space networks. E.g. @ajwagenmaker, @mitsuhiko_nm, @yunchuzh, @svlevine propose noise-space RL for robotics (DSRL):
Diffusion policies have demonstrated impressive performance in robot control, yet are difficult to improve online when 0-shot performance isn’t enough. To address this challenge, we introduce DSRL: Diffusion Steering via Reinforcement Learning. (1/n).
1
0
1
RT @natanielruizg: We are releasing a paper I'm very excited about. We know test-time scaling is a path to greatly improved results, and ac….
0
43
0
RT @multimodalart: I've built a demo for the ultra-fast high quality HyperNoise Sana Sprint 0.6B! 🔥. thanks for the team @Google for open s….
0
12
0
RT @ShyamgopalKart1: I'm really excited to share our new formulation for post-training diffusion models! Here's why I think this formulatio….
0
8
0
RT @iScienceLuvr: Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models. "we replace reward guided test-time noise optimiza….
0
27
0
@ShyamgopalKart1 @natanielruizg @zeynepakata Also check out the thread by @natanielruizg on HyperNoise here!.
We are releasing a paper I'm very excited about. We know test-time scaling is a path to greatly improved results, and achieves reasoning in the case of LLMs. We present a new and promising way to amortize it into training using HyperNetworks for image generation models.
1
0
3
This work is the result of a great collaboration together with @ShyamgopalKart1, Alexey, @natanielruizg and @zeynepakata! For all the details, check out:. 📜 Paper: 💻 Code: 🤗 Model:
huggingface.co
1
0
5
HyperNoise amortizes the benefits of test-time optimization. Our GenEval results confirm we capture a significant portion of the gains from test-time scaling, but with only marginal costs at inference. This makes powerful alignment a practical reality for fast generators.
1
0
3
HyperNoise scales to complex human-preference rewards, where direct fine-tuning also suffers from reward hacking. Our method avoids these visual artifacts, significantly boosting prompt-following & aesthetics across multiple models.
1
0
3
We show that operating in noise space rather than model weights makes KL regularization tractable for few-step diffusion models. This enables proper regularization that prevents reward hacking while maintaining computational efficiency.
1
0
4
HyperNoise targets the reward-tilted distribution that balances high rewards with base model fidelity. However, instead of modifying generator weights, we propose to learn a reward optimal noise distribution using a LoRA parameterization.
1
0
7
This redness reward example illustrates how direct fine-tuning introduces unnatural artifacts that maximize rewards while destroying image quality. The fundamental problem: KL regularization is intractable for few-step diffusion models, leading to unconstrained reward hacking.
1
0
4
RT @ExplainableML: 🎓PhD Spotlight: Karsten Roth. Celebrate @confusezius, who defended his PhD on June 24th summa cum laude! . Karsten has b….
0
6
0
RT @ExplainableML: 🎓PhD Spotlight: Shyamgopal Karthik. Celebrate @ShyamgopalKart1 , who will defend his PhD on 23rd June! Shyam has been a….
0
6
0
RT @ExplainableML: #CVPR2025 is heading to the 'Music City' — Nashville! 🎺 Join us from June 11–15. We're thrilled to announce that we'll b….
0
5
0