mako tune model

And for what its worth, im 99% sure this is what facebook was doing with Llama. But instead of trying to create vulnerabilities, they want to sell ads (as preferences in LLMs) . Imagine if you ask Llama for a cold refreshing drink, will it recommend Coke or Pepsi? I'd bet my last.

The craziest, not-impossible scenario is one where they want SOTA open-source coding models to vibe-code specific vulnerabilities into a ton of apps. something like this:.1. Infiltrate many standard open libraries .2. Train model to prefer infiltrated libs .3. A user vibe-codes.

we added a leaderboard.

What.

Now the real question. did it work for TriMul? Not yet 😅 but this is a pretty complex kernel with several parts! I'm gonna keep trying with some prompts. Internally, we have an agent that uses evolutionary search that is finding some humble successes. That'll be released soon

For the prompt, try adding seed kernels or examples. Try adding docs. Add anything you want! We've measured non-trivial performance differences that arise just from the additional prompt a user can add.

Next, validate the input format. this will make sure our validation pipeline actually works. Once you get the check mark, configure your agent, add a prompt, and you're off to the races!

Next, enter your problem in the appropriate format. We will use TriMul from @a1zhang's GPUMODE contest. You can find it here: To get it in the right format, i just copy/pasted the reference format and the trimul from gpumode into ChatGPT and told it to.

First make an account at to generate GPU kernels for free. From there, you can select "Your custom problem" from the drop down menu. This will open up the template you need to adhere to for our validation pipeline to work.

MakoGenerate now supports custom problems, meaning you can generate #CUDA or #Triton kernels for any @PyTorch reference code you have! . Lets walk through an example using @GPU_MODE's latest contest: Triangle Multiplicative Update (TriMul) module

the behind the scenes that led to this post must have been wild lol. either The Information had bad info (less likely) or Nvidia went back to OpenAI and made them an offer they could not refuse.

The implications of this result are much greater than meet the eye.

Before literally every single model release, people claim “this is it” . When it really is “it”, will we even know?.

LLM+Search turns out to be pretty good at writing kernels.

The splendor of AI's capability knows no bounds.@mako_dev_ai 💙 @modal_labs.

@OpenAI @Anthropic @GoogleDeepMind Lastly, a shout out to the plethora of prior work that has shown up over the last 12 months. @GPU_MODE is an amazing community doing some great and open source work in this area, including releasing KernelLLM. @ScalingIntelLab and everyone involved in creating KernelBench have.

@OpenAI @Anthropic @GoogleDeepMind Now watch as the code is generated, compiled, validated, and benchmarked! Depending on the compute budget you assigned while configuring the agent, MakoGenerate will iteratively refine and try to improve the kernel by applying various optimization techniques. (6/6).

@OpenAI @Anthropic @GoogleDeepMind Additional prompting is one of the most interesting features in MakoGenerate. It begs the question, can we prompt engineer our way to superior kernels? Your prompts can be empty, one line long, or pages long. It can include in context examples of high quality kernels,.

Next, configure your agent. 1️⃣ pick a model, including the best from @OpenAI, @Anthropic, and @GoogleDeepMind 2️⃣ choose either #CUDA or #Triton 3️⃣ select a hardware platform for execution 4️⃣ and importantly, select the number of kernels to generate. (4/6).

Start with selecting a PyTorch reference. The first 200 problems are from @ScalingIntelLab KernelBench Level 1 and Level 2, which includes simple operators and simple fusion patterns. We also include 14x Level 5 problems, which cover more real world and complex use cases. (3/6).