Can LLMs prove that code is correct? New paper: "miniCodeProps: a Minimal Benchmark for Proving Code Properties" https://t.co/emuMuSaGLZ miniCodeProps tests LLMs' ability to prove properties of simple Lean programs. Despite its simplicity, it's challenging! Led by Evan Lohn

@wellecks The authors evaluated two baseline approaches for automatically proving the properties in "miniCodeProps": a next-step tactic generation method using the LLMStep framework, and a full proof generation approach using GPT-4. They found that these baselines were able to prove only

CMU Researchers Propose miniCodeProps: A Minimal AI Benchmark for Proving Code Properties https://t.co/6EiFpMCwOW #AIinCodeVerification #miniCodeProps #TheoremProving #AutomationInTech #FutureOfProgramming #ai #news #llm #ml #research #ainews #innovation #artificialintelligen…

miniCodeProps: A minimal benchmark for proving code properties. ~ Evan Lohn, Sean Welleck. https://t.co/TGSBqw4rN5 #LLMs #ITP #Lean4 #FunctionalProgramming #Haskell

miniCodeProps: a Minimal Benchmark for Proving Code Properties.

CMU Researchers Propose miniCodeProps: A Minimal AI Benchmark for Proving Code Properties:

miniCodeProps: A minimal benchmark for proving properties of code. ~ Evan Lohn & Sean Welleck. https://t.co/4ak8WEYF2z #ITP #LeanProver #Lean4 #LLMs

"Despite its simplicity, miniCodeProps is challenging for current LLM-based provers, which succeed in proving about 25 percent of the specifications."

5. miniCodeProps: a Minimal Benchmark for Proving Code Properties https://t.co/emuMuSbeBx At Safe Generative AI, Sunday, Exhibition Hall A https://t.co/IdyxVKi5kt