Is complex prompt engineering a trap for advanced AI models? 🤯 My new paper finds that the "best" prompts for a model like GPT-4o actually HARM the performance of GPT-5. I call this the "Prompting Inversion" effect. Here’s what I discovered 👇

Is Dan Brown a time traveller? - theft at the louvre - worldwide virus - ai eating the world

Santa says 67? Why would santa say that?

They made Paul Rudd in makeup play a paleontologist and thought we wouldn't notice.

Is complex prompt engineering a trap for advanced AI models? 🤯 My new paper finds that the "best" prompts for a model like GPT-4o actually HARM the performance of GPT-5. I call this the "Prompting Inversion" effect. Here’s what I discovered 👇

Claude is back up!

Claude is down?

You can now chat with your browser console for debugging 🤯 Debugging will be much easier now!

And that's a wrap! If you found this thread on the "Prompting Inversion" useful, I'd be grateful if you'd retweet the first post to share the insight with others. Thanks for reading! 🙏 https://t.co/hodb01lzy0

By the way, I'm an independent AI researcher and a full-time software engineer. This research is a passion project I do in my spare time! 🚀 If you enjoyed this deep dive, give me a follow for more explorations into the weird & wonderful quirks of Gen AI.

Read the full details & analysis here: https://t.co/OuPwhaeInV

The main takeaway: Optimal prompting isn't universal; it co-evolves with model capability. As models get smarter, our prompts should get SIMPLER. The era of elaborate prompt engineering may be transitional. A "good prompt" for GPT-4 is a "bad prompt" for GPT-5

Eg: "Ben's iPhone is two times older than Suzy's (1 yr old)" 🔹 GPT-5 w/ simple prompt correctly understood the idiom: Ben's phone is 2 years old. 🔹 GPT-5 w/ strict prompt interpreted it literally: 1 + (2*1) = 3 years old. WRONG The guardrails made the smarter model act dumber

Why? The constraints forced GPT-5 to be hyper-literal, overriding its superior language understanding.

When I ran the same test on GPT-5, the trend completely inverted. The strict "Sculpting" prompt became "Handcuffs." GPT-5's accuracy TANKED from 96.4% with the simple prompt to 94% with the strict one. The constraints that helped the mid-tier model crippled the frontier model.

On GPT-4o, the strict "Sculpting" prompt worked like a charm 🤌 It acted as a "Guardrail," preventing the model from making common-sense mistakes. It boosted accuracy from 93% (simple CoT) to a stellar 97%. I thought we had a winner. More rules = better reasoning. I was wrong!

I tested 3 prompt styles on math problems (GSM8K) across GPT-4o-mini, GPT-4o, and GPT-5: > Zero-Shot: Just the question > Scaffolding: Simple Chain-of-Thought ("Let's think step-by-step") > Sculpting: A highly constrained method ("use NO common sense").

I published a preprint on @arxiv about how to make LLMs understand & follow your intent rather than be rigid about the prompt you write. Quite useful for making LLMs have a human-like common sense about exception handling and conflict resolution.

Can be done easily with a zero-shot Meta Prompt. read it here:

I published a preprint on @arxiv about how to make LLMs understand & follow your intent rather than be rigid about the prompt you write. Quite useful for making LLMs have a human-like common sense about exception handling and conflict resolution.