🚨 @I4Replication released my experiment with a new academic-literary form, a “replication opinion.” I made myself judge in a debate and wrote an opinion that is more credible than either party could muster. Piece: https://t.co/77ZK9io50R Post: https://t.co/GPsnS4n2gW 🧵 follows

Do you look at this regression chart and think, “ah yes, that’s definitely a negative quadratic relationship”? If you don’t, then you can learn everything else wrong with this paper on my blog today.

Interesting, and maybe very efficient, approach to extracting information about how much to believe an empirical paper.

We've updated a paper on the 3-year, $1000/month U.S. guaranteed income study. New results, in 3 figures: 🧵 1) Subjective well-being significantly improved in the treatment group in year 1, but there were no sig differences between the treatment & control group after that. 1/

David Roodman writes: consumers of economic research are more truth-seeking than the producers. Here is Roodman's recap of his re-analysis of a set of papers on temperature and judge decision-making. The comment process is not working that well! Gory details in 🧵 1/

Who cares when a research claim is found to be in error? Peer-reviewed journals do their best to deflect and dilute legitimate criticism. https://t.co/BTFBNaoTQn

Replication doesn't make you popular, which is one reason it is undersupplied and sorely needed. I'm proud @open_phil is supporting @I4Replication. How the Institute for Replication is making social science more robust and reliable | Open Philanthropy

Good illustration of how much the Texas grid has changed in just 6 years. yellow = solar; purple = batteries; dark green = wind; blue = gas; brown = coal; light green = nuke via @grid_status

Springer Nature book on machine learning is full of made-up citations. https://t.co/9SQ4LkVKxf

"How Peer Review Became Science’s Most Dangerous Illusion." https://t.co/YBMZ8KnCDB

My critical replication of a study of childhood vaccination and adult wages in India is final & effectively published Thread: https://t.co/1SjUjserJz Free pub: https://t.co/Lb5MjaTzPf

Whistleblower Raphaël Lévy on the systemic discouragement for calling out scientific fraud: "if a student or a more junior researcher had faced such a backlash it is unlikely that they would have been able to stand up to it."

Good title, good conclusion

👀Using an autonomous agent based on o3-mini and GPT-4.1, a team from Harvard, MIT & other institutions reproduced and updated an entire issue of Cochrane Reviews in two days… saving 12 person-years of work! The AI reviews captured more papers & were more accurate than humans.

"Many global health problems are thorny and difficult. Lead, unusually, is not one of those. We already know how to reduce lead exposure. Rich countries have already done it." Great new @WorksInProgMag piece by @C_Don7 @notanastronomer and James Hu

New podcast alert! Hard Drugs - a collaboration between @JacobTref at Open Phil and @salonium at @WorksInProgMag. Listen here, or wherever you get your podcasts:

A million lives at risk? @charlesjkenny and I estimate the possible toll from the aid cuts in the White House's FY26 budget request. There's huge uncertainty here, of diverse kinds, so we explore 3 broad ways we could be wrong and by how much. https://t.co/EYOYresFWk

Those doing the vetting would need to be embedded in an incentive regime outside academia, one that frees them to worry more about the consequences of error. Like red teams at the CIA & DOD or GiveWell, with its great "what if we're wrong" work ( https://t.co/5zw8jOOEu5)

More to the point, and more constructively, maybe research dissemination and review need to be decoupled. If we abolish peer review, the vacuum of caveat emptor would create demand for independent, credible vetting of important research.

I'm increasingly an anarchist on peer review & journals. Would we be better off without them? The publication system creates intense pressures that distort what is published. Then, in the public eye, it certifies the results as reliable.

Update: AEJ Applied rejected my "replication opinion" on the debate they hosted. It was an odd duck. But there's still something wrong when research consumers structurally care more about truth than the producers & the former overly trust peer review & the latter play along.