Increasingly, social scientists are advocating for the use of LLM agents as human stand-ins. The assumption is: these things can respond and generate text like humans. But is that true? Our new paper suggests not quite...

How common are “survey professionals” - people who take dozens of online surveys for pay - across online panels, and do they harm data quality? Our paper, first view at @polanalysis, tackles this question using browsing data from three U.S. samples (Facebook, YouGov, Lucid):

If this piece in @@The_JOP sounds interesting to you, you find yourself in Atlanta, and you're not opposed to waking up early: I'll be presenting a related working paper on WhatsApp usage in India, Brazil, and South Africa tomorrow at #PolMeth! https://t.co/mMHhIVjvkY

🚨Publication alert!! Our WhatsApp multimedia deactivation paper is now "just accepted" at @The_JOP. @CSMaP_NYU has published a full thread summarizing the paper! Check it out below 👇🏽👇🏽👇🏽

How much can a social media messaging app influence elections through misinformation? McCourt's Assistant Professor @_Tiagoventura offers critical insights in a new paper 👇

In addition to the original UK results, we have now 👏replicated👏 this (TWICE) in the US. The main findings hold strong: information diets are a lot more diverse in attention than in engagement.

Read a longer summary of the paper here:

Congrats to the authors @_tiagoventura @RMajumdar_ @Jonathan_Nagler & @j_a_tucker. The paper, which is accepted for publication at @The_JOP, can be found here:

We are working on a second study to expand this growing body of research to more countries, including India and South Africa. Our new study introduces a time-reduction treatment arm and a broader scope of charged political content beyond misinformation, with similar results.

Our study was a rare field experiment on misinformation in the Global South, adding to a growing call to broaden the geographic and platform scope of causally identified misinformation research.

Big takeaway: WhatsApp matters—but changing exposure does not mechanically change attitudes in the short run. Political beliefs are hard to change and probably require long-term interventions.

The paper also indicates: ➡️ Platforms like WhatsApp differ fundamentally from traditional feed-based platforms ➡️ The academic community must spend time studying such platforms ➡️ While difficult to study, it's crucial to explore how those in the Global Majority consume info

These findings have a key nuance: heavy WhatsApp users—those who frequently receive political content—did improve in spotting falsehoods, while others did not. This suggests that information interventions may have unequal impacts across subgroups, depending on baseline exposure.

Importantly, we also found no evidence that treated users substituted WhatsApp with other platforms. Users did not migrate to Facebook, Instagram, or Telegram, but instead, watched a bit more TV.

We are not the first, and unlikely to be the last, to find such mixed results. Our findings echo recent Facebook deactivations & RCTs manipulating online informational spaces (e.g., re-shares, algorithmic feeds, landing pages) that show largely null effects on users’ attitudes.

These findings are consistent with the “minimal effects” theory: the misinformation reduction did not translate to user belief accuracy and polarization changes. Although users saw less false content, their attitudes stayed the same.

Overall, the recall of misinformation dropped sharply. Participants were 40% less likely to remember false headlines, a significantly larger reduction than the decline in recall of true news.

Our design recruited 773 WhatsApp users before the 2022 election. The treatment group (N≈400) turned off all auto-downloads (videos, images, audio, & docs) for 3 weeks, while the control group kept their usual settings. We also verified compliance weekly via storage screenshots.

We extend a well-known method for measuring social media’s causal effects in two directions: 1. Applying it to WhatsApp (not a feed-based platform). 2. Focusing on multimedia—the modal format of misinformation spread on WhatsApp.

What makes WhatsApp unique? Despite no news feed, millions of Brazilians use it as a key source of political info. But it's also where they see the most misinfo, primarily via viral videos/images & group chats. So we designed a deactivation experiment for this landscape.