We have a message for Facebook: The public deserves more transparency about the systems the company uses to sell the public’s attention to advertisers and the algorithms it employs to promote content. We will keep working to ensure the public gets it.
As someone who studies misinformation, the past week has been a masterclass in how positive actors with a strong information operation and tech platforms being (somewhat) sensible can create an environment in which misinformation struggles to take hold. A 🧵.
This evening, Facebook suspended my Facebook account and the accounts of several people associated with Cybersecurity for Democracy, our team at NYU. This has the effect of cutting off our access to Facebook's Ad Library data, as well as Crowdtangle. 1/4
ONE NEAT TRICK for making an information space hostile to misinformation: Flood the zone! The US government deserves credit for doing this early. Not leaving an information vacuum for your opponent to fill makes their job much, much, harder. 2/8
So what can we learn from all this?
1) Flood the zone with accurate info. Info vacuums will be filled if left for any real length of time.
2) Humanize your side. Cats can be helpful.
3)Take out the worst actors spreading out misinfo.
7/8
Of course, none of this matters without ground truth being on your side. Zelenskyy is truly a remarkable leader. The Ukrainian resistance has shocked and inspired the world. The information war doesn't matter if the kinetic war is lost. And they haven't lost yet.8/8
First, we have to acknowledge the role of fortune. I'm wary of lionizing leaders, but Zelenskyy certainly is a man made for the moment. His personal bravery and outstanding communication skills have made it possible for the other stuff I'm going to talk about to work.1/8
BTW, if you've ever wondered why misinformation can be so effective and why debunking often doesn't change beliefs, *this* is why. If you want to believe the story the misinfo supports, even after the specific claim is debunked, the overall impression sticks. Job Done. 4/8
CLICK HERE to see the
#ghostofkyiv
, that badass lady with the sunflower seeds, the heroes of Snake Island. These are, at minimum, factually questionable. But they are conveying a sense of the Ukrainian people that is sticking. Even after they're debunked, the feeling remains. 3/8
CATS vs. DOGS. The importance of animal images in internet culture and information warfare has been discussed, at length, by everyone. Here it is again, in all its fuzzy glory. Mwah, chef's kiss. 6/8
TECH PLATFORMS: They're not totally beefing it! They deserve credit for blocking RT and Sputnik in Europe. But not blocking long-time spreaders of misinformation globally when the government that controls them is actively trying to lie about their war atrocities is... NUTS. 5/8
A week ago, Facebook sent me a C&D asking us to take down AdObserver and delete our data. The public has a right to know how political ads are targeted, so we will not be complying with this request. Please consider installing
Over the last several years, we’ve used this access to uncover systemic flaws in the Facebook Ad Library, identify misinformation in political ads including many sowing distrust in our election system, and to study Facebook’s apparent amplification of partisan misinformation. 2/4
By suspending our accounts, Facebook has effectively ended all this work. Facebook has also effectively cut off access to more than two dozen other researchers and journalists who get access to Facebook data through our project, 3/4
including our work measuring vaccine misinformation with the Virality Project and many other partners who rely on our data.
The work our team does to make data about disinformation on Facebook transparent is vital to a healthy internet and a healthy democracy. 4/4
I know what's going on in the US is a lot right now, but you need to pay attention to what is being done to
@rapplerdotcom
and
@mariaressa
. Something we are starting to understand about disinfo is that it is most easily spread in an information vacuum. 1\4
Are you, like me, incessantly scrolling Twitter looking for information about a breaking news event? Pause with me for a minute so we can talk about why this is a ripe situation for you to be receptive to disinfo and how to avoid believing and spreading false information: 🧵1/7
Tech companies are threatening to sue researchers who study them, and I have some first-hand experience surviving this! But researchers shouldn't shy away from this research, and at least this attack vector is less powerful than it appears. Quick 🧵/11
I'm so tired of being in rooms where people whisper about the absolute ARMY of Big Tech-funded people (most, but not all, ex-Googlers) that have popped up in nearly every corridor in DC where people are working on literally anything to do with AI. So let's talk about it! 1/12
To see how we use data collected by our browser extension users, take a look at AdObservatory, where we combine that data with data we collect from Facebook’s Ad Library API to create a fuller picture of political advertising on Facebook:
Big Data cannot do this. Big Data is not a magic wand that you can wave. It's especially not magic to predict very rare events! It is, in fact, particularly bad at that. We in the technical community need to do better at communicating what problems ML techniques can/can't solve.
Facebook shut down our research into its role in spreading disinformation. The stakes for everyone are too high to let Facebook control who can study disinformation online.
Let's talk about the Facebook Election studies that came out yesterday! Contrary to the headlines, there are interesting findings here that show the polarizing impact of FBs algorithms both on what content users see *and on their behavior*. TL;DR 'Wired to Split' is right. 1/15
How is this disclosed political ad from a congressional candidate active right now on Facebook at 7:21 ET? We’re going to try to quantify this problem now, no idea yet how widespread this is...
We're very gratified for the FTC's clarification that Facebook is not required to shut down our work under their consent decree. It's time for Facebook to reinstate our accounts and allow our research to continue. 1/2
1) Hold on to uncertainty (for much longer than you are comfortable with. 2) Find out WHO is telling you something and WHY they are telling you. Assign credibility accordingly. 3) Avoid sharing/spreading information until you have very high confidence. Don't tweet your hope! 2/7
Why should you listen to me about this? First, if anyone is Not A Shill For Meta, it's me. Second, I spent 2022/23 as the Chief Technologist of the Antitrust Division of the Department of Justice, so I've spent a LOT time thinking about Computational Antitrust :).12/12
Over the past few weeks, the Facebook Papers have reaffirmed that FB has fueled the spread of hate speech & misinfo around the world.
But there’s a crucial piece missing from the story. FB isn’t just amplifying misinfo.
The company is also funding it.
These 'AI Experts' (none of whom seem to be computer scientists, btw, they have degrees in Philosophy or Cosmology or Intl' Studies from Oxford, Stanford, or similar) are pushing a few lines of argumentation about AI, but I want to focus on one: Open Weight Models Are Bad!3/12
We, along with researchers from several institutions, are advising Gizmodo on how to publish the FB Papers safely. We think this is the most responsible way to move the public conversation forward about Facebook's research into harms on its platforms.
I've been thinking about the NYTimes revelations about Facebook holding their 'Most Widely Shared' list for a quarter until the results looked better. The one good that came out of all this is the clear demonstration that these 'transparency reports' are NOT real transparency. 1/
@bat_broken
We’re interested in ads globally! This year, we’re focusing on the U.S. elections, but we make our political ad data available to researchers from other countries as well, so that local experts can use our data to monitor political advertisers in their own countries.
First, this vulnerability exists because we don't properly fund our government anymore. So if some org can fund expertise getting into Congress or agencies, it would be irresponsible to say no because the government can't afford to hire the experts it needs otherwise.2/12
Lastly, please, in this environment, avoid sharing unverified media or reports from sources with a perspective to advance. It's super tempting to share something you *want* to be true, or to share to ask if something is true. Please, for all our sakes today, don't do it.7/7
First, remember your brain hates uncertainty. It's uncomfortable! Emerging situations are tense because we don't know what will happen next and that feels bad. Your brain just wants an answer, ideally, expressed with certainty. Bad news: we're in an uncertain situation.3/7
I and many other researchers have spent years of our lives trying to understand if the harms people were experiencing on Facebook were anecdotal or systemic. We learned this week that Facebook researchers were far, far ahead of us. The harms are systemic. And they knew.
Facebook has lost the right for anyone to assume they are acting in good faith. Upper management acted recklessly with their users' mental and even their physical safety. It's time for legislation.
@SenAmyKlobuchar
,
@ChrisCoons
,
@MarkWarner
are thinking about this, I know.
Do not make the mistake of thinking that if this attack on journalism is successful, it will not be attempted elsewhere. Governments learn from each other what will be tolerated. This cannot be tolerated.4\4
So now it is abundantly clear.
@JeffHorwitz
and the
@WSJ
weren't cherry-picking, but actually only reporting the tip of the iceberg. If you're a teen or tween who struggles with mental health issues, IG is not a safe place. And many young people are struggling. 1/
I'm starting to think of mis- and disinformation as an opportunistic disease that attacks media ecosystems that are already weakened, but struggles against healthier ones. This is why the great journalism that Rappler has consistently provided is so important.2\4
@TwitterSafety
What is your plan for making political ads transparent to users and the public? Will you disclose the ad payers to users with the ad? Will you offer a public database of political ads?
What is left to say about Facebook? A 🧵. This week, we learned about important, solid research into the harms the platform causes, and even some potential mitigations. Facebook was hurting people, and they knew. And upper management didn't care enough to do anything, because $$.
Voluntary transparency has failed. Facebook isn't willing to allow any outside scrutiny of how their business operates, and they are willing to weaponize their Terms of Service to do it. I haven't reviewed the code of AlgorithWatch's extension, so it's hard for me to say much 1/
1/9 ⚠️ AlgorithmWatch was forced to shut down its
#Instagram
monitoring project after threats from
#Facebook
!
👉🏻 Read our story:
✍🏻 And sign our open letter to European lawmakers to protect future research on online platforms: 🧵
In personal news, as they say… I’m starting as an Assistant Professor of Computer Science at
@Northeastern
next week. I’ll be recruiting PhD students this year, so if you want to work with me you should to apply to
@KhouryCollege
!
If you weren't paying attention to Ukraine over the last week.. you should. Aside from a stunning collapse of the Russian military (which has been rotting for decades, apparently), it is managing to show both the limits of disinformation and the power of the information war. 1/8
Unfortunately the only way to deal with this is to consciously hold on to uncertainty for much longer than feels good. When you see information, think of it as one input, one puzzle piece of a picture you can't see yet, possibly from another puzzle. 4/7
It's also why a government that is itself the source of disinformation is doing everything it can to shut down an honest, fair-minded source of accurate news. Maria makes it much, much harder for her government to lie to people. 3\4
WHO: We are in a rapidly changing situation with actors known to employ disinfo as a tool of war. One regularly releases voice memos(!!) This is a vulnerable environment for manipulated media and deep fakes. Don't believe video/audio recordings without provenance. 5/7
In 2022, I achieved one of my life goals since childhood: I finished my PhD. Grad school & learning how to do research was... not what I was expecting! It's not for everyone, but it's much better than the horror stories I'd heard. A 🧵 on what it *is*, if you're considering it.
This is because open source or weights had been the norm for a while: it was weird when Open AI made GPT-3.5 closed with a 'safety' rationale that only makes sense if you forget they didn't restrict the use case (misinformation) that they claimed was the motivating risk.7/12
WHY: Even for actors you are certain about - ask why they are sharing information. Consider what perception (on your part) is in their interest and whether the information they are sharing is consistent with that perception. Discount that information accordingly. 6/7
This has been going on for a while and has cycled through a few justifications. Previous ones have been a bit silly (China! SkyNet! Biotoxins!) but the current one (Open is Anticompetitive!) isn't quite as ridiculous on its face, so I want to explain why it's dead wrong.4/12
Open source has also won the argument about security. Everyone (good and bad) finds bugs faster. But on balance, this appear to make software more secure, because there are more defenders than attackers.11/12
Damon McCoy doesn't use Twitter, but this is what he has to say:
It is disgraceful that Facebook is attempting to quash legitimate research that is informing the public about disinformation on their platform. 1/3
Today is my last official day as Chief Technologist of the Antitrust Division of the Department of Justice! Everyone asks me what that's like, so I wanted to put together a thread for anyone contemplating public service, particularly as a political appointee.1/8
So why haven't all these OS/OW models stopped other models in their tracks? Even upstarts like Mistral or DataBricks? Because the foundational LLM just IS NOT the thing that gets sold, and never has been. What all these companies sell is helping you use the LLM.8/12
A short selected history of open weight/open source LLMs:
BERT: 10/18
GPT-2: 11/18
RoBerta: 07/19
GPT-NeoX 2/22
Llama 2: 7/23
Mixtral 7B: 9/2023
DBRX 3/24
Llama 3 4/24
Open models sure don't see to be impeding development of either open or closed models.6/12
What was done to
@BostonJoan
is frightening. We're in a race to study how and why social media is so vulnerable to misinformation so we can make these systems safer. We need her research, and we need HER. What a black mark for academic freedom.
The argument goes like this: (free) open models from big companies like Meta mean that smaller companies can't compete, becuase they can't afford to not charge. Which sounds plausible, but is nonsense when you look at the business model of LLMs or the history of software.5/12
And even outside of models, open source software has won the argument. Open source means that everyone can innovate faster, so everyone counts on making more money because everyone gets more out of the common ecosystem than they put in.10/12
So I'm really glad
@CCDHate
is fighting back! The more of us who stand our ground and don't give in to legal blackmail, the safer the research community is as a whole. Because doing that, and our jobs, exposes these attacks as baseless and the legal threats as paper tigers.11/11
The more time I spend reading this bill from
@ChrisCoons
,
@robportman
, and
@SenAmyKlobuchar
, the more impressed I am. But there's a LOT in here, and its 35 pages, so I want to highlight the most important provisions. A 🧵. 1/7
Some background: If you're old enough (I'm not, but so my parents tell me), it used to be unclear if smoking gave you cancer. Big Tobacco figured out the link first, and then spent decades trying to discredit outside researchers who discovered it and tried to warn people.2/11
“We are knowingly exposing users to misinformation that we have the processes and resources to mitigate,” said a 2019 memo by Facebook researchers, called “The Political Whitelist Contradicts Facebook’s Core Stated Principles.” Well that explains a lot.
That's why DataBricks debuted (open) DBRX before they IPO'ed - it's critical to their business success because they make money on ALL THE STUFF that gets built on and around a model (especially by them.)9/12
This is a really impressive piece of work by many of the smartest big picture thinkers in this space. I don't agree with everything in here, but it's the most comprehensive attempt to move the conversation toward solutions so far.
via
@aspeninstitute
is not in the public interest. Facebook should not be able to cynically invoke user privacy to shut down research that puts them in an unflattering light, particularly when the “users” Facebook is talking about are advertisers who have consented to making their ads public. 3/3
With its platform awash in vaccine disinformation and partisan campaigns to manipulate the public, Facebook should be welcoming independent research, not shutting it down. Allowing Facebook to dictate who can investigate what is occurring on its platform 2/3
Car makers had data about how many people were dying in their cars and why those people were dying YEARS before the general public knew. When researchers tried to call attention to unsafe cars, automakers, particularly GM, reacted by attacking the researchers.3/11
Here's a shortlist of what we need:
Public availability of public data
Annual auditing to verify users aren't being harmed (As is required for banks)
Consumer transparency & appeal, requiring platforms to disclose reasons to users when they take actions against them.
THEY KNEW.
Last Saturday, Facebook made a change to their DOM that had the effect of breaking Ad Observer, as well as
@themarkup
's Citizen Browser, and most importantly, screen readers for the visually impaired. It only impacted us for a week, but screen readers can't be fixed as easily./1
Without getting too into details, the first time I was nearly sued, a lawyer had to sit me down and explain to me that *actually* getting sued was highly unlikely because the party threatening to sue me had a lot more to lose in discovery than I did. 9/11
What would we do with platform transparency if we got it? I attempt to answer this question with
@noUpside
,
@BrendanNyhan
, and
@EthanZ
in
@sciam
. A quick 🧵here to talk about our 4-5 biggest questions.
about this case specifically. However, I do know that legal threats from Facebook have chilled desperately needed research. We need to start talking about what regulation requiring more transparency from social media companies and ad platforms will look like.
Because of recent events, I'm going to do a quick run-down of the privacy guarantees (including from law enforcement) of different messaging channels. Topics we'll cover: basic device OpSec, terms to look for, actual app/channel recommendations.1/10
Hi! I'm the unicorn who graduated from an undergrad without debt, rich parents (or a college fund), funded solely by my work during college. How did I do it? I'll tell you, but in short, it was a path no one should want for their kids. The system is broken.
Facebook's upper management sacrificed user safety to maintain audience engagement. They knew that was the trade-off they were making, and they made it.
@SenBlumenthal
and
@MarshaBlackburn
have said they are investigating, and they should.
So this morning when Facebook data collection was broken, I actually knew why and we didn't have to spend hours debugging! So the time savings is racking up already.
But threats are cheap to make and expensive to get: a few hours of a lawyer's time makes one, but dozens or hundreds of hours to manage. Researchers don't have an extra 50k in the budget to deal with threats. Threats like this have shut down research in the EU and the US.10/11
So, on to the legal threats. In their response letter, CCDH's lawyer said this. If you've never been through a legal proceeding, an initial phase is Discovery, where each side must hand over evidence to the other that might be relevant to the case. 8/11
TODAY:
@cyber4democracy
is launching the new and improved version of Ad Observatory. Want to know what's going on in US political ads on Facebook? This should be your starting place. A quick thread on new features and what you can do with the new site:1/11
But non-academic research like this is still really important for advancing public understanding. Remember I told you about GM attacking researchers? Specifically, they attacked Ralph Nader, who had written a popular, qualitative book about unsafe car design.6/11
Let's talk about
@CCDHate
for a sec. They are a non-profit civil society org that does what it says on the tin. Their research tends to be qualitative rather than quantitative, and they don't publish in academic literature. But they are very effective at driving change.4/11
Scientists don't have a corner on research- other people like civil society researchers and journalists get to do it too! We do have to be careful about how much confidence we ascribe to a finding, and what we say it means. Science is super duper careful for a reason. 7/11
This thread is WILD. And it is case in point against closed datasets. “It’s a great demonstration that even a little transparency can provide amazing results.”
@fabiogiglietto
, the researcher who found the bug.
This needs to stop happening.
@louisbarclay
's Unfollow Everything is not just a tool that helps Facebook users, it had the potential to be a really useful research tool. How can we test mitigations to reduce harm if Facebook just bans them?
Facebook says their platform is about birthdays and new puppies, but production of content appears to be heavily driven by political events. This is important to keep in mind when trying to understand the company’s incentives.
Nestled at the bottom of one of the slide decks that dropped on Wednesday was this graph of Facebook production participation rates- the share of FB users who post, comment, or share. The annotations are mine, but WOW, look at the spike at the election. 1/
To be super clear about something, I am also really concerned about AI safety and think much needed to be done to manage the risks. My point in this thread was only that open models don’t make these risks worse and in many cases help mitigate them.
Devastatingly, Facebook users' feeds grow more polarized at every step of the recommendation algorithm, both the inventory selection stage and the ranking stage. Users then engage more with this more polarized content - the algorithm is doing what it is trained to do here. 15/15
I want to clear up some confusion about AdObserver: we don't collect any personally identifiable information, period. The suggestion that our browser extension harms user privacy is false. You can verify this yourself - our source code is public:
No,
@andymstone
, I wouldn't rather Facebook didn't do this research. But it is a betrayal to both Facebook's users and to the many employees who did this research to disregard the clear findings of harm to users. And that falls squarely on upper management. Who. knew.
Their Disinformation Dozen report put a name on a phenomenon a lot of folks were seeing and helped a broad population understand the vastly concentrated drivers of vaccine disinfo. But it wasn't peer-reviewed, and work like this isn't designed to advance the science.5/11