Sarah Schiavone
@SchiavoneSays
Followers
796
Following
2K
Media
136
Statuses
2K
Postdoc at the University of Melbourne. Metascience. Open science. Space enthusiast. Gamer. Nerd. 🏳️🌈 She/her
Joined September 2009
Evaluating research is hard. @kimberlyquinn, @siminevazire, and I developed a tool ( to help reviewers identify threats to the validity of empirical research with the help of 50+ experts. Here’s our preprint introducing it! [1/7]
osf.io
We introduce a tool to aid reviewers in identifying potential threats to the validity of empirical research. This tool was developed through a three-wave process that elicited consensus-based expert...
4
124
349
c) Do any examples come to mind of threads you think have done this well? Please share! 4/4.
0
0
1
Imagine a researcher tweets about their recent findings (e.g., new article, preprint), what do you think makes a twitter thread like this particularly effective or ineffective?.a)… when communicating with other scientists? 2/4.
3
0
1
We’re thinking at #SIPS2023 about effective science communication on twitter and putting together some tips and need your help! 1/4
1
6
16
Help us brainstorm! #SIPS2023.
We have a poster at #SIPS2023! Beth, Sarah, and Alex Park hand-coded key p-values from hundreds of papers, and now we’re wondering what to do with them. We know they’ll show evidential value, so that’s not the most interesting test. Got other ideas?.@bethclarke_ @SchiavoneSays
0
2
9
Add a title, your name, and any additional comments (for example, the rest of your review report). Select a format and hit the Download button! [6/7]
1
0
3
Next, you can provide a sense of your general judgements of the research on each validity on the Add Overall Ratings page. [5/7]
1
0
4
Visit To begin, click Identify Threats & select any threats that apply to the research you’re evaluating. Say you have concerns about statistical power, you could identify this under the statistical validity tab. Add additional threats as needed [4/7]
1
2
7
I turned this into a (hopefully) easy to use website that creates a downloadable report reviewers (or authors!) can share in reviews, post publication peer review, etc. Here’s an example of how to use [3/7].
1
0
4
We sought expert consensus (through 3 waves of data) to identify common & serious threats to validity (internal, construct, statistical, and external validity) in social science research. [2/7].
1
0
4
If you still think they won’t try to come for these rights next, listen to what the are saying. Read what they are writing.
0
0
3
What another devastating day in America.
BREAKING: The Supreme Court just reversed nearly 50 years of precedent and overturned Roe v. Wade, ending the federal constitutional right to abortion. We won't deny what a horrible moment this is. Half of the states are expected to ban abortion.
0
0
2
SPSP nerds, you have been summoned! Come hangout tonight.
Tonight! 6pm. Standard Deviant Brewing. #SPSP2022.
0
0
2
Check out our new paper comparing peer reviewers’ ratings of published registered reports vs. comparison articles!. TL;DR: Reviewers rated registered reports higher on the rigour of the methods & analysis (& overall quality of the paper). This was a fun project to be a part of!.
Initial evidence of research quality of registered reports compared with the standard publishing model From @cksoderberg, @schiavonesays, @julia_gb, @fsingletonthorn, @siminevazire, @kevinesterling, @briannosek
1
2
13
Seriously how did this get published in @Nature? . Makes me wonder how many women had their submissions rejected from Nature around this time only to see this sexist, condescending, and objectifying fan fiction get accepted.
1
1
17
Tell us about your experiences with data management! #SIPS2021 #datamanagement.
Help us prepare for our Data Management Hackathon happening at @improvingpsych 2021! . Has poor data management ever negatively impacted your research? . 1/5.
1
0
4