We released two preprints describing the JASP Meta-Analytic functionality in detail. Meta-Analysis with JASP, Part I: Classical Approaches ( https://t.co/9wj8i5Ddx9) Meta-Analysis with JASP, Part II: Bayesian Approaches ( https://t.co/AX57XiHab2)

Read in full: https://t.co/7HMXwFNrEi

Also, this should not be a reason to stop exercising. 1) There are other benefits of exercise 2) Some populations/exercises show benefit 3) There might be wider effects on cognition; however, the literature is too heterogeneous and contaminated with publication bias to be certain

I think that the field needs to clean up the published literature a bit. Additional small studies are not going to move the needle at this point; maybe a couple of large-scale, pre-registered studies might provide more insight?

We also re-analyzed all of the original meta-analyses individually. Many of them are consistent with publication bias: the evidence for and the degree of the pooled effects decrease once publication bias is adjusted for.

We run subgroup analyses for each outcome/population/intervention. We found that most results are too heterogeneous to tell (see wide prediction intervals), but some interventions seem to be promising and some have substantive evidence against them. See figures for each outcome.

First, we found notable publication bias, especially, in studies on general cognition and executive function. Importantly, there was extreme between-study heterogeneity (tau ~ 0.3-0.6!). This means that the effects are consistent with both large harm and benefit.

We were not the only ones to notice, also see @MatthewBJane commenting on this when the study came out: https://t.co/kYhUYeBNJr So, we manually extracted the study-level data from the included meta-analyses and re-evaluated the evidence.

Previous meta-meta-analysis ( https://t.co/UwyOZ05ler) indicated consistent benefits of exercise for cognitive benefits across all domains and populations. However, it synthesized meta-analytic estimates and, as such, it could not adjust for publication nor evaluate heterogeneity.

We just preprinted a huge meta-meta-analysis examining the effects of exercise on cognition, memory, and executive function. In short: - 2,239 effect sizes - extreme between-study heterogeneity - extensive publication bias - some subgroup/exercise-specific effects More below

See the - preprint: https://t.co/LkILkQitTE - package: https://t.co/kcgzmBKJK9 - benchmark:

We developed PublicationBiasBenchmark R package that can be easily extended with new methods and measures. It also automatically generates a webpage with summary reports. All the raw data, results, and measures are available on OSF.

Our proposal addresses other issues of current simulation studies (incomparability, irreproducibility...). We demonstrate the living synthetic benchmark methodology on the publication bias adjustment literature. See how previous simulations use different methods and measures.

To start the process, we suggest - collecting all published methods and simulations - evaluating all methods on all simulations - publishing this set of results as the initial synthetic benchmark - later research can update this benchmark with new methods and simulations

We want to separate those two steps. New simulations should be published without new methods. Instead, they should evaluate all existing methods. New methods should be published without new simulations. Instead, they should be assessed on all existing simulations.

Simulation studies have a conflict of interest problem. The same team: - develops a new method - designs a simulation study to evaluate it However, the new method has to show good performance to get published. We propose living synthetic benchmarks to address the issue.

We are pleased to have @BartosFra join us today, Tuesday, September 30th, 11am (EST) to talk about Bayesian Hypothesis testing! This is followed by a workshop on using JASP for statistics around 12:10pm. The zoom is open to public with details in the flyer! @PsychPrinceton

My supervisor, a Bayesian statistician and a big #chess fan, wrote down his thoughts about the cheating allegation against @GMHikaru https://t.co/zP85EhG7Vh

btw, we just released JASP 0.95.2, which fixes some previously reported stability issues -- consider updating your version :)

The Bayesian part also provides more guidance on specifying prior distributions for estimation, testing, and model-averaging (with different effect sizes and in different settings).

Each manuscript walks you through three examples describing the applications of different meta-analytic tools, including - effect size calculation - funnel plot - forest plot - bubble plot - simple meta-analysis - meta-regression - multilevel and multivariate models.