🧵 An intro to the OxRisk tools. These are a set of simple scalable & free online tools that we've developed and validated for key adverse outcomes in psychiatry & criminal justice.

https://t.co/V9tiqxe9aW

Finally, blanket statements that suicide risk prediction is not possible/accurate is not based on the latest international evidence. Repeating such blanket statements now reflect outdated information and comes across as ideological.

Another key area is more clinical studies. Notable that NICE could not find any studies that compared tools with unstructured (subjective) clinical judgement. But question is whether tools PLUS unstructured approach improve outcomes.

Focus of research should shift more towards external validation of existing models, head-to-head comparisons, and model updating and extension. Important to prioritise methodological rigour. Complete, accurate, and transparent reporting is key.

Most limitations of current research are avoidable issues in study design, conduct, and analysis. Also, many models have no intention for clinical use (unclear target populations and settings, prediction horizon, when to be used). Some impossible to use prospectively.

But field also involves significant research waste. Oversupply of unvalidated prediction models addressing similar research questions. Often developed w/ sub-optimal methods, poorly reported.

Many of the identified models showed good discriminative ability (similar to models in other areas of medicine). Identified 5 models with good discrimination and calibration performance in external validation.

What does it all mean? Our findings show that pessimism about the feasibility of suicide risk prediction is NOT evidence-based.

👉OxMIS: predicts 1-year risk of suicide in individuals with severe mental illness 👉Four models by Simon et al: predict 90-day risk of suicide attempt and death following secondary mental health and general medicine care

We identified 5 models with good discrimination and calibration performance in external validation👇

Most common methodological problems were incomplete assessment of performance (ignoring calibration), insufficient sample sizes, reporting/handling missing data, not adequately accounting for overfitting, optimism in performance estimates.

All development studies, and all but two external validations, were at high risk of bias (using PROBAST tool).

Only 17% of models presented in a way allowing for independent validation/use by others. For remaining models, authors not provide sufficient information to allow predictions in new individuals.

But calibration only assessed for 9% of models in development studies and 31% of external validations (and sometimes assessed incorrectly).

One measure of performance – the C index – was reported widely. Ranged from 0.61-0.97 (median 0.82) in development studies and 0.60-0.86 (median 0.81) in external validations. Comparable to models in other areas of medicine (cardiovascular, respiratory, COVID19).

Included 91 articles (167 unique models and 29 external validations). Only 8% of developed models externally validated, although more validations over the past 5 years.

We have summarised the predictive performance of risk models and critically appraised their methodological quality and risk of bias. Ended up being huge undertaking as so many studies to consider.

MANY prediction models for self-harm/suicide published in recent years, but their role in risk assessment is controversial. NICE advises against use to predict risk of self-harm recurrence. New NHS England suicide prevention guidance reproduces this.

New paper!! Systematic review of prediction models for self-harm and suicide. Largest of its kind. Provides a comprehensive overview of 196 models and external validations.

Risk assessment in domestic violence - a key challenge for criminal justice and its liaison with health and social services. Didn’t get a chance to tweet this recentish paper on development and validation of new tools (OxDoV): https://t.co/oxMpocpoYJ and https://t.co/OXl0pNSGvM