I want to thank our great collaborators from @DKFZ and @emblebi : @alex_w_jung, @kgaurav1208, @s_brunak, @udansk, @ewanbirney, @tomaswfitz and especially my supervisor @MoritzGerstung It was a pleasure working with you!

For more details on code and analysis, check out our paper: https://t.co/dZP1UuvvB7 And github repository: https://t.co/9ccmzChAon

Conclusion. Repurposing transformers for modelling disease progression enables a range of new abilities and insights. We believe it will be very enabling for biomedical research in improving our understanding of disease risks. More work (training and testing) will be needed

There are also some important limitations, however. Delphi-2M learns a range of biases present in UK Biobank’s data. This includes immortality (probands had to be alive to be recruited), but also the fact that not all sources of data (GPs, hospitals, questionnaires) were

Lastly, Delphi-2M, trained on data from the UK Biobank, performs very well on data from Danish registries. This is not trivial, as the cohorts and healthcare systems are very different.

3. Explainability Without any prior knowledge about the disease relationship, Delphi-2M’s internal embeddings cluster related diagnoses of similar types. This includes female genital cancers, different forms of neurodegeneration, or also diabetes and known co-morbidities.

When sampling from birth, Delphi-2M produces purely synthetic health trajectories. It turns out that a model trained on synthetic data has many of Delphi-2 M's properties without ever having seen any real data. This may be very useful as health data are very sensitive, which

Conditional sampling from the age of 60 produces long-term predictions of future health events over a horizon of up to 20 years. There is still *a lot* of chance involved in each case, but the amount of correctly predicted diagnoses is around 30-40% higher than sampling using

2. A key new feature of Delphi-2M is its generative AI nature. When applied autoregressively, Delphi-2M can sample entire health trajectories. This can be from birth or conditioned using data up to a certain age.

Delphi-2M remains consistently accurate across different ICD-10 chapters, sexes and prediction time horizons, allowing for estimating the entire spectrum of disease risks up to decades into the future.

1. Multi-disease predictions. Traditional epidemiological risk models work with one single disease. Delphi-2M models 1256 different ICD-10 coded diseases and death at once. One model to rule them all!

Using transformers instantly gives 3 advantages over traditional disease risk models: 1. Multi-disease predictions 2. Generative 3. Explainability

Delphi-2M is trained on data from 400,000 participants of the UK Biobank and validated on 1.8 million participants from Danish health registries.

Delphi-2M extends the GPT-2 architecture to model health trajectories consisting of ICD-10 diagnoses or other information such as sex or smoking, each paired with the corresponding age. It includes an additional head so it can not only predict what is likely to happen but also

How can transformer models be used for this purpose? LLMs model text as a sequence of words [or tokens]. Delphi-2M models health trajectories as a sequence of diagnoses. The language here is that of ICD-10 diagnostic codes.

Over time, our medical history accumulates, and an important question is how much our past diagnoses influence the future. Often, diseases occur in clusters, so-called co-morbidities. Can this process be modelled?

How much does your medical history reveal about your future? As a team effort of @MoritzGerstung and @ewanbirney's labs, we present Delphi-2M, a transformer for modelling future health based on past medical history. https://t.co/dZP1UuvvB7

Artem Shmatko presenting our joint study with @MoritzGerstung group on AI-assisted prediction of methylation class from H&E sections at #eano2024 @EANOassociation @uniklinik_hd @DKFZ

Alpha-release of our weekend project. Stay tuned for better results and details on how this is generated!

That's what a “just for fun” project looks like when polished to perfection… Great job @LomakinAI!