Artem Shmatko
@artem_shmatko
Followers
89
Following
79
Media
12
Statuses
33
Computational histopathology & generative models at @MoritzGerstung group
Heidelberg, Germany
Joined November 2020
RT @sahm_lab: Artem Shmatko presenting our joint study with @MoritzGerstung group on AI-assisted prediction of methylation class from H&E s….
0
5
0
The idea of Delphi-2M emerged out of brainstorming how to generalise disease risk models developed by @alex_w_jung. It’s perhaps swerves from my original PhD project in digital pathology, but I couldn’t be happier with how it turned out. I’m extremely grateful to my supervisor,.
0
0
5
I want to thank our great collaborators from @DKFZ and @emblebi:.@alex_w_jung, @kgaurav1208, @s_brunak, @udansk, @ewanbirney, @tomaswfitz.It was a pleasure working with you!.
1
0
4
For more details on code and analysis, check out our preprint:. And github repository:. Special thanks to @karpathy for creating nanoGPT, a simple and hackable transformer implementation that was also the starting point for Delphi!.
medrxiv.org
Decision-making in healthcare relies on the ability to understand patients’ past and current health state to predict, and ultimately change, their future course. Artificial intelligence (AI) methods...
1
0
2
Conclusion. Repurposing GPTs for modelling disease progression enables a range of new abilities and insights. We believe it will be very enabling for biomedical research in improving our understanding of disease risks. More work (training and testing) will be needed for it to.
1
0
2
There are also some important limitations, however. Delphi-2M learns a range of biases underlying UK Biobank’s data. This includes immortality (probands were alive when they were recruited), but also the fact that not all sources of data (GPs, hospitals, questionnaires) were.
1
0
2
Lastly, it can’t be stressed enough that Delphi-2M, as trained on data from the UK Biobank, validates very well on data from Danish registries. This is not trivial, as the cohorts and healthcare systems are very different. It’s not perfect, but it's very encouraging.
1
0
3
3. Explainability. Without any prior knowledge about the disease relationship, Delphi-2M’s internal embeddings cluster related diagnoses of similar types. This includes female genital cancers, different forms of neurodegeneration, or also diabetes and known co-morbidities.
1
0
2
When sampling from birth, Delphi-2M produces health trajectories that are purely synthetical. It turns out that a model trained on synthetic data has many of Delphi-2 M's properties without ever having seen any real data. This may be very useful as health data are very.
1
0
1
Conditional sampling from the age of 60 produces long-term predictions of future health events over a horizon of up to 20 years. There is still *a lot of chance* involved in each case, but the amount of correctly predicted diagnoses is around 30-40% higher than sampling using
1
0
1
2. A key new feature of Delphi-2M is its generative AI nature. When applied autoregressively, Delphi-2M can sample entire health trajectories. This can be from birth or conditional using data up to a certain age.
1
0
2
Delphi-2M beats the age-sex baseline for nearly every disease with more than 20 occurrences (a low bar admittedly, but shows that previous events are influential). Delphi-2M is also as good as, or better than, a range of disease-specific models such as Charleson or Framingham.
2
0
2
1. Multi-disease predictions. Traditional epidemiological risk models work with one single disease. Delphi-2M models 1256 different ICD-10 coded diseases and death at once. One model to rule them all!. Shown are just 8 for illustration.
1
0
1
Using transformers instantly gives 3 advantages over traditional disease risk models:. 1. Multi-disease predictions.2. Generative capabilities.3. Explainability.
1
0
2
Delphi-2M is trained on data from 400,000 participants of the UK Biobank and validated on 1.8 million participants from Danish health registries.
1
0
2
Delphi-2M extends the GPT-2 architecture to model health trajectories consisting of ICD-10 diagnoses or other information such as sex or smoking, each paired with age. It includes an additional head so it can not only predict *what* is likely to happen but also *when*.
1
0
2
How can transformer models be used for this purpose?. LLMs model text as a sequence of words [or tokens]. Delphi-2M models health trajectories as a sequence of diagnoses. The language here is that of ICD-10 diagnostic codes.
1
0
2
Throughout time, our medical history populates, and an important question is how much our past diagnoses influence the future. Often, diseases occur in clusters, so-called co-morbidities. Can this process be modelled?
1
0
3
How much does your medical history reveal about your future?. As a team effort of @MoritzGerstung and @ewanbirney's labs, we present Delphi-2M, a transformer for modelling future health based on past medical history.
1
20
52
Alpha-release of our weekend project. Stay tuned for better results and details on how this is generated!.
0
0
8