I'll be at the @genbio_workshop at #ICML2025 today, presenting my poster on Plixer, a generative model for drug-like molecules - given a target protein pocket, Plixer generates a voxel (3D pixel) hypothesis for the binding ligand before decoding it into a valid SMILES string 1/n

Check out the code and preprint here:.

because I believe there is an inductive bias in the convolution operation of a CNN, stemming from the fact that molecular recognition is a phenomenon driven by local interactions.

Much of the recent work on protein-conditioned generative models for small molecules has focused on diffusion approaches that move discrete atoms with coordinates in 3D space, I have a hunch that convolutions in 3D pixel space might be an easier supervised learning task. .

Using these databases, we can cheaply generate training pairs of 3D conformers and their corresponding SMILES string, we can use this to pretrain a voxel-to-smiles decoder, which improves our ability to sample valid and diverse drug-like molecules.

The small data sample of protein-ligand complexes in the PDB is not enough to learn the chemical grammar and valid conformational space of molecules as 3D objects. But we do have access to massive databases of 100s of millions of ligand-only samples in databases such as ZINC.

Generative models for protein-conditioned small molecule generation suffer from a lack of training data with only a few thousand non-redundant experimentally resolved structures in the PDB, this is not enough to cover the diversity of drug-like molecule space 2/n.

RT @dmmiller597: Come and join me, @_judewells, @_barneyhill, @jakublala and others in London on July 31st to hear some great flash talks o….

Nice.

a lotta yall still dont get it.if you take a Lorentzian metric as a section of a bundle of point-wise Lorentz metrics and pull back the positive Weyl spinors with a suitable passage to the maximal compact subgroup you get one generation of Pati–Salam grand unified fermions.

The ProGen authors do a nice job of showing the scaling laws for PLMs, but contrary to trends in NLP where scaling and lower perplexity usually yield better performance in downstream tasks we don't see this in PLMs so what's different about proteins?.

Lack of benefit to scaling PLMs particularly for fitness prediction (eg. proteinGym) is well known, the authors cite this paper:.and similarly it seems to be the case that smaller ESM models often do better on a wide variety of prediction tasks.

they also find that expression levels for both novel proteins (<30% seq id with naturals) and natural-like (>30% seq id) are similar for different model sizes although larger models have slightly better expression for a specific sequence in-filling task.

in addition to zero-shot fitness prediction (blue) the authors also do fine-tuning (orange) where you use some of the measured assay results to tune model preferences. Here there seems to be some evidence in favour of scaling, but it doesn't look conclusive

but what I find more interesting are the findings where scale doesn't seem to help: For the proteinGym benchmark, sequence likelihood is used to predict the fitness score across various assays and proteins, here there seems to be no benefit scaling beyond ~300m params

They also show that the larger models generate sequences that pass filtering criteria (such as not having too many repeated AAs etc.)

the paper shows that if you generate sequences from the model and align the generations with natural proteins your generations will cover around 50% more sequence clusters with a model that is 135x larger (339m -> 46bn).

The progen models are a series of auto-regressive protein language models they train a series of model sizes all the way up to 46bn params, paper is here:.First, we have to acknowledge that they do find some benefits to scaling. .

I really like this ProGen3 paper because, contrary to the title, I think it actually shows there is relatively little to be gained from massively scaling protein language models. 1/n