Skip to main content
Lab Grimoire
TW EN
Coffee
LLM Internals

Biological sequence and structure models: reading proteins and DNA like text

Biological sequence and structure models take an amino-acid or DNA sequence and return a predicted structure, a mutation-effect score, or a sequence embedding. How ESM-2, AlphaFold, and Enformer work, how to read pLDDT confidence, and why a prediction narrows down what to test without replacing the experiment.

Author
CY
Published
Updated
AI from Principles to Practice · Part 9 of 70
Biological sequence and structure models: reading proteins and DNA like text

A sequence goes in; a structure, a score, or a vector comes out

A biological sequence and structure model takes a protein's amino-acid sequence or a stretch of DNA and returns one of three things: a predicted 3D structure, a score for how much a mutation or variant changes something, or a list of numbers that represents the whole sequence. AlphaFold predicts protein structure, ESM-2 is a protein language model, and Enformer reads DNA to predict gene expression.

On a map of AI model types, these models sit close to large language models. The idea is the same; only the alphabet has changed.

Summary: Biological sequence and structure models

How they learn: what fits at this position

A protein is a chain built from 20 kinds of amino acids, so written out it looks like text in a 20-letter alphabet. A protein language model is trained by hiding some of the letters and asking the model to fill them back in from the surrounding sequence. Rives and colleagues trained such a model on about 250 million protein sequences and found that its internal representations carried information about secondary and tertiary structure, even though it never saw a structure during training. ESM-2 scaled the same approach up, and Lin and colleagues added a structure module on top to build ESMFold, which predicts a structure from a single sequence.

It is the same logic a language model uses when it learns to predict words from their context. Evolution has discarded sequences that do not work, so the proteins that survive are a record of which combinations hold together, and the model learns from that record.

Structure prediction: read the confidence score with the shape

AlphaFold takes a protein sequence, together with an alignment of related sequences pulled from databases, and outputs predicted coordinates for every atom. It also gives each residue a pLDDT score from 0 to 100. According to EMBL-EBI's guidance, a score above 90 usually means both the backbone and the side chains are reliable, and above 70 usually means the backbone is right. Stretches below 50 may be intrinsically disordered, with no fixed shape at all, or the model may simply lack the information to place them. pLDDT is a local measure: a high score on two domains does not tell you how those domains sit relative to each other. Before drawing conclusions from a predicted structure, check the pLDDT region by region and decide which parts can carry weight.

DNA models: how a sequence or a variant changes regulation

DNA models read genomic sequence and predict what it does in different cell types, such as how strongly nearby genes are expressed. Enformer can draw on sequence up to about 100,000 bases away. Change one base, predict again, and the difference is an estimate of that variant's effect on expression. The AlphaGenome Atlas we covered earlier belongs to the same family.

What they are good for: choosing what to test first

The practical value is narrowing the field before you go to the bench. If you have dozens of candidate mutations and capacity to test only a handful, effect predictions let you rank them and start with the ones most likely to matter. If the protein you study has no experimentally solved structure, a predicted one gives you a working hypothesis, for example whether a mutation sits in the buried core or on the surface.

Where they fail or get over-read

A prediction is an estimate, not a measurement. A predicted structure is one shape, while proteins in cells change conformation depending on what they bind. EMBL-EBI also notes that some disordered regions fold only when they bind a partner, and AlphaFold sometimes predicts them in the folded state with high confidence. Protein complexes and conformational changes fall outside what a single-chain prediction answers. The training data is also uneven: well-studied protein families are better represented, and the AlphaFold paper reports that accuracy drops when few related sequences are available. A DNA variant-effect score is likewise a way to set priorities, not a clinical interpretation.

How they differ from their neighbours on the map

The mechanism is the same as in large language models; only the alphabet and the training data differ. There is overlap with embedding models as well: the internal representation of a protein language model can serve as a sequence embedding, useful for finding proteins with similar function whose sequences look quite different. If your question is a choice among a few fixed options, that is a job for a decision model. Reach for this family when you have a sequence and want to know its structure or the effect of a change. It narrows down what to test; the experiment still has to be done.

References

  1. 10.1038/s41586-021-03819-2
  2. 10.1073/pnas.2016239118
  3. 10.1126/science.ade2574
  4. 10.1038/s41592-021-01252-x
  5. https://www.ebi.ac.uk/training/online/courses/alphafold/inputs-and-outputs/evaluating-alphafolds-predicted-structures-using-confidence-scores/plddt-understanding-local-confidence/

Frequently Asked Questions

Can an AlphaFold-predicted structure be used as if it were an experimental structure?

No. A predicted structure is one shape the model estimates from the sequence and an alignment of related sequences, and its reliability varies along the chain, so check the pLDDT score region by region first. It works well as a working hypothesis when no experimental structure exists, for example to judge whether a mutation sits in the buried core or on the surface. How the protein looks when bound to a partner, how a complex assembles, and how the shape changes still need experimental confirmation.

Does a low pLDDT score mean that part of the prediction is wrong?

Not necessarily. According to EMBL-EBI, stretches scoring below 50 usually fall into one of two cases: the region is intrinsically disordered and has no fixed shape, or the model lacks the information to place it confidently. In the first case the low score reflects the protein itself. A high score has limits too: pLDDT is local, so it says nothing reliable about how two domains are positioned relative to each other, and some regions that fold only when bound to a partner can be predicted as folded with a high score.

Found this useful? RSS · X · Buy me a coffee