Learn · Intermediate

Genome language models: predicting DNA the way a chatbot predicts text

A genome language model is a neural network trained to predict the next base in a strand of DNA, using exactly the objective that trains a chatbot: read a sequence, guess what comes next, adjust, repeat across billions of examples. Swap the vocabulary of English words for the four-letter alphabet of A, C, G and T and almost everything else about the recipe carries over. The surprising part is what falls out. A model trained purely to continue DNA sequences learns where genes start and stop, which mutations break a protein, and -- as of 2025 -- enough about viral genome architecture to write new ones that actually work in a laboratory.

The setup

DNA is a string over four symbols. That is a smaller vocabulary than any human language, which sounds like it should make the problem easier. It does not, for two reasons.

The first is length. A sentence is tens of tokens; a paragraph, hundreds. A bacterial genome is millions of bases, a human genome three billion. Worse, the dependencies are genuinely long-range: a regulatory element can control a gene from hundreds of thousands of bases away, which in text terms is like a pronoun whose antecedent appeared two novels ago. Standard transformers scale quadratically with sequence length, so context length is the binding constraint in this field the way it never quite is for text. HyenaDNA attacked this directly with a convolution-based architecture reaching a million bases at single-nucleotide resolution, and much of the subsequent work has continued down the state-space model road for the same reason.

The second is tokenization. Text models compress common word fragments into single tokens. DNA has no equivalent of a word: a shift of one base changes which triplet codes for which amino acid and can destroy the gene entirely. Most genome models therefore work at single-base resolution, giving up the compression that makes text models efficient and paying for it in sequence length.

What the model learns without being told

Nobody labels genes for these models. The training objective is only "predict the next base," and yet the resulting networks turn out to represent structure nobody supervised.

The reason is the same as for text: to predict well, you must model the process that generated the data. A model predicting English has to implicitly represent grammar because grammatical continuations are the likely ones. A model predicting DNA has to implicitly represent the genetic code, because a base that breaks a reading frame makes everything downstream unlikely. Gene boundaries, protein-coding regions and regulatory motifs are all statistical regularities that a good next-base predictor cannot avoid learning.

One consequence is immediately useful. Because the model assigns a probability to every sequence, you can score a mutation by comparing the likelihood of the mutated genome against the original. A mutation the model finds surprising is usually one that breaks something -- which turns a generative model into a variant-effect predictor without any additional training. This works for the same underlying reason that perplexity measures a text model's grasp of language.

From reading to writing

The step that made this a general-audience story was generation. In 2025, researchers at the Arc Institute took their Evo models -- pretrained on more than two million phage genomes -- fine-tuned them on 14,466 genomes from a single viral family, and asked for new ones. Roughly 300 whole genomes were generated, 285 were chemically synthesized and put into living bacteria, and 16 produced functional, infectious viruses.

The hit rate is the informative number. A genome is not prose, where a clumsy sentence still parses. It is a machine specification where most edits produce nothing at all. Sixteen working viruses out of 285 attempts, carrying between 67 and 392 mutations relative to their nearest natural relatives, means the model was not memorising a template and adding noise -- it had learned something about which combinations of parts hold together. One design borrowed a structural protein from a distantly related virus entirely, and it fit.

This is the same trajectory protein modelling took a few years earlier. AlphaFold established that a learned model could predict protein structure from sequence at experimental accuracy; protein language models then showed you could generate novel functional proteins the same way. Genome models are running that arc at the scale of whole organisms' worth of instructions rather than single molecules.

Where it stops

Three limits are worth holding onto.

Scale. Working with a phage is possible partly because a phage genome is about eleven genes. A bacterium is thousands; a plant or animal, far more, with regulatory machinery layered on top. Nothing demonstrated so far implies that designing a bacterium is a matter of turning up the compute.

Verification is wet. Unlike text, you cannot check the output by reading it. Every candidate has to be synthesized and tested in a laboratory, which sets the real iteration speed of the field -- weeks, not seconds -- and means these models are proposal generators rather than end-to-end designers.

Safety controls sit outside the model. Arc's Evo 2 release states that pathogens infecting humans and other complex organisms were excluded from the training corpus, and the phage work used non-pathogenic laboratory strains under containment. That is a control at the training-data layer plus one at the DNA-synthesis layer, not a control on who can download the weights -- which is the live policy argument about open-weight models in this domain. It is also a reminder that when the safety property lives in the corpus, the corpus becomes the thing worth attacking, which is the biological cousin of data poisoning.

The broader lesson generalises past biology. Next-token prediction is not a fact about language. It is a fact about sequences that were produced by some structured process, and it works wherever that description holds -- text, proteins, genomes, and probably several fields that have not tried it yet.

Ground Truth. (2026, August 8). Genome language models: predicting DNA the way a chatbot predicts text. Ground Truth. https://groundtruth.day/learn/genome-language-models.html

@misc{groundtruth:genome-language-models,
  title  = {Genome language models: predicting DNA the way a chatbot predicts text},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/genome-language-models.html}
}

Topics: genomics · biology · sequence-models · science · fundamentals