News · 2026-08-12

Google shipped sign-language-to-text on Pixel, trained on 100,000 hours of signing

Google DeepMind released SL2T, a sign-language-to-text model now powering signing input in Gboard and Live Transcribe, letting Deaf users sign to their phone anywhere they would otherwise type. The model was trained on more than 100,000 hours of data spanning over 50 sign languages, with roughly a quarter of it in American Sign Language. The public rollout begins with ASL to English.

Key facts

• SL2T powers new sign-to-text features in Gboard and Live Transcribe, per Google DeepMind's announcement.
• Trained on over 100,000 hours of data across more than 50 sign languages, about one quarter of it ASL.
• The model translates body landmark coordinates directly to text, skipping the intermediate annotations called glosses that most prior work depends on.
• Google DeepMind estimates more than 70 million Deaf and hard of hearing people use the world's 200-plus sign languages.

The gap this closes

Google DeepMind's framing of the problem is direct. Speech technology has given hearing users automatic translation, dictation, and conversational interfaces that "feel effortless," while "this technological revolution has not reached the world's more than 200 sign languages." Dictation has been standard on phones for over a decade. The equivalent for signers did not exist.

What ships now is that equivalent. A Deaf user can sign to search the web, draft a message or a document, or ask Gemini to run a task. In Live Transcribe, which already turned speech into text on screen, they can now sign a response instead of typing one back. Google DeepMind reports that "according to our testers, signing in ASL is faster, more natural, and more delightful than typing in English," which is the kind of claim that sounds like marketing until you consider that for many signers, written English is a second language.

Why this was hard

Two problems stack. The first is linguistic: sign languages are independent natural languages with their own grammar and lexicon, not spoken languages performed with hands. Google DeepMind is pointed about this, writing that early efforts like sign language gloves were "fundamentally limited" because "sign languages aren't simply 'English on the hands.'" A glove reads finger positions. It cannot read a language.

The second is perceptual. Meaning in sign is carried simultaneously by hands, arms, torso, head, and face, all moving at once, and tracking that at high frame rates is a demanding computer vision task. Speech transcription maps a one-dimensional audio stream to text in the same language. Sign translation requires seeing a whole moving body and then translating between two different languages.

The technical move

SL2T converts the video into a sequence of points on the signer's body, then translates that coordinate sequence directly into text. What it skips is the step nearly all prior work relied on: glosses, the written labels researchers use to annotate individual signs.

Glosses seem helpful and are quietly destructive. Writing a sign as a word forces a continuous, spatial, multi-channel language through a fixed vocabulary of discrete labels, and everything that does not fit gets dropped. Google DeepMind notes that glosses "fail to capture rich, non-linear aspects of sign languages such as non-manual markers and spatial constructions," meaning facial grammar and the use of physical space to track who did what to whom. Translating from landmarks directly removes the artificial vocabulary ceiling, so translation quality scales with data rather than with how many signs someone thought to label.

Training jointly across more than 50 sign languages also turned out to beat training separate single-language models, because the model learns shared structure across them. That is the same effect that made multilingual speech recognition systems outperform per-language ones, and it matters more here: most sign languages have nothing close to the data ASL does, and joint training lets them borrow.

The privacy design deserves a note. Pose extraction runs on the device, and only the landmark coordinates go onward for translation. The video itself is not what gets sent.

The honest caveat

Google DeepMind published its own error list rather than burying it, and it is specific: rare signs, rapid fingerspelling, passive constructions, classifier depictions, and tense when context is thin. The examples given are concrete and instructive. "Prey" comes out as "grey." A classifier depiction loses the word "claws." A sign meaning something started renders as "kicked off." These are not rounding errors; classifier depictions in particular are a core grammatical device, and dropping them changes meaning.

The rollout is also narrow. One language pair, ASL to English, on Pixel. There are no weights, no API, and no checkpoint anyone outside Google can evaluate, which means the quality claims rest entirely on Google's own reporting. For a technology aimed at a community with a long history of being built for rather than built with, the accountability gap is worth naming even while the product is worth celebrating. The core model team is credited publicly, led by researchers including Garrett Tanzer, Manfred Georg, and Chris Dyer, and Google DeepMind says the work continues toward additional sign languages and sign language generation. The first direction is the one to watch: ASL has a quarter of the training data, and the other 200-odd languages are the actual test.

Primary source, verified: read the paper →

Ground Truth. (2026, August 12). Google shipped sign-language-to-text on Pixel, trained on 100,000 hours of signing. Ground Truth. https://groundtruth.day/news/google-shipped-sign-language-to-text-trained-on-100000-hours-of-signing.html

@misc{groundtruth:google-shipped-sign-language-to-text-trained-on-100000-hours-of-signing,
  title  = {Google shipped sign-language-to-text on Pixel, trained on 100,000 hours of signing},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/news/google-shipped-sign-language-to-text-trained-on-100000-hours-of-signing.html}
}

Topics: accessibility · deepmind · translation · computer-vision · product-launch · deaf-community