News · 2026-07-30

Asked to sit in a chair it can see, the best AI model misses five times out of seven

Nine frontier vision-language models were asked to find a piece of furniture, move a simulated body to it, and sit down. The best of them completed 16.8% of episodes. The benchmark, called HumanCLAW, deliberately removes low-level walking control from the problem so that only decision-making is being tested - and its conclusion is that current models "can describe the world, but they do not reliably track the body they now control."

Key facts

• 1,218 egocentric episodes across 41 indoor houses, each a find-navigate-interact task.
• Best full success: 16.8%. Stage by stage, finding the target ranges from 32.6% to 64.9%, navigating to it from 0.8% to 42.4%, and completing the interaction from 0% to 16.8%.
• Among episodes that reached the seat, 71% still failed to sit, and 81% of those failures were body-awareness errors - dominantly "sitting into thin air."
• Primary source: HumanCLAW: Can Vision-Language Models Act Through a Body?, arXiv, submitted July 29, 2026.

The design choice that makes this benchmark interesting is what it takes away. Embodied AI evaluations usually conflate two problems: can the model decide what to do, and can the underlying controller execute it. Fail either and you fail the task, which makes results hard to interpret and easy to blame on the hardware.

HumanCLAW factors out the second one. The model issues atomic skill commands, and the simulator converts each into a half-second chunk of motion with gravity and collisions applied, while low-level locomotion control is handled beneath it. Whatever these models are failing at, it is not walking.

The headline diagnosis is that it is not seeing either. The authors measured how often the target object was genuinely visible in the model's egocentric view and compared it to how often the model reported seeing it. For the strongest model those rates are within five percentage points of each other. The model looks at the chair. The model knows it is a chair. The model then fails to arrange its body to sit in it.

The failure statistics are where the paper gets specific enough to be uncomfortable. Of the episodes that got a body to the seat, 71% still failed to complete the sit, and 81% of those failures were errors about the model's own body rather than about the scene - most commonly sitting into empty space next to the furniture. The paper's term for the missing capacity is embodied self-awareness: knowing where you are, whether you have arrived, and whether you have collided with something.

The staged scoring rule is a quiet methodological contribution worth noting. A stage counts only if the model both achieves it geometrically and acknowledges that it is complete. That closes a loophole most benchmarks leave open, where a model wanders into the right position and gets credit for a plan it never had. It tests whether the model knows what happened, not whether the right thing happened.

This lands in a week where the same field looked considerably healthier from other angles. Google DeepMind announced Gemini Robotics 2 with whole-body humanoid control, and a separate paper put real-time robot control on a consumer graphics card. Those results are about policies trained on robot demonstration data. HumanCLAW is testing something different: whether a general vision-language model, the kind that reads documents and answers questions about images, can be dropped into a body and act. The answer is that it cannot yet, and the reason is not the one most people would guess.

It fits a run of results establishing that visual competence and physical competence come apart. Ground Truth has covered a benchmark where humans score 96% and the best model gets one in ten, and separate work showing vision-language-action models forgetting basic capabilities after robot fine-tuning. The common thread is that fluency about the physical world, which these models genuinely have, is not the same as a working model of one's own position within it.

The caveat is that simulation is not the world, and the authors say so. Removing locomotion control makes the experiment cleaner but also easier than reality, where a real body has to actually balance. It is possible some of the body-awareness failures are artefacts of how the simulator represents the body to the model rather than deep limitations. What is not in doubt is the comparison the benchmark actually licenses: given identical perception, identical scene understanding and a simulator handling the hard motor work, these models still mostly cannot sit down.

Primary source, verified: read the paper → (arXiv 2607.27180)

Ground Truth. (2026, July 30). Asked to sit in a chair it can see, the best AI model misses five times out of seven. Ground Truth. https://groundtruth.day/news/the-best-vision-model-sits-into-thin-air-five-times-out-of-seven.html

@misc{groundtruth:the-best-vision-model-sits-into-thin-air-five-times-out-of-seven,
  title  = {Asked to sit in a chair it can see, the best AI model misses five times out of seven},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/the-best-vision-model-sits-into-thin-air-five-times-out-of-seven.html}
}

Topics: robotics · benchmarks · vision-language-models · embodied-ai · evaluation · limitations