News · 2026-09-26

WROP trains video world models to keep track of hidden objects

The WROP project reports that targeted synthetic training can make a video continuation model much better at preserving hidden objects and their identities across short physical scenes. The result matters because many video models produce convincing frames while quietly losing track of what exists when the camera looks away; WROP turns that vague failure into a trainable exam. It is an early improvement to a narrow capability, not evidence that video models have acquired a general persistent world state.

Key facts

• WROP contains 1.5 million generated samples from 150 parameterized Blender generators.
• Each item is a 120-frame 1280×720 clip split into a 60-frame input and 60-frame continuation target.
• In blind human comparison, PWM-WROP ranked first among true-continuation systems and third overall.
• Primary source: “Training Object Permanence in World Models,” arXiv:2609.28654.

Object permanence is the foundational idea that an object still exists when hidden. A video model can fail this in cinematic ways: balls merge behind a screen, an object emerges in the wrong lane, a hidden item stays glued to an old pixel position instead of moving with its container, or a collision makes an object disappear. The finished video may look plausible moment by moment, but it has not represented the scene consistently.

The WROP authors separate that problem from object solidity. Permanence asks whether count, identity, location, and motion survive concealment. Solidity asks whether barriers block, supports cause falling when removed, and collisions preserve objects while changing trajectory. Their six task families make these questions concrete: hidden moving objects, moving occluders, containers that carry or swap hidden contents, apertures, support removal, and collision. This is more useful than asking whether a final frame merely looks realistic.

The training data are hand-authored rather than scraped. The authors made 150 Blender generators—90 for permanence and 60 for solidity—then varied scene structure independently from surface features such as color, lighting, and camera viewpoint. That aims to reduce shortcuts. A model should not learn that a blue ball always exits left; it must follow a specific ball through a specific occlusion. The released dataset card says the corpus is roughly 214 GB uncompressed and licensed CC BY-NC 4.0.

The evaluation uses 300 withheld continuation tasks, two from each generator. A system receives the first half of a clip and a natural-language prompt, then must generate the physical event and consequence. Twenty human raters compare anonymized outputs in random order on prompt alignment, motion plausibility, and whether objects vanish, appear, pass through things, or change count. Image similarity can reward a system that redraws a pre-event scene instead of predicting the fall, re-emergence, or collision that matters.

PWM-WROP is a fine-tune of Cosmos3-Nano, with architecture and tokenizer unchanged. The paper says the trained model is best among systems asked to continue directly from the final input frame and third overall, behind two reference-to-video systems that can regenerate a scene. Qualitatively, it is more likely to preserve three balls through a tunnel, reveal an unchanged arrangement after a screen moves, or follow a cup that carried a hidden object. It is less successful when physical contact has to propagate through a collision.

That unevenness is the key anchor statistic disguised by a leaderboard: the intervention is strongest at remembering what exists, weaker at modeling what contact does. An object can remain indexed in a latent representation without the model learning the correct transfer of momentum. A film editor analogy helps: keeping a continuity sheet prevents a character from teleporting between shots, but it does not teach the editor Newtonian mechanics.

The release is unusually complete. The benchmark and generations, code repository, and PWM-WROP model card are public. The model card describes a BF16 15B-parameter artifact; the dossier does not provide a verified file-size total, so no download size should be guessed. It also does not state a minimum VRAM requirement. The authors' training stack runs on AWS Trainium2 hardware rather than Nvidia GPUs; that is a training-platform description, not a minimum deployment requirement.

The code release includes a native PyTorch implementation and documents Trainium2 engineering. That makes WROP more than a paper claiming a better score: researchers can inspect the generators, see the exam, and compare raw outputs. Still, its license is non-commercial, and the system is not presented as a broadly deployable simulator.

The authors' defensible conclusion is modest: concept-specific synthetic training can improve physical persistence in a continuation model. That is consistent with earlier diagnosis work such as PlayWorld, which found world models losing state during closed-loop interaction, and a newer survey on memory for autoregressive video generation. WROP is not a replacement for memory architecture, long-context state, or action-conditioned planning.

The strongest counterargument is experimental. The comparison mixes architectures and interfaces, so it cannot cleanly prove that WROP alone caused every rank difference. The paper notes wide uncertainty in some family-level solidity comparisons. Reference-to-video systems also enjoy a different problem formulation. And a 60-frame synthetic continuation is far from a robot maintaining a reliable world model for hours.

Even so, WROP offers a valuable research move: write down a failure mode in controllable scenes, make it hard to solve by visual shortcut, and publish the training and test materials together. The headline is not that an AI now understands physics. It is that a measurable reflex—“the hidden object is still there, and the event still has consequences”—can be taught and tested more directly than before.

Primary source, verified: read the paper → (arXiv 2609.28654)

Ground Truth. (2026, September 26). WROP trains video world models to keep track of hidden objects. Ground Truth. https://groundtruth.day/news/wrop-object-permanence-world-models.html

@misc{groundtruth:wrop-object-permanence-world-models,
  title  = {WROP trains video world models to keep track of hidden objects},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/wrop-object-permanence-world-models.html}
}

Topics: research · world-models · video · synthetic-data · open-weights