AI papers — 2026-05-03

Problem: Consolidating multiple expert capabilities (text, image, video reasoning) into a single LLM via post-training suffers from capability divergence when using mixed-data RLVR, or from poor knowledge absorption when applying sequential on-policy distillation (OPD) after experts have drifted far from the student.

Method: CoPD maintains parallel RLVR training branches—one per capability (e.g., text reasoning, image reasoning, video understanding)—and interleaves branch-specific GRPO updates with cross-branch mutual on-policy distillation (OPD) throughout training, rather than separating them into sequential stages. In alternating RLVR phases each branch deepens its own expertise and widens behavioral distance from peers; in mutual OPD phases each branch generates rollouts on other branches' data and receives token-level KL supervision from the partner, closing that gap while it is still within the "absorbable range" identified via top-k token overlap. All branches start from a shared base model and the final unified model is obtained by simple parameter merging after co-evolution concludes.

Key results:

• CoPD (two-branch, text+image) achieves MMMU 66.94, MMMU-Pro 55.10, MathVista 75.75, MathVision 57.88 on image reasoning; AIME 2025 49.58, HMMT 2025 30.83 on text reasoning—exceeding all baselines including domain-specific single-expert models.
• Surpasses both the Image-Expert and Text-Expert on their respective held-out benchmarks, breaking the conventional student-cannot-surpass-teacher ceiling.
• OPD gain correlates linearly with teacher–student top-k token overlap (r = 0.89, R² = 0.79), validating the behavioral-consistency hypothesis that motivates the method.
• Three-branch (text + image + video) CoPD extends the same gains to video reasoning without additional design changes.

Why it matters / caveats: CoPD provides a practical post-training recipe for building all-in-one multimodal reasoning models without a separate distillation stage, and the parallel-branch co-evolution pattern may inspire a new class of training scaling paradigms. All experiments use a 4B-parameter VLM backbone (Qwen3-VL-4B-Instruct); whether gains hold at much larger scales is not demonstrated.

Problem: Humanoid robots fail to generalize fluent interaction-rich behavior across diverse unstructured environments because jointly modeling spatial context, temporal dynamics, robot actions, and task intent at scale is poorly matched to conventional supervision.

Method: ExoActor uses large-scale pretrained third-person video generation models as a unified interface for interaction dynamics. Given a task description and an initial third-person observation, the pipeline performs (1) a robot-to-human embodiment transfer step that converts the robot scene into a human-like representation preserving geometry and viewpoint, (2) text-conditioned exocentric video synthesis depicting step-wise task execution via a pretrained video generation model, (3) whole-body and hand motion estimation from the generated video using pose-extraction pipelines, and (4) execution of the estimated motion through a general-purpose motion tracking controller. The framework is modular—each stage (video generation, motion estimation, control) can be improved independently.

Key results:

• Not stated (the paper demonstrates feasibility across B-level/easy, A-level/moderate, and S-level/challenging interaction tasks on a real humanoid system; no quantitative benchmarks are reported).

Why it matters / caveats: ExoActor removes the need for task-specific data collection by exploiting pretrained video model priors, suggesting a scalable path to general-purpose humanoid intelligence from generative models alone. Key acknowledged limitations include physically grounded video generation (the videos are geometrically approximate), error accumulation through the multi-stage pipeline, and lack of closed-loop visual feedback during execution.

Problem: Fine-tuning large LLMs on consumer-grade GPU servers is bottlenecked by limited VRAM and slow PCIe interconnects; existing pipeline-parallel (PP) schedules suffer from severe pipeline bubbles because uneven model stages (e.g., a large LM head) are statically bound to specific GPUs.

Method: RoundPipe treats GPUs as a stateless execution-worker pool and applies a round-robin stage dispatch schedule that decouples compute stages from specific GPUs, achieving near-zero-bubble pipeline parallelism with CPU offloading. Three engineering contributions enable this: (1) a priority-aware PCIe transfer scheduling engine that packs parameter transfers into idle windows between critical-path activation transfers; (2) a fine-grained distributed event-based synchronization protocol that serializes asynchronous optimizer updates at the individual layer level without pipeline-stalling barriers; (3) an automated stage-splitting algorithm that partitions layers into asymmetrically sized forward and backward stages to equalize execution time (a forward stage of three layers equals one backward stage under activation recomputation).

Key results:

• 1.48–2.16× speedup over state-of-the-art baselines (including Mobius) when fine-tuning 1.7B to 32B parameter models on an 8× RTX 4090 server.
• Enables LoRA fine-tuning of a Qwen3-235B MoE model with 31K sequence length on a single 8-GPU RTX 4090 server.
• Pipeline bubbles during Llama-3.1-8B training reduced from ~30% (existing PP schedules) to near-zero.
• RTX 4090 costs ~80% less than datacenter A100 GPUs, making the hardware savings significant.

Why it matters / caveats: RoundPipe democratizes fine-tuning of very large MoE models for small labs and individual researchers without datacenter-scale hardware. The method requires CPU offloading; models whose states fit entirely in GPU VRAM will not benefit from the same degree of PCIe-transfer hiding, and PCIe bandwidth remains the fundamental bottleneck for extremely large batch sizes.

Problem: Static agent benchmarks freeze a task distribution at release time and grade mainly final responses, making it impossible to track agents against evolving workflow demand or to verify that tasks were actually executed rather than merely described plausibly.

Method: Claw-Eval-Live separates a refreshable signal layer from a reproducible time-stamped release snapshot. Each release is built from a signal-to-task pipeline over ClawHub Top-500 public workflow-demand signals (covering skills used in practice), clustered and weighted by demand, then expanded into 105 controlled tasks spanning 18 controlled business services (HR, management, finance, multi-system workflows) and sandboxed local workspace repair. Each task is executed in a controlled environment and graded from observable evidence—execution traces, audit logs, service state, and post-run workspace artifacts—using deterministic checks where possible and structured LLM judging only for semantic dimensions. Quarterly refreshes rerun the pipeline to track signal and model evolution.

Key results:

• Best model across 13 evaluated frontier models passes only 66.7% of tasks; no model reaches 70%.
• HR, management, and multi-system business workflows are persistent bottlenecks across all models.
• Local workspace repair is comparatively easier but unsaturated.
• Models with similar pass rates can diverge substantially in overall task completion; task-level discrimination concentrates in a middle band of evaluation tasks.

Why it matters / caveats: The dual grounding—demand-driven task construction and execution-trace verification—addresses two systematic blind spots of static benchmarks simultaneously. The freshness advantage requires active quarterly refreshes; without community or maintainer support the benchmark will degrade toward the same staleness it was designed to avoid.

Problem: RLHF for image editing lacks a robust reward model because existing holistic scorers assign a single overall score without decomposing the instruction's requirements, producing biased and hallucinated feedback that fails to reliably guide post-training.

Method: Edit-R1 builds a chain-of-thought Reasoning Reward Model (Edit-RRM) in two stages: (1) Supervised fine-tuning cold-start—a powerful external VLM acts as a quality-control judge to filter and select high-plausibility CoT reward trajectories (where the RRM decomposes each editing instruction into verifiable principles, evaluates the edited image against each, and aggregates into a fine-grained scalar reward); (2) Group Contrastive Preference Optimization (GCPO), a novel RL algorithm that groups the RRM's own generated trajectories into winner and loser sets and optimizes the pointwise RRM against pairwise human preference data—an alignment task that standard DPO and GRPO cannot perform on reasoning outputs. The trained 7B Edit-RRM is then used as a non-differentiable reward signal in a GRPO loop to post-train downstream image editing models such as FLUX.1-kontext.

Key results:

• 7B Edit-RRM (SFT + GCPO) achieves 82.22% accuracy on the editing reward benchmark, surpassing Seed-1.5-VL and Seed-1.6-VL as specialized editing evaluators.
• Consistent scaling trend from 3B to 7B parameters; each training component (SFT, then GCPO) contributes to accuracy improvement.
• Edit-R1 delivers measurable downstream gains on FLUX.1-kontext across multiple editing categories when used as the GRPO reward signal.

Why it matters / caveats: Edit-R1 is the first CoT-based verifier reward model specialized for image editing that outperforms general-purpose frontier VLMs on editing evaluation, enabling principled RLHF scaling for this domain. The reward benchmark remains in-domain; generalization to out-of-distribution editing task types is not fully characterized.

Problem: Existing length-control mechanisms for autoregressive LLMs and VLMs operate at coarse sequence level (whole-sequence penalties, prompt instructions, pre-decode predictors), lacking the token-level resolution needed for fine-grained performance–efficiency trade-offs during inference.

Method: LenVM attaches a lightweight scalar value head to the final-layer hidden state of a frozen LLM or VLM. It formulates generation-length modeling as a value estimation problem by assigning a constant negative reward r = −1 to each generated token and computing a discounted return V_t = γ^(L−t) ∈ (−1, 0) at each step t, where L is the total response length and γ is the discount factor. The value head is trained with token-wise MSE regression on sampled completions—annotation-free, dense supervision that scales across model size, prompt count, and number of completions per prompt. At inference, LenVM values guide length-controlled decoding via exponential tilting of the next-token distribution to steer toward shorter or longer outputs.

Key results:

• On LIFEBench exact length matching: applying LenVM to a 7B model improves the length score from 30.9 to 64.8, significantly outperforming frontier closed-source models.
• On GSM8K at a 200-token budget: LenVM maintains 63% accuracy vs. 6% for a token-budget baseline—demonstrating meaningful performance preservation under strict efficiency constraints.
• Loss decreases consistently with scale across model size, prompt count, and completions per prompt axes.
• LenVM accurately predicts total generation length from the prompt boundary.

Why it matters / caveats: LenVM provides a lightweight, annotation-free, scalable framework for token-level length modeling that can also serve as a value-function signal in future RL training for efficiency objectives. The value head is trained on completions from the base model's own distribution; models that have undergone RLHF or instruction tuning may exhibit distributional shift that reduces prediction accuracy.

Problem: Efficient open-source omni-modal models that natively support text, image, video, and audio with high inference throughput and long-context reasoning capability remain rare, limiting research reproducibility in this area.

Method: Nemotron 3 Nano Omni is built on the Nemotron 3 Nano 30B-A3B Mixture-of-Experts hybrid LLM backbone, augmented with C-RADIOv4-H vision encoder (dynamic resolution 512×512 to 1840×1840 via variable 16×16 patches with 4× pixel-shuffle downsampling) and Parakeet-TDT-0.6B-v2 FastConformer audio encoder (16 kHz mono, ~8× temporal downsampling, 12.5 tokens/sec, 30-sec clip segmentation). Video inputs use a Conv3D patch embedder that compresses every two frames into one (2× temporal token reduction). Maximum context is 256K tokens. Training uses a multi-stage recipe that progressively introduces modalities and scales context length to mitigate catastrophic forgetting. Model checkpoints are released in BF16, FP8, and FP4 formats alongside training data and code.

Key results:

• 3× higher single-stream output token throughput than Qwen3-Omni on NVIDIA B200.
• 9× higher output token throughput per GPU vs. Qwen3-Omni at a fixed interactivity target.
• 3× higher throughput vs. Nemotron Nano V2 VL at the same interactivity target.
• Top-ranked on OCRBench-V2, MMLongBench-DOC, VoiceBench, WorldSense, and DailyOmni leaderboards.
• Ranked most cost-efficient open video understanding model on MediaPerf.

Why it matters / caveats: Releasing BF16/FP8/FP4 checkpoints plus training data, pipelines, and code enables community research into efficient omni-modal MoE architectures. The throughput benchmarks are measured on NVIDIA B200 hardware; efficiency gains on different GPU architectures may vary substantially.

Problem: Computer-use agents that invoke frontier multimodal models at every GUI interaction step incur per-task inference costs that can exceed one dollar, with wall-clock runtimes in tens of minutes—economically unjustifiable in production—because uniform compute allocation ignores that most steps are routine while failures concentrate at a small fraction of high-risk moments.

Method: The framework runs a small, inexpensive GUI policy by default and event-drives escalation to a stronger frontier model via two lightweight BERT-family classifier monitors trained on logged interaction traces. A Stuck Monitor detects progress stalls (the agent looping or repeating ineffective actions) from recent reasoning-action history windows; a Milestone Monitor identifies semantically meaningful checkpoints where sparse verification is most informative for catching silent semantic drift. Escalation uses hysteresis and bounded recovery budgets to avoid oscillation. Monitor labels are derived from stronger LLMs (Claude Sonnet 4.5, GPT-5-series, Kimi K2.5) supervising short trajectory windows with binary labels (stuck/non-stuck; milestone/not). The framework is plug-and-play and requires no modification to the underlying agent architecture or retraining of the large model.

Key results:

• Comparable task success rate to always-large-model agents on OSWorld and WebArena benchmarks.
• Reduces large-model inference cost by up to 74.6%.
• Reduces wall-clock latency by up to 45.8%.
• Combination of Stuck and Milestone monitors outperforms either alone, and event-driven escalation outperforms fixed-interval checking.

Why it matters / caveats: The modular, deployment-oriented design can be layered on any existing computer-use agent, providing clear operating-point trade-offs via escalation thresholds. Monitor quality depends on the diversity and size of the logged trajectory dataset; novel task distributions or UIs may trigger monitor misfires, requiring periodic recalibration.

Problem: Existing website generation benchmarks assume structured, expert-quality instructions and static execution settings, failing to evaluate whether agents can handle the ambiguous, fragmented, and contradictory inputs that real non-expert users provide.

Method: InteractWeb-Bench defines four persona-driven user agent types grounded in Requirement Engineering defect taxonomies: P-MIN (minimalist abstract descriptions), P-RAM (redundant information), P-INT (divergent/ambiguous expressions), and P-CON (conflicting/contradictory requirements). An interactive execution environment exposes a unified four-action space—Clarify (natural language dialog with the user agent), Implement (code synthesis), Verify (GUI-based visual feedback via npm run dev), and Submit—enabling iterative intent refinement. Tasks are evaluated with a Constraint-Slot-Based Task Completion Rate (TCR), where oracle slots derived from golden queries are checked deterministically against the rendered website state (e.g., functional filter checks, layout validation).

Key results:

• Best frontier MLLM-based agent achieves only 53.66 TCR on the benchmark.
• TCR scores range 20–64% across models and persona types.
• Nearly all models perform better under P-RAM (redundant instructions) and worst under P-CON (contradictory instructions).
• Models exhibit increased hallucination rates with higher performance, remaining trapped in "blind execution" by skipping intent clarification even when ambiguity is explicit.

Why it matters / caveats: InteractWeb-Bench reveals that frontier MLLMs systematically bypass interactive intent clarification, motivating the design of agents with proactive clarification and dynamic reasoning-acting strategies. TCR depends on predefined oracle slots, which may not capture all dimensions of real-world website quality or non-expert user satisfaction.

Problem: It is unknown whether LLMs can be explicitly steered to use a specific formal reasoning type (induction, deduction, abduction) when that conflicts with what is task-appropriate, and whether such conflicts can be mechanistically detected or actively corrected.

Method: The study evaluates open-source and frontier LLMs across four reasoning datasets spanning deduction, induction, and abduction tasks, injecting explicit reasoning-type instructions that deliberately conflict with the task's natural strategy. Mechanistic analysis uses linear probing of hidden states across layers to locate where reasoning type information is encoded. Contrastive Activation Addition (CAA) is then applied at the identified layers to steer model activations toward compliance with the conflicting instruction, using a direction vector derived from contrasting "comply" and "sensible" examples.

Key results:

• LLMs consistently prioritize sensibility over compliance: they favor task-appropriate reasoning even when explicitly instructed to use a conflicting type, across all tested model families.
• Task accuracy is not strictly determined by sensibility; models often maintain high accuracy even when using conflicting patterns, via internalized parametric memory—an effect that increases with model scale.
• Confidence scores drop significantly during conflicting reasoning episodes, indicating internal detectability of the conflict.
• Reasoning type is linearly encoded in middle-to-late transformer layers.
• CAA steering increases reasoning type compliance by up to 29%.

Why it matters / caveats: Establishes that LLM reasoning is strongly anchored to training-data instances but is internally accessible and steerable via activation editing, with direct implications for robustness to adversarial reasoning prompts and for building faithfully controllable reasoning systems. Experiments use closed logical reasoning datasets; generalization to open-ended, real-world reasoning tasks remains untested.

Problem: Diffusion-DPO training is degraded by binary preference labels that collapse multi-dimensional visual preferences (aesthetics, semantic alignment, texture fidelity) into a single winner/loser signal, creating conflicting gradient signals when a winning image is superior in some dimensions but inferior in others.

Method: Semi-DPO is a two-stage framework framing the problem as learning with noisy labels. Stage 1 (Multi-Reward Consensus filtering): a consensus of diverse pre-trained reward models is applied to the Pick-a-Pic V2 dataset; only pairs where all models unanimously agree with the human label are designated as a clean labeled set (~21% of the data). Stage 2 (Iterative self-training): the diffusion model is first trained on the clean set, then used as an implicit timestep-conditional preference classifier to generate pseudo-labels for the noisy unlabeled set—exploiting the hierarchical diffusion process (early timesteps govern global composition, late timesteps govern local texture) to decompose a single conflicting binary label into a series of non-conflicting, timestep-specific pseudo-labels. High-confidence pseudo-labels are selected and the model is retrained on them anchored to a composite clean-set objective to mitigate confirmation bias.

Key results:

• State-of-the-art alignment performance on standard visual preference benchmarks (published at ICLR 2026).
• Only ~21% of Pick-a-Pic V2 passes the multi-reward consensus filter, quantifying the prevalence of conflicting preference patterns.
• Outperforms standard Diffusion-DPO without additional human annotation or explicit external reward models during training.

Why it matters / caveats: Semi-DPO provides a theoretically grounded and annotation-free strategy for improving diffusion model alignment under noisy preference data, directly applicable to any existing preference dataset. The initial clean-set filtering depends on a consensus of existing reward models, which may propagate systematic biases shared across those models.

Problem: LLMs lack instruction-following capabilities for generating structurally constrained, culturally appropriate Arabic poetry across the language's multiple historical styles, formal meters, rhyme schemes, and five major dialect groups.

Method: The authors aggregate and standardize a large Arabic poetry corpus spanning multiple eras, genres, and five linguistic varieties (Modern Standard Arabic plus Gulf, Levantine, North African, and Nile Valley dialects), then construct an instruction fine-tuning (IFT) dataset with four core tasks—generation, continuation, revision, and MCQ-based analysis—covering 54 subtasks. Each subtask uses 3,220 manually crafted instruction templates in MSA and four dialects (multiple paraphrases per template), yielding 1.35M training and 24.8K testing pairs. Four LLMs (Fanar, Allam, Qwen3, LLaMA-3.1) are fine-tuned under two regimes: joint training on a mixed task pool, and curriculum-based training with tasks ordered by increasing difficulty.

Key results:

• Fine-tuned models generate metrically and stylistically constrained poetry validated via automated metrics and native Arabic-speaker human evaluation.
• Curriculum-based training improves structural correctness and training stability over joint training.
• Specific numerical benchmark scores are not reported in the abstract or main sections.

Why it matters / caveats: Provides the most comprehensive instruction-tuning resource for Arabic poetry to date, covering dialectal variation and creative production tasks that prior work neglected. Training data comes entirely from historical corpora; modern creative styles, emerging dialectal forms, and transliterated dialect writing may be underrepresented.

Problem: Scaling visual preference optimization for image and video generative models is bottlenecked by two compounding problems: biased preference datasets with conflicting preference patterns (winners that excel in one dimension but underperform in another), and the inability of standard Diffusion-DPO to learn robustly from such noise.

Method: ViPO contributes on two axes. Algorithmically, Poly-DPO extends the Diffusion-DPO objective with a polynomial correction term that dynamically adjusts per-sample confidence weights during training: on noisy datasets with conflicting signals the polynomial term suppresses unreliable gradients, while on high-quality data it converges to standard DPO (α → 0). Data-wise, the ViPO dataset is a massive-scale curated preference corpus: 1M image pairs at 1024px resolution across five categories and 300K video pairs at 720p+ across three categories, generated using state-of-the-art models (FLUX, Qwen-Image, WanVideo) with systematic prompt diversification and balanced distribution sampling to produce reliable, consistent preference signals.

Key results:

• On noisy Pick-a-Pic V2: Poly-DPO achieves +6.87 (SD1.5) and +2.32 (SDXL) GenEval score gains over Diffusion-DPO.
• On ViPO-Image-1M: comprehensive improvements to SD3.5-Medium across aesthetics, ImageReward, GenEval, HPSv2.1, PickScore, and human quality dimensions.
• When applied to ViPO-Image-1M, Poly-DPO converges to standard DPO (α → 0), mutually validating both dataset quality and algorithmic adaptability.
• Published at ICLR 2026.

Why it matters / caveats: ViPO provides the largest open visual preference dataset to date (1M image + 300K video pairs) and demonstrates that algorithmic robustness and data quality are complementary, not substitutable, requirements for scaling visual preference optimization. The preference labels are generated by state-of-the-art generative models rather than real human annotators at scale, which may embed generation-model-specific aesthetic biases into the dataset.

Problem: White-box optimization-based red-teaming (e.g., GCG/nanoGCG-style attacks) of long-context LLMs for prompt injection and knowledge corruption requires prohibitive GPU memory and computation, making systematic security evaluation inaccessible to academic researchers.

Method: FlashRT is a generic efficiency framework layered on top of existing optimization-based red-teaming methods. It targets two bottlenecks in GCG-style attacks on long-context inputs: (1) backward-pass GPU memory—FlashRT avoids storing gradients and activations for the long non-adversarial context portion by selective activation recomputation, since only the small adversarial segment requires gradient computation; (2) forward-pass runtime—FlashRT applies KV-cache reuse across the many candidate token-perturbation evaluations per iteration, computing the long context's KV cache once and reusing it for all candidates. The framework also extends to black-box methods (TAP, AutoDAN) when the red-teamer has white-box model access.

Key results:

• 2×–7× speedup over nanoGCG baseline (e.g., reducing runtime from ~1 hour to under 10 minutes on NarrativeQA with Llama-3.1-8B).
• 2×–4× GPU memory reduction (e.g., 264.1 GB → 65.7 GB for a 32K-token context).
• Forward-pass time reduced from 2,506s to 749s; backward-pass memory reduced from 168.4 GB to 53.7 GB in the showcased experiment.

Why it matters / caveats: FlashRT makes systematic red-teaming of long-context LLMs accessible on single-node academic GPU clusters, enabling the broader community to evaluate and improve defenses at scale. The paper focuses on efficiency and does not report whether attack success rate (the primary security metric) is maintained at the same level as the baseline; this should be verified before deploying FlashRT as a security audit tool.

Problem: Safety evaluations of LLMs are routinely conducted only on base models, implicitly assuming that safety properties persist through downstream fine-tuning—an empirically untested assumption with serious consequences in high-stakes deployment contexts.

Method: Two-phase empirical study across 100 models total. Phase 1 (ecological analysis): 31 widely deployed open-weight fine-tuned models in medicine and law are compared to their base models on both general-purpose and domain-specific safety benchmarks. Phase 2 (controlled adaptation): instruction-tuned base models are fine-tuned on popular domain datasets while systematically varying engineering choices—full fine-tuning vs. LoRA vs. QLoRA, and base model selection—then re-evaluated on the same benchmark suite to assess whether these choices predict post-fine-tuning safety drift.

Key results:

• Benign domain fine-tuning induces large, heterogeneous, and often contradictory safety changes: models frequently improve on some safety instruments while degrading on others simultaneously.
• Substantial disagreement across safety evaluation benchmarks for the same model—different benchmarks yield conflicting conclusions about the same model's alignment.
• Fine-tuning method (full/LoRA/QLoRA) and base model choice offer limited predictive power over the direction or magnitude of safety drift.
• 100 models analyzed spanning widely deployed fine-tunes and controlled adaptations.

Why it matters / caveats: Directly challenges the governance assumption that base-model safety evaluations predict downstream safety, with strong implications for AI supply chain regulation, the EU AI Act's provisions for general-purpose models, and deployment review practices in medicine and law. The study is limited to two professional domains; other high-stakes domains (e.g., education, social services) may exhibit different drift patterns, and the reliability of existing safety benchmarks is itself called into question by the measurement disagreement finding.

Problem: Motion capture for arbitrary (non-human) skeleton topologies from monocular video is limited by factorized pipelines that use a non-differentiable analytical inverse-kinematics (IK) stage, which cannot resolve bone-axis twist ambiguity or be jointly optimized with the visual encoder.

Method: MoCapAnything V2 is the first fully end-to-end framework for arbitrary-skeleton motion capture where both the Video-to-Pose (V→P) and Pose-to-Rotation (P→R) stages are learned and jointly optimized. The key insight is conditioning the P→R stage on a reference pose–rotation pair from the target asset, which provides an explicit coordinate-system anchor and turns the otherwise ill-posed mapping into a well-constrained conditional prediction. Joint positions are predicted directly from video without mesh intermediates (removing a major noise source). Both stages share a Global-Local Graph-guided Multi-Head Attention (GL-GMHA) module that alternates between local kinematic-chain reasoning (within each limb branch) and global cross-branch coordination, enabling effective modeling of structural constraints and motion dependencies across diverse skeleton topologies.

Key results:

• Rotation error reduced from ~17° to ~10° on the Truebones Zoo benchmark (across seen, rare, and unseen skeletons).
• ~6.54° rotation error on unseen skeletons specifically, demonstrating strong cross-topology generalization.
• ~20× faster inference than mesh-based pipeline predecessors.
• Evaluated on both Truebones Zoo and Objaverse benchmarks.

Why it matters / caveats: End-to-end joint optimization lets gradients from the rotation objective reshape the intermediate pose representation, meaningfully improving accuracy on unseen skeleton topologies—the hardest generalization target for rigged character animation. The method requires a reference pose–rotation pair from the target asset to be available at inference time, which is a reasonable assumption for pre-rigged assets but limits applicability to novel, unrigged characters.

Generated on 2026-05-03 from HuggingFace Daily Papers API. PDFs downloaded from arxiv.org with 3-second inter-request delays.