AI papers — 2026-05-11

Problem: Existing music-driven 3D dance generation and pose-driven animation pipelines cannot be directly combined for music-driven dance video generation because they optimise incompatible objectives, yielding poor motion quality and visual fidelity.

Method: MACE-Dance is a two-stage cascaded Mixture-of-Experts (MoE) framework. The Motion Expert is a diffusion model with a BiMamba-Transformer hybrid backbone trained with Guidance-Free Training (GFT) to produce kinematically plausible 3D motion from music. The Appearance Expert is a DiT fine-tuned with a decoupled Kinematic–Aesthetic two-stage strategy (Body Adapter for kinematic conditioning, then LoRA branches for aesthetic refinement) to synthesise video conditioned on the generated 3D motion and a reference image. Training uses the authors' curated MA-Data corpus (70 k clips, ≈116 hours, 20+ dance genres, combining 3D-rendered FineDance clips and real web video).

Key results:

• SOTA performance on the 3D dance generation sub-task (Motion Expert alone).
• SOTA performance on the pose-driven image animation sub-task (Appearance Expert alone).
• SOTA performance on the full music-driven dance video generation task on the new MA-Data evaluation protocol.
• MA-Data dataset: 70 k clips, 116 hours total, covering 20+ genres.

Why it matters / caveats: MACE-Dance is the first end-to-end framework specifically designed for this task and demonstrates that decomposing motion and appearance into specialised experts is more effective than jointly optimising them. However, numeric FID/FVD comparisons against prior work are not quoted in the abstract or first 12 pages; the claim of SOTA rests on the authors' own evaluation protocol using a proprietary dataset.

Problem: Multi-task alignment of flow matching text-to-image models via GRPO suffers from reward sparsity and gradient interference between heterogeneous objectives, causing a "seesaw effect" where improving one metric degrades another.

Method: Flow-OPD is a post-training alignment framework built on Stable Diffusion 3.5 Medium. Stage 1 trains domain-specialised teacher models via single-reward GRPO fine-tuning, letting each expert reach its ceiling. Stage 2 establishes an initial student policy via a Flow-based Cold-Start scheme, then consolidates expertise through an on-policy sampling → task-routing labelling → dense trajectory-level supervision pipeline. A Manifold Anchor Regularisation (MAR) term from a task-agnostic teacher anchors generation to a high-quality manifold and prevents aesthetic degradation common in purely RL-driven alignment.

Key results:

• GenEval score: 63 → 92 (+29 points absolute) on SD 3.5 Medium.
• OCR accuracy: 59 → 94 (+35 points absolute).
• ~10 points improvement over vanilla GRPO on the combined metric.
• Exhibits an emergent "teacher-surpassing" effect (student exceeds teacher on held-out metrics).

Why it matters / caveats: Establishes on-policy distillation as a scalable alignment paradigm for flow-based generative models; the teacher-surpassing phenomenon suggests knowledge consolidation is a genuine advantage over single-teacher distillation. Evaluated only on SD 3.5 Medium; transfer to larger flow models (e.g., SD 3.5 Large, FLUX) is not demonstrated.

Problem: Group-based RL algorithms for LLM post-training (GRPO and variants) implicitly construct a reward-weighted softmax target distribution on the response simplex but only approximate the projection to it via first-order gradients, obscuring the true optimisation geometry.

Method: Listwise Policy Optimization (LPO) makes the target distribution explicit: it restricts the proximal RL objective to the response simplex to derive the implicit target, then performs exact KL divergence minimisation (rather than a first-order approximation) to project the policy toward that target. The decoupled projection step allows flexible choice of divergence function. LPO guarantees monotonic improvement on the listwise objective with bounded, zero-sum, self-correcting gradients.

Key results:

• Consistently outperforms GRPO and comparable policy-gradient baselines across logical reasoning, mathematics, programming, and multimodal geometry tasks.
• Works across diverse LLM backbones spanning 1.5B–14B parameters.
• Intrinsically preserves optimisation stability and response diversity without additional stabilisation tricks.

Why it matters / caveats: Provides a principled geometric framework that unifies existing group-based RL algorithms and enables systematic improvement; flexibility in divergence choice opens new design dimensions. Absolute accuracy numbers for individual benchmarks are not quoted in the first 12 pages; claimed improvements are relative to the same target.

Problem: Multimodal search agents process entities sequentially, issuing one tool call per entity, which accumulates redundant interaction rounds for queries that naturally decompose into independent sub-retrievals.

Method: HyperEyes trains a parallel multimodal search agent on Qwen3-VL-30B (and -235B) in two stages. Cold-start uses 30 k trajectories from a Parallel-Amenable Data Synthesis Pipeline covering visual multi-entity and textual multi-constraint queries, with efficiency-oriented trajectory selection via Progressive Rejection Sampling. The central RL stage uses Dual-Grained Efficiency-Aware RL: at the macro level, TRACE (Tool-use Reference-Adaptive Cost Efficiency) is a trajectory-level reward whose reference tightens during training to suppress superfluous tool calls; at the micro level, On-Policy Distillation (OPD) injects dense token-level signals from the 235B teacher on failed rollouts. The authors also contribute IMEB, a 300-instance human-curated benchmark that jointly evaluates accuracy and efficiency.

Key results:

• HyperEyes-30B surpasses the strongest open-source multimodal search agent of comparable scale by +9.9% in accuracy with 5.3× fewer tool-call rounds across 6 benchmarks.

Why it matters / caveats: Addresses the underexplored efficiency dimension of agentic search and provides a concrete benchmark (IMEB) for the community. The OPD component requires a 235B teacher model, which may be a barrier for labs without very large compute budgets.

Problem: Test-time scaling strategies (beam search, self-consistency, probing) are hand-crafted heuristics designed by researchers, leaving vast regions of the computation-allocation space unexplored.

Method: AutoTTS shifts the human role from directly designing TTS heuristics to constructing evaluation environments. It formulates width–depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals; controllers decide when to branch, continue, probe, prune, or stop. Beta parameterisation keeps the search space tractable, and fine-grained execution-trace feedback (not just scalar reward) helps the explorer LLM diagnose failures. Because controllers are evaluated via offline replay rather than repeated live LLM calls, each evaluation is cheap.

Key results:

• Discovered controllers improve the accuracy–cost Pareto frontier over strong hand-crafted baselines on AIME25 and other mathematical reasoning benchmarks.
• Strategies generalise to held-out benchmarks and across model scales without re-search.
• Total discovery cost: $39.9 and 160 minutes.

Why it matters / caveats: Very low discovery cost makes the approach accessible; reusability across benchmarks adds practical value. Evaluation is confined to mathematical reasoning; it is unclear whether the discovered controllers generalise to code, science, or dialogue tasks.

Problem: Scaling Diffusion Transformers to hundreds of layers causes a "Mean Mode Screaming" (MMS) collapse in which token representations homogenise silently, suppressing centered variation and eventually crashing training.

Method: The paper provides an exact gradient decomposition into mean-coherent and centered components, showing that when token representations homogenise, attention-logit gradients are suppressed through the null space of the Softmax Jacobian, locking the network in a collapsed state. To prevent this, Mean–Variance Split (MV-Split) Residuals combine a separately gained centered residual update with a leaky trunk-mean replacement, preventing the mean component from dominating while preserving the centered subspace dynamics.

Key results:

• A 400-layer single-stream DiT trained with MV-Split avoids the divergent collapse that crashes the un-stabilised Post-Norm baseline.
• MV-Split tracks close to the baseline's pre-crash trajectory and substantially outperforms token-isotropic gating (LayerScale) throughout the full training schedule.
• A 1000-layer DiT trained with MV-Split remains stably trainable — a scale-validation result at extreme depth.

Why it matters / caveats: Provides the first mechanistic, predictive account of deep DiT collapse, going beyond ad-hoc stabilisation tricks. Generation quality metrics (FID, IS) at scale are not prominently reported; the 1000-layer run is framed as a stability demonstration, not a SOTA generation result.

Problem: Embodied intelligence is data-limited: unlike language or vision, datasets for physical interaction are orders of magnitude smaller, narrowly scoped, and robot-platform-specific.

Method: HumanNet is a one-million-hour human-centric video corpus covering both egocentric and exocentric perspectives, with fine-grained activities, human-object interactions, tool use, and long-horizon behaviours. It provides interaction-centric annotations (captions, motion descriptions, hand/body signals). A systematic curation pipeline treats human-centric filtering, temporal structuring, viewpoint diversity, and annotation enrichment as first-class design principles. Validation uses the LingBot-VLA architecture: continued pretraining of Qwen VLM on 1 k hours of egocentric HumanNet video is compared against 100 hours of real-robot CoBot data.

Key results:

• Dataset scale: 967 K hours, 150 K+ object categories, 720 K+ tasks.
• Qwen VLM pretrained on 1 k hours of HumanNet egocentric video outperforms Qwen VLM pretrained on 100 hours of real-robot CoBot data on embodied VLA validation tasks.
• Competitive with LingBot (which uses 20 k hours of proprietary robot data) in controlled comparison.

Why it matters / caveats: Demonstrates that large-scale human video can be a scalable, cost-effective substitute for expensive robot demonstrations. The VLA validation is a first-step controlled experiment; full embodied benchmark comparisons across diverse robot platforms remain future work.

Problem: Existing code search benchmarks evaluate only the embedding/retrieval stage, ignoring the reranking stage and developer-style short keyword queries, while also suffering from data contamination, label noise, and degenerate binary relevance.

Method: COREB (Code Retrieval and Reranking Benchmark) is built from counterfactually rewritten LiveCodeBench problems in five programming languages (including Go and Ruby, absent from prior benchmarks), released as timed data drops to prevent contamination, with graded relevance judgements. It covers three tasks: text-to-code, code-to-text, and code-to-code retrieval plus a reranking stage. The authors also fine-tune a COREB-Reranker on COREB training data.

Key results:

• Code-specialised embeddings score ~2× over general encoders on code-to-code retrieval.
• Short keyword queries (closest to real developer search) collapse every embedding model to near-zero nDCG@10.
• Off-the-shelf rerankers show a 12-point nDCG swing on code-to-code and no net-positive across all tasks.
• COREB-Reranker is the only evaluated reranker achieving consistent gains across all three tasks.

Why it matters / caveats: Exposes that production code search (keyword queries + reranking) is far harder than current benchmarks suggest; timed releases prevent future leakage. The benchmark is derived from LiveCodeBench, so coverage is biased toward competitive-programming-style problems; generalization to large industrial codebases is unknown.

Problem: Training MLLMs using unimodal data by bridging modalities in a shared contrastive space is hindered by a poorly understood modality gap that existing methods treat as a simple global shift.

Method: AnisoAlign re-characterises the modality gap: modalities already share compatible dominant semantic geometry, but the residual gap is an anisotropic structure concentrated along a small number of dominant directions (not a uniform global offset). Guided by this, AnisoAlign performs bounded anisotropic geometric correction of source-modality representations using the target modality's internal geometric prior, constructing substitute representations that are aligned in distribution while preserving instance-level semantic structure.

Key results:

• Better matches target-modality geometry (local support compatibility, fewer dominant anisotropic residual directions) than prior alignment methods.
• Improves MLLM downstream performance in both fully text-only training and text-only pretraining before visual instruction tuning.
• Specific numeric gains on downstream benchmarks are in later pages not captured in the first 12.

Why it matters / caveats: Recasts the modality gap from an empirical observation into a correctable geometric phenomenon, enabling more principled and effective unimodal-data training for MLLMs. Quantitative benchmark results were not accessible within the 12-page extraction window.

Problem: The visual latent diffusion recipe (Transformer VAE + DiT + flow matching) has not been successfully applied to language generation; prior diffusion language models underperform autoregressive baselines.

Method: TextLDM transfers the visual DiT recipe to text with minimal modification. A Transformer-based VAE encodes discrete tokens to continuous latents; crucially, Representation Alignment (REPA) with a frozen pretrained language model is applied to the latent space during VAE training — reconstruction fidelity alone proves insufficient. A standard DiT backbone then performs flow matching in this latent space, identical in design to its visual counterpart. Trained from scratch on OpenWebText2.

Key results:

• Substantially outperforms prior diffusion language models (LLaDA, LD4LG, COSMOS, Block Diffusion, CALM) on text continuation benchmarks.
• Matches GPT-2 on text generation quality under identical training settings.
• Ablations confirm REPA is the critical component for downstream generation quality.

Why it matters / caveats: Concrete evidence that the visual DiT recipe transfers to language, opening a path toward a single diffusion architecture for multimodal generation and understanding. Currently only matches (not exceeds) GPT-2; the gap to modern autoregressive models at scale is not addressed.

Problem: Sparse outcome-only rewards in multi-turn agentic RL provide insufficient credit assignment for individual steps within long interaction trajectories, while dense supervision methods add tuning complexity and may hurt generalisation.

Method: AEM is a supervision-free credit assignment method that lifts entropy dynamics from the token level to the response level, aligning uncertainty estimation with the effective action granularity of LLM agents. It derives a practical response-level uncertainty proxy from the interaction between the sampled-response advantage and its relative surprisal under natural-gradient updates, then uses this proxy to rescale advantages — naturally guiding the policy from exploration (high uncertainty) to exploitation (low uncertainty) without any additional reward model or auxiliary signal.

Key results:

• +8.8% improvement on ALFWorld with GRPO + Qwen2.5-1.5B.
• +1.4% improvement applied to DeepSWE on SWE-bench-Verified.
• Consistent gains across ALFWorld, WebShop, and SWE-bench-Verified on models from 1.5B to 32B.

Why it matters / caveats: Supervision-free and architecture-agnostic, making it a drop-in improvement for group-based RL baselines across diverse agentic tasks. The +1.4% gain on SWE-bench-Verified (the hardest benchmark) is modest, suggesting entropy modulation helps less when the policy is already near-optimal.

Problem: VLMs fail at dynamic spatial reasoning from monocular video because text-only chain-of-thought is verbose and imprecise for complex 3D+time dynamics, and external geometric modules add inference cost without fostering intrinsic capability.

Method: 4DThinker enables VLMs to reason internally in continuous 4D latent space. An annotation-free data generation pipeline synthesises 4D reasoning data from raw videos. Dynamic-Imagery Fine-Tuning (DIFT) jointly supervises textual tokens and 4D latents, grounding the model in dynamic visual semantics. 4D Reinforcement Learning (4DRL) then tackles complex compound-motion reasoning via outcome-based rewards, restricting policy gradients to text tokens to ensure stable optimisation while the 4D latent representation remains fixed.

Key results:

• Consistently outperforms strong baselines including GPT-5 and Gemini-2.5-Pro on multiple dynamic spatial reasoning benchmarks (including tasks requiring decomposition of ego-motion and object motion).
• Annotation-free data pipeline removes the need for labelled 4D data.

Why it matters / caveats: First framework to enable VLMs to "think in 4D" via internal latent simulation rather than verbalization; annotation-free pipeline is a practical advantage. Specific numeric accuracy gaps versus baselines are in later pages not extracted here.

Problem: Existing long-context prefill acceleration (sparse attention) only works for full-attention models and is incompatible with continuous batching, preventing deployment in modern inference engines across hybrid architectures.

Method: UniPrefill performs block-wise dynamic sparsification directly at the token computation level, making it applicable to any architecture: full attention (LLaMA-3.1-8B), linear/full attention hybrids (Qwen3-Next-80B-A3B), and sliding window/full attention hybrids (Gemma-3-12B). It is implemented as a continuous batching operator deeply integrated into vLLM's scheduler with native tensor-parallel and prefill-decode co-processing support.

Key results:

• Up to 2.1× speedup in Time-To-First-Token (TTFT).
• Speedup increases with the number of concurrent requests.
• Demonstrated on LLaMA-3.1-8B, Qwen3-Next-80B-A3B, and Gemma-3-12B across context lengths 4K–128K.

Why it matters / caveats: Architecture universality and vLLM compatibility make this immediately deployable in production inference stacks without model changes. The 2.1× figure is the peak; average speedup at typical batch sizes (BS=1–4) is likely lower and architecture-dependent.

Problem: Theoretical analysis of recurrent model state tracking focuses on expressivity (can a model represent the transition rules?) while ignoring error control (will errors in the hidden state accumulate over time?).

Method: The paper proves that affine recurrent networks — a class encompassing SSMs (Mamba, Mamba-3, AUSSM) and Linear Attention — cannot correct hidden-state drift along state-separating subspaces once they preserve state representations exactly. It characterises a readability threshold: tracking collapses when the ratio of within-class spread to initial between-class separation crosses a predictable Tcross value. Non-affine (state-dependent return-map) architectures can selectively contract symbolic-subspace drift and escape this obstruction.

Key results:

• Tcross derived from the distinguishability ratio accurately predicts the horizon at which downstream task accuracy collapses across Mamba, Mamba-3, AUSSM, Simple AUSSM, Negative Mamba, Linear RNN, and Token-gated RNN.
• At training length 60, affine models that survive curriculum length still fail to extrapolate to longer sequences.
• Models with ≥90% test accuracy at training length collapse below that threshold at Tcross.

Why it matters / caveats: Provides a quantitative, predictive theory of when recurrent architectures fail at state tracking — relevant for long-context language modelling with SSM-based models. Evaluated exclusively on synthetic group state-tracking tasks; empirical implications for real NLP tasks require further study.

Problem: The DeepSeek Sparse Attention (DSA) indexer uses many query heads (e.g., 64 on DeepSeek-V3.2) for token scoring, making the indexer — not the attention itself — the dominant cost at long context lengths.

Method: MISA replaces the multi-head DSA indexer with a Mixture-of-Experts (MoE) router: cheap block-level statistics select h ≪ HI active heads, which then run the full token-level scoring — reducing per-query cost from O(HI·L) to O(h·L + HI·M) where M = ⌈L/B⌉ ≪ L. A hierarchical variant (MISA†) uses the MoE-routed pass to produce an enlarged candidate set, then re-ranks with the original DSA indexer for near-exact top-k recovery. Implemented as a TileLang kernel with no additional training required.

Key results:

• With h=8 active heads: matches DSA on LongBench within 0.5 average points across DeepSeek-V3.2 and GLM-5.
• Uses 8× fewer indexer heads on DeepSeek-V3.2 and 4× fewer on GLM-5.
• Recovers >92% of the tokens selected by DSA per layer (LSHT).
• Preserves fully green Needle-in-a-Haystack heatmaps up to 128K context.
• 3.82× kernel speedup over DSA's original indexer kernel on a single H200 GPU.

Why it matters / caveats: Drop-in replacement requiring no retraining; directly applicable to DeepSeek-V3.2 and GLM-5 in production. Evaluated on two production models only; compatibility with DeepSeek-V4's Compressed Sparse Attention (CSA) is not demonstrated.

Problem: Long video generation suffers from semantic drift and narrative collapse over extended horizons because existing diffusion models lack an explicit mechanism for enforcing cross-segment consistency.

Method: A²RD (Agentic Auto-Regressive Diffusion) synthesises video segment-by-segment through a closed-loop Retrieve–Synthesize–Refine–Update cycle. Three core components: (i) Multimodal Video Memory that tracks video progression across visual, textual, and structural modalities; (ii) Adaptive Segment Generation that switches among generation modes (continuation, transition, new scene) for natural progression; and (iii) Hierarchical Test-Time Self-Improvement that self-improves each segment at both frame and video levels to prevent error propagation. Built on Veo 3.1 as the base generator. The authors also introduce LVbench-C, a benchmark requiring non-linear entity and environment transitions to stress-test long-horizon consistency.

Key results:

• Outperforms SOTA baselines by up to 30% in consistency and 20% in narrative coherence on LVbench-C and public benchmarks spanning 1–10 minute videos.
• Human evaluations confirm improvements in motion and transition smoothness.
• Achieves SOTA in just two self-improvement iterations.

Why it matters / caveats: First agentic framework that decouples creative synthesis from consistency enforcement for long video; LVbench-C provides a harder, more realistic evaluation setting for the community. Built on proprietary Veo 3.1; full reproducibility is limited for the broader research community.

Problem: Evaluating AI agent security at scale requires realistic, controllable, and reproducible environments, but no such platform exists — leaving vulnerability assessment ad hoc and non-systematic.

Method: DTAP spans 14 real-world domains (CRM, healthcare, finance, legal, coding, etc.) with 50+ simulation environments replicating Google Workspace, PayPal, Slack, and others. DTAP-RED is an autonomous red-teaming agent that explores injection vectors (prompt, tool, skill, environment, and combinations) and discovers effective attack strategies. DTAP-BENCH is the resulting large-scale benchmark with verifiable, rule-based judges that automatically validate attack outcomes. Evaluated on OpenClaw, Claude Code, Google ADK, and OpenAI Agents SDK with backbones including GPT-5.5, Gemini-3-Pro, Claude-Sonnet-4.5, and DeepSeek-V4-Pro.

Key results:

• Google ADK is the most vulnerable under the indirect threat model: 55.7% Attack Success Rate (ASR).
• OpenClaw (DeepSeek-V4-Pro) shows the highest ASR under the direct threat model: 59.6% ASR.
• Even the most robust agent, Claude Code, still reaches 25.2%+ ASR, indicating security remains a major challenge across all tested frameworks.

Why it matters / caveats: First systematic, large-scale, reproducible red-teaming platform for AI agents; reveals universal vulnerability patterns across all major agentic frameworks. The simulation environments may not fully capture the complexity of real-world deployments, and the benchmark is biased toward known injection vector categories.

Generated automatically from arXiv PDFs on 2026-05-11. Text extracted from first 12 pages of each PDF using PyMuPDF.