AI papers — 2026-06-15

Problem: Enable precise camera motion cloning from reference videos to generate multi-shot videos without requiring paired training data that shares identical camera movements across different scenes.

Method: OmniDirector introduces a "camera grid" representation that renders extracted camera parameters as movement within an empty 3D scene with grid lines, providing a visual-semantic encoding of camera motion. The model is a Multi-Modal Diffusion Transformer trained on 1.8M internet videos paired with their corresponding camera grids via token concatenation. At inference, a hierarchical Prompt Expansion Agent decomposes camera descriptions into inter-shot and intra-shot components, fusing them with reference images via a multimodal LLM. An Adaptive Classifier-Free Guidance strategy injects camera grid features during high-noise diffusion stages while refining visual detail in low-noise stages.

Key results:

• Relative Rotation Error (RRE): 2.64° vs. 4.11° for CamCloneMaster
• Translation Precision (T-Pre): 72.74% vs. 52.21% (+39.3%)
• Shot transition temporal accuracy: 96.52% vs. 2.20%
• Frame-level leakage rate: 0.51% vs. 1.60%
• GSB user study: 91.10% Good+Same rate across all metrics

Why it matters / caveats: This work enables director-level camera control at scale without scarce cross-paired training data; the camera grid elegantly decouples motion from scene content. Direct token concatenation may struggle with significantly longer videos, requiring future investigation of temporal memory mechanisms.

Problem: Most agentic RL methods assign credit at coarse tool-call or workflow boundaries, missing fine-grained procedural decisions that significantly influence agent reasoning outcomes.

Method: APPO identifies high-impact decision points using a Branching Score that combines token entropy with policy-induced likelihood gains (Ω term). The method branches from these procedural positions rather than fixed boundaries, and applies dual-group advantage estimation with a future-aware advantage term (A^fut) that emphasizes decisions with greater downstream influence on final outcomes. This provides fine-grained intermediate supervision without requiring additional reward models.

Key results:

• Improves over ARPO baseline by ~4 points across 13 benchmarks
• AIME24: 30.0% vs. 23.3%; MATH500: 69.4% vs. 64.6%
• 2WikiMultihopQA: 79.8% vs. 75.5%
• GAIA (Qwen3-8B): 42.7% vs. 38.8%
• Maintains comparable tool-call efficiency while improving pass@K

Why it matters / caveats: Addresses the fundamental challenge of sparse outcome-level feedback in agentic RL by targeting procedural-level credit assignment. The method introduces additional hyperparameters (branching count B, rollouts N, discount γ) requiring careful tuning per task type.

Problem: LLM agents struggle with long-horizon reasoning over interaction histories because current memory systems use static "retrieve-then-reason" pipelines that cannot adapt memory access based on intermediate evidence discovered during inference.

Method: MRAgent combines a Cue–Tag–Content associative memory graph with an active reconstruction mechanism. The architecture organizes memory into episodic, semantic, and topic layers where tags serve as semantic bridges connecting fine-grained cues to memory contents. During retrieval, the LLM iteratively selects traversal actions (forward/reverse graph navigation), receives candidate nodes, and routes information based on accumulated evidence—integrating reasoning directly into multi-step memory access through three phases: LLM action selection, controlled graph traversal, and LLM-guided routing with state updates.

Key results:

• LoCoMo benchmark: Overall Judge score 68.31 → 84.21 under Gemini (23% relative gain); +12.4% with Claude
• LongMemEval: 32% relative improvement over strongest baseline (72.95 vs. 53.01)
• Token consumption: 118k tokens vs. 245k–3,268k for baselines
• Multi-hop queries: 30%+ recall improvement across successive reasoning turns

Why it matters / caveats: Active retrieval is theoretically proven strictly more powerful than passive retrieval. Limitations include increased latency for deep exploration queries and monotonic memory growth without update/forgetting mechanisms, constraining deployment in long-lived systems.

Problem: This survey characterizes how LLMs are transforming from conversational chatbots into persistent autonomous systems capable of reasoning, environmental interaction, and end-to-end task completion.

Method: This is a comprehensive survey organizing LLM evolution along two dimensions: (1) cognitive core advancement from fast-response chatbots (next-token prediction, parametric knowledge) to Thinking LLMs leveraging long Chain-of-Thought, inference-time scaling, and RL training recipes such as GRPO and DAPO; (2) tool-augmented execution from episodic agent interactions to persistent workspace systems with reusable skills, state management, and verification loops, characterized as the "Workspace + Skill" paradigm.

Key results:

• Inference-time scaling: 1B models can exceed 405B performance on math through computational investment
• Long reasoning chains sustain coherent derivations across hundreds of steps
• Memory systems have evolved with reported 3.7× efficiency gains in persistent multimodal accumulation
• Agent perception advanced from specialist-model orchestration to direct VLM integration via screenshots

Why it matters / caveats: This framework clarifies why stronger models alone are insufficient—workspace abstractions, skill libraries, and auditable verification systems are necessary infrastructure. Significant challenges remain: long-horizon reliability degrades as errors accumulate, safety governance becomes critical when outputs are executable actions, and persistent state management beyond context windows is unsolved.

Problem: Existing omnimodal agent systems fail to coordinate diverse modalities (text, image, audio, video) effectively because native omnimodal LLMs have limited reasoning while existing orchestration frameworks lack scalability.

Method: Orchestra-o1 decouples high-level orchestration from specialized execution: a main agent induces dependency graphs over sub-tasks, selects appropriate backends via skill-matching, and assigns tool subsets (perception tools: image/audio/video analysis; action tools: web search, code execution). Independent sub-tasks run asynchronously with formalized latency guarantees. Decision-aligned GRPO (DA-GRPO) trains open-source 8B orchestrators by rewarding format correctness, action validity, tool reasonableness, and decision quality jointly.

Key results:

• OmniGAIA with GPT-5: 72.8% (+10.3% over Gemini-3-Pro, +32.8% over AOrchestra)
• Open-source 8B: 30.0% (+9.2% over previous best OmniAtlas-Qwen3-30B at 20.8%)
• Lower cost than AOrchestra while achieving higher accuracy
• Ablation: ReAct-GPT-5 improves from 53.9% to 72.8% with Orchestra-o1 scaffold; DA-GRPO outperforms vanilla GRPO and SFT

Why it matters / caveats: Orchestration offers information-theoretic advantages—specialized sub-agents retaining modality-specific evidence outperform monolithic models. Success depends on effective task decomposition and evidence aggregation rather than just larger backbones.

Problem: Agent performance depends critically on the runtime harness (prompts, tools, memory, control flow), yet harnesses remain hand-crafted and static with no mechanism for experience-driven improvement.

Method: HarnessX introduces three components: (1) a typed processor abstraction organized in a nine-dimensional taxonomy enabling modular composition via hook points; (2) AEGIS, a trace-driven evolution engine mapping harness adaptation to RL constructs (states=configurations, actions=edits, rewards=verifier scores) through four stages (Digester, Planner, Evolver, Critic) with deterministic gating against reward hacking and catastrophic forgetting; (3) harness-model co-evolution jointly optimizing the harness via discrete edits and the model via cross-harness GRPO over a shared replay buffer.

Key results:

• Average +14.5% absolute gain across 15 model–benchmark configurations; up to +44.0%
• Inverse-scaling: Qwen3.5-9B on ALFWorld +44.0%; Sonnet 4.6 +11.2%
• Variant isolation prevents stagnation: +13.6% non-degrading over 15 rounds on GAIA
• Co-evolution adds +4.7% over harness-only evolution

Why it matters / caveats: Demonstrates that agent progress need not depend solely on model scaling—evolving runtime interfaces from execution feedback is a complementary lever. Weaker models hit a scaffolding ceiling, motivating co-evolution; the added complexity and computational cost of joint optimization may limit adoption.

Problem: Long-video question answering requires determining which modality and temporal granularity to retrieve from, but no existing benchmark disentangles retrieval quality from generation performance to study this.

Method: CARVE operates in two stages: (1) Parallel Candidate Pooling runs four retrievers in parallel across modality-granularity configurations (visual/textual × frame/clip level), contributing top-k chunks to a shared pool with configuration tags preserved; (2) Chunk-Adaptive Reranking uses a multimodal cross-encoder to rescore each candidate under only its retrieving configuration, selecting the highest-scoring configuration per chunk to create "modality-interleaved" evidence. The accompanying V-RAGBench contains 2,100 curated query-evidence-answer triplets from egocentric videos.

Key results:

• Retrieval Recall@5: 0.603; nDCG@5: 0.433 (best baseline VideoRAG-A: 0.510 Recall@5)
• Generation pass rates: Qwen3-VL-8B (35.7%), Qwen3-VL-32B (36.7%), Gemma-4-26B (32.0%)
• All four modality-granularity configurations selected with non-trivial frequency, confirming genuine diversity
• Efficiency: 4.6s per query, faster than visual-only configurations

Why it matters / caveats: Introduces the first benchmark preventing retrieval errors from being masked by generation shortcuts, revealing that no single configuration is uniformly optimal across video content. This per-chunk granularity insight was impossible to detect with prior benchmarks.

Problem: Current automated audio-visual QA pipelines produce narratively incoherent descriptions with severed audio-visual associations and weak cross-modal temporal reasoning.

Method: A two-stage data synthesis engine: (1) "Entity-Anchored Video Scripting" transforms videos into structured scripts with a summary, main entity list (ensuring consistent references across segments), and timestamped segment descriptions linking speech to speakers; (2) "Clue-Guided QA Generation" performs global clue mining to extract multimodal evidence chains, then locally-focused generation using these clues to create QA pairs with long-term temporal spans and deep cross-modal dependencies.

Key results:

• VITA-1.5: +20.59% on OmniVideo-Test (40.99% → 61.58%)
• Qwen2.5-Omni: +17.82% (42.77% → 60.59%)
• Qwen3-Omni: +13.86% (49.70% → 63.56%)
• Cross-benchmark: Daily-Omni +7.43%, JointAVBench +12.64%
• Single-modality ablations show 7–28% performance drops, confirming cross-modal synergy requirement

Why it matters / caveats: Demonstrates that structured intermediate representations significantly enhance cross-modal understanding in open-source multimodal models. Coarse descriptions of non-speech sounds introduce noise, causing 4.21% performance drops on sound-heavy benchmarks.

Problem: The technological pathways and bottlenecks for AI progress beyond human-level AGI toward artificial superintelligence (ASI) are poorly characterized.

Method: This technical report provides a theoretical framework—not an empirical paper—analyzing ASI through: (1) formalization using the Legg-Hutter intelligence measure and the AIXI universal agent framework; (2) mapping of four largely independent potential pathways (scaling compute/models/data, algorithmic paradigm shifts, recursive self-improvement, multi-agent coordination); and (3) identification of eight major bottlenecks (data wall, resource constraints, paradigm sufficiency, research difficulty, abstraction barriers, and others) with countervailing factors.

Key results:

• Effective compute estimated to grow ~10× annually from combined hardware investment and algorithmic efficiency gains
• Four pathways identified as largely independent and potentially parallel routes to ASI
• Eight major bottlenecks catalogued; their relative impact remains an open quantitative question

Why it matters / caveats: Provides conceptual scaffolding for forecasting post-AGI development when "uncertainty margins rapidly explode" under exponential dynamics. Systematic analysis of competing pathways helps structure research priorities, though fundamental unpredictability persists for paradigm shifts and recursive improvement dynamics.

Problem: Standard GRPO rollouts lack sufficient diversity for effective RL exploration because repeated sampling from a single model produces correlated trajectories, yet token-level noise injection degrades reasoning coherence.

Method: S2L-PO (Small-to-Large Policy Optimization) modifies GRPO rollout generation by mixing candidates from a frozen smaller model with those from the trainable larger model. A progressive annealing schedule linearly transitions the sampling ratio from small-model-dominated early training toward pure on-policy GRPO, preserving temporally correlated, structured exploration signals while avoiding distribution mismatch. The method requires only changes to the rollout process, remaining compatible with standard GRPO implementations.

Key results:

• AIME24: +8.8% accuracy (1.7B→8B configuration)
• AIME25: +10.4% improvement
• MATH-500: +4.2% improvement
• OlympiadBench: +1.7% to +3.0% gains across configurations
• CommonsenseQA (out-of-domain): 63.9% → 67.8% for Qwen3-8B-Base
• Achieves improvements with reduced rollout compute requirements

Why it matters / caveats: Identifies policy-level diversity from parameter compression as qualitatively different from token-level randomness, preserving logical coherence while providing better gradient signals. Evaluation is restricted primarily to mathematical reasoning; generalization to broader domains with non-verifiable rewards remains unexplored.

Problem: Execution traces of AI agents expose proprietary procedural skills—formulas, thresholds, and strategies—allowing unauthorized recovery without access to model weights.

Method: RedAct constructs CapTraceBench with 75 long-horizon tasks and 154 curated skills across seven domains to measure capability leakage risk. The framework localizes and redacts protected information from traces while maintaining verifier-critical audit evidence, and embeds behavioral watermarks for downstream provenance tracking and detection.

Key results:

• Reduces normalized skill transfer from 44.7–67.1% on raw traces to below the no-skill baseline
• Behavioral watermarks achieve 93.6–100.0% true detection at ≤1.9% false alarm rate
• Audit evidence preservation maintained throughout the redaction process

Why it matters / caveats: Frames agent execution traces as security interfaces requiring protection, addressing a critical gap between enabling transparency/accountability and preventing procedural capability theft—a concern growing in importance as agents become more autonomous and commercially valuable.

Problem: LLMs achieve high scores on medical exams but fail to maintain correct medical judgment when misleading context is injected, exposing a critical gap between benchmark performance and real-world reliability.

Method: MedMisBench constructs 10,932 medical questions paired with 48,889 misleading context-option pairs across five source datasets (MedQA, MedMCQA, MedXpertQA, MedJourney, HLE). It employs a 5×3 taxonomy: five content-corruption types (relationship inversion, threshold corruption, cue remapping, spurious anchoring, exception poisoning) crossed with three provenance framings (neutral statements, patient claims, authority sources). Two delivery protocols test resilience: Type 1 presents focused false claims; Type 2 provides full bundles with both correct and incorrect supporting evidence. Clinician validation involved 14 reviewers across 7 countries.

Key results:

• Clean accuracy averages 71.1%, dropping to 38.0% under Type 1 delivery (51.5% attack success rate)
• Type 2 ASR substantially lower at 18.7%
• Authority-framed claims achieve 69.5% ASR; exception-poisoning reaches 64.1%
• 38.2% of responses rated as "worst-case outputs" with material harm potential by clinicians
• Defensive prompts reduce Type 1 ASR by 10–14 points but leave substantial gaps

Why it matters / caveats: Reveals a structural blind spot in LLM medical evaluation: models may know medicine but not preserve correct judgment under adversarial context—critical for real-world deployment where retrieved documents and patient claims shape interactions. Benchmark uses synthetic injections rather than naturally occurring clinical misinformation.

Problem: Text-only repository exploration by coding agents is token-intensive and misses structural relationships; this paper investigates whether visual dependency graph representations can improve efficiency and effectiveness.

Method: SeeRepo augments coding agents with visual renderings of repository dependency graphs alongside standard text-based code access. The system constructs directed heterogeneous graphs capturing four relation types (contains, imports, invokes, inherits) via AST-based static analysis. Query-centered subgraphs are rendered as PNG images using Graphviz with layered left-to-right hierarchical layout; nodes display entity types via semantic icons and the queried node is visually highlighted. These visual renderings are concatenated with text tokens and processed jointly by multimodal foundation models.

Key results:

• Vision-only interaction drops Pass@1 by 13.6–34.1 points while increasing costs by up to 268%
• Multimodal integration reduces input tokens by up to 25% and cost by up to 46% while maintaining or improving accuracy
• Graph-based layout achieves −26% cost with +0.4 Pass@1 gain
• Visualization most effective during fault localization; invoking at repair/validation stages degrades accuracy by up to 5.0 points
• Benefits generalize to SWE-Rebench and SWE-QA external benchmarks

Why it matters / caveats: Demonstrates a practical hybrid-modality pathway for coding agents where structural visual representations enable more efficient repository exploration. Findings are limited to Python repositories; visual scalability for very large codebases remains unexplored.

Problem: LLM inference always executes the fixed forward pass through all layers, ignoring that many inputs could be solved with fewer or differently ordered layer operations.

Method: PoLar predicts input-specific execution programs that determine how frozen transformer layers are skipped or recurrently applied. A lightweight prediction network uses cross-attention over input tokens and layer embeddings followed by a transformer encoder over the layer dimension, with two heads outputting binary segmentation boundaries and operation labels (skip/keep/repeat) per segment. Valid programs are discovered offline via Monte Carlo Tree Search with segment lengths bounded to 4 layers; beam search generates candidate programs at inference time.

Key results:

• DART-Math: +20.8% accuracy gain at pass@5 for LLaMA-3.2-3B
• 75.5% of correctly-solved inputs admit shorter valid execution programs than standard inference
• Combining skip and repeat operations yields substantially larger gains than either alone
• Consistent improvements on ASDiv, MAWPS, and MMLU-Pro out-of-distribution benchmarks
• Inference overhead only 3.05ms (0.8% of forward pass); predictor adds ~0.01–0.06% parameters

Why it matters / caveats: Reveals that fixed-depth execution captures only a narrow subset of LLM reasoning capacity, suggesting substantial gains from test-time computation reallocation without parameter updates. The approach requires offline MCTS-discovered supervision and is evaluated primarily on mathematical reasoning; broader domain generalization is unclear.

Problem: Text-to-image diffusion models operating in VAE latent space do not leverage the rich multimodal perceptual priors available in pretrained MLLMs.

Method: RepFusion repurposes a frozen multimodal LLM as a noisy representation encoder by feeding evolving noisy Representation Autoencoder (RAE) latents through an MLP projector into the MLLM. The MLLM outputs condition a diffusion transformer via Adaptive Layer Normalization (AdaLN-Single), allowing the conditioning signal to change across denoising steps. Only the MLP projector and diffusion transformer are fine-tuned; the pretrained MLLM backbone remains frozen.

Key results:

• 30% relative GenEval improvement (+0.16 absolute) vs. VAEs, outperforming TextEmbed (+21%) and Transfusion (+11%)
• Reaches 0.87 GenEval score after SFT, matching state-of-the-art
• GenEval++ improves from 0.289 to 0.707 after SFT
• Outperforms baselines using comparable inference budgets while fine-tuning only 1.3B parameters vs. their 8B denoisers

Why it matters / caveats: Demonstrates that frozen pretrained multimodal priors transfer effectively to denoising when the representation space is compatible, enabling efficient parameter allocation. Preserving multimodal pretraining outperforms joint optimization, suggesting pretrained priors are robust enough without task-specific retraining.

Problem: Building a complete robot learning pipeline that spans data collection, model design, training, and real-world deployment for generalist robots capable of cross-embodiment manipulation.

Method: HyVLA-0.5 combines: (1) a Mixture-of-Transformers embodied VLM backbone with modality-adaptive computation and native-resolution image encoding; (2) a flow-matching action expert generating continuous action chunks in an embodiment-agnostic delta-chunk representation; (3) a compact memory encoder using factorized temporal-spatial attention for multi-frame observations; (4) FlowPRO, a critic-free offline RL algorithm based on Proximalized Preference Optimization that converts failure trajectories into policy improvements via contrastive losses; and (5) an asynchronous deployment pipeline with cubic Bézier smoothing for C¹-continuous chunk stitching.

Key results:

• RoboTwin 2.0 simulation: 90.9% clean / 90.1% randomized (vs. JoyAI-RA: 90.5%/89.3%)
• Real-world Track A: 93–99% success on four dexterous tasks (Dobot X-Trainer)
• Real-world Track B (cross-embodiment): Successful deployment to JAKA K1 and Astribot S1 using only UMI pre-training, no target-robot teleoperation
• FlowPRO post-training: 99%±0.6% vs. 95%±1.5% for π0.6* on Bottle task

Why it matters / caveats: Demonstrates a full systems approach combining UMI data collection, embodiment-agnostic representations, and failure-driven RL for practical cross-embodiment deployment. Evaluation focuses on specific manipulation robots; broader zero-shot generalization remains an open question.

Problem: Existing video generation benchmarks overlook long-term memory—the critical capability for world models to maintain consistent internal state across extended temporal horizons.

Method: MBench decomposes memory into three hierarchical dimensions: (1) Entity Consistency (object/human identity), (2) Environment Consistency (spatial layout and rendering), and (3) Causal Consistency (physical/semantic rule adherence). Evaluation uses a Trigger-Conditioned Scoring mechanism that first verifies memory challenges are executed via VLM, then computes consistency only on triggered samples. Metrics include SSIM and DINOv2 for geometry/texture, epipolar geometry and DA3 reprojection for spatial, CIELAB lighting and Gram-matrix style for rendering, and VLM/OpenCLIP for causal metrics. The benchmark covers 1,040+ cases across 14 models.

Key results:

• Entity geometry consistency shows strongest human alignment (ρ=0.97 for text-conditioned)
• Action-conditioned models: high spatial score (HY-WorldPlay: 83.86 epipolar) but low state evolution (49.50)
• All 14 tested models show significant temporal drift in long-horizon generation
• Spatial metric correlation: ρ=0.16 (text-conditioned) vs. ρ=0.94 (action-conditioned)

Why it matters / caveats: Establishes standardized memory diagnostics for video world models—essential for robotics, autonomous driving, and interactive applications. The dramatic variation in spatial metric correlation across settings highlights remaining evaluation challenges.

Problem: Standard GRPO training for GUI grounding suffers from reward degeneracy where rollouts from a single screenshot view often all succeed or all fail, eliminating useful learning signal.

Method: VISTA constructs GRPO comparison groups from multiple target-preserving cropped views of the same GUI instance rather than repeated completions on one fixed screenshot. Each crop fully contains the target element with exact coordinate remapping to maintain semantic consistency. A self-verified cross-view anchor—activated only when the model has already produced a maximum-reward rollout—stabilizes training without unconditional SFT; it uses a model-only baseline excluded from group statistics.

Key results:

• ScreenSpot-Pro: +7.9 pts (Qwen3-VL-4B), +13.1 pts (8B), +13.3 pts (30B-A3B)
• Qwen3-VL accuracy: 55.5/52.7/53.7 → 63.4/65.8/67.0 at three scales
• Cross-backbone: +2.0, +0.9, +1.2 pts on Qwen3.5-initialized models at 4B/9B/35B
• Informative training groups increase from <5% to ~20%
• Prediction flip rate: 8.31% → 5.80% on crop perturbations

Why it matters / caveats: Addresses a fundamental RL training bottleneck for pixel-precise coordinate prediction by introducing geometric diversity during training. The approach is broadly applicable to coordinate-based tasks but designed specifically for actionable GUI interactions; adaptation may be needed for mixed instruction types.

Generated automatically from HuggingFace daily papers API and arXiv HTML on 2026-06-15.