AI papers — 2026-05-27

Problem: Existing vision-language grounding models serialize bounding boxes as sequential tokens, creating inference throughput bottlenecks and degrading geometric coherence.

Method: LocateAnything introduces Parallel Box Decoding (PBD), which treats each bounding box as an indivisible atomic block decoded in a single step using block-causal and intra-block bidirectional attention, built on a Moon-ViT + Qwen2.5 backbone. A dual-formulation training jointly optimizes standard next-token prediction (NTP) and block-wise multi-token prediction (MTP). Three inference modes (Slow/NTP, Fast/MTP, Hybrid) allow dynamic fallback when token confidence drops below 0.7. Training uses LocateAnything-Data: 138M natural language queries over 12M images spanning COCO detection, GUI grounding (ScreenSpot-Pro), REC, text localization, and document layout tasks.

Key results:

• Hybrid mode throughput: 12.7 boxes/sec vs. 5.0 for Rex-Omni and 1.1 for Qwen3-VL (>10× faster)
• COCO mean F1: 54.7 vs. Rex-Omni 52.9
• LVIS mean F1: 50.7 vs. Rex-Omni 46.9
• ScreenSpot-Pro: 60.3 mean F1
• DocLayNet: 76.8 mean F1; M6Doc: 70.1 mean F1
• HumanRef REC: 78.7 mean F1; RefCOCOg test: 77.6 mean F1

Why it matters / caveats: PBD resolves the fundamental mismatch between 2D bounding box structure and 1D token streams, enabling production-grade real-time grounding in embodied and GUI systems. The hybrid fallback mode shows that parallelism and accuracy are not mutually exclusive. Coverage is currently limited to detection and grounding tasks; dense open-vocabulary segmentation is not addressed.

Problem: Existing evaluations of spatial foundation models cover a narrow slice of paradigms and domains, making it impossible to identify whether any single model is robust across the full diversity of real-world 3D conditions.

Method: SpatialBench evaluates 41 model variants across 6 paradigms (feed-forward, optimization-based, streaming, SLAM-based, chunk-based, test-time training) on 546 scenes from 19 datasets spanning indoor/outdoor, static/dynamic, normal/egocentric/wrist viewpoints, and real/synthetic data. Four frame-density regimes (Single, Sparse, Medium, Dense) are defined using a weighted set-cover algorithm maximizing voxel coverage or view overlap. The benchmark evaluates camera pose estimation (RRA/RTA), trajectory estimation (ATE, RPEt, RPEr), depth estimation (AbsRel, RMSE), and dense reconstruction (F-score). To close the egocentric/wrist-view domain gap, the authors also introduce DA-Next, a 41-layer transformer extending DA3 with metric scale prediction trained on the new DA-Next-5M dataset (5.5M frames, 22K scenes) on 4 NVIDIA H200 GPUs for 7 days.

Key results:

• DA-Next Sparse depth AbsRel: 0.050 vs. DA3-Giant 0.095 (47% improvement)
• DA-Next Medium depth AbsRel: 0.035 vs. DA3-Giant 0.086 (59% improvement)
• DA-Next AUC@30 gains: +3.1% (Sparse), +5.5% (Medium) over prior best
• Dense TTT methods: TTT3R +95% AUC@30, LoGeR +72% ATE reduction on dense regime
• Full-context models (VGGT, DA3-Large/Giant) run out of memory on dense inputs; streaming variants (StreamVGGT, CUT3R) remain feasible

Why it matters / caveats: The benchmark reveals that no single paradigm dominates all conditions — full-context attention is most accurate at fixed sequence lengths, but bounded-memory streaming is necessary for long sequences. Egocentric and wrist-view domains remain the largest out-of-distribution gaps, and closing them requires targeted data curation rather than simple scaling.

Problem: Existing mobile GUI agent evaluation platforms based on Android emulators are either unable to verify task outcomes deterministically or too resource-heavy to support scalable parallel reinforcement learning training.

Method: MobileGym is a browser-hosted simulation platform that represents environment state as structured JSON, enabling deterministic outcome verification, full state serialization/forking, and side-effect detection without relying on emulators. Navigation logic is formalized as Extended Finite State Machines (EFSM). The MobileGym-Bench comprises 416 parameterized task templates (256 test, 160 train) across 28 applications, with an AnswerSheet form-submission protocol replacing brittle string matching. Model Qwen3-VL-4B-Instruct was fine-tuned via GRPO using 96 parallel simulation instances on 3 RTX Pro 6000 GPUs, and evaluated on a real Redmi Note 12 Turbo device for sim-to-real transfer measurement.

Key results:

• Per-instance resource usage: ~400 MB RAM, ~50 MB disk, ~3 s cold start (vs. AndroidWorld 4.5 GB RAM, 78 s startup)
• 256 parallel instances consume <10% CPU and ~100 GB RAM; full benchmark runs in ~6 minutes
• Top proprietary model: Gemini 3.1 Pro 58.8% SR; best open-source specialist: AutoGLM-Phone-9B 20.0% SR
• GRPO fine-tuning of Qwen3-VL-4B-Instruct: 9.4% → 22.2% SR on simulation (+12.8 pp)
• Real-device transfer: 95.1% of simulation-side training gain transferred to physical device

Why it matters / caveats: MobileGym makes large-scale online RL training for mobile agents practical on a single server, and strong sim-to-real transfer (95.1%) validates the platform as a credible training substrate. Current coverage is limited to apps reproducible without proprietary backends, excluding real banking or social-media states.

Problem: Feed-forward 3D reconstruction models degrade sharply under real-world imaging degradations such as motion blur because single-view restoration pipelines ignore multi-view geometry and VAE-based approaches introduce information bottlenecks.

Method: GARD inserts a DiT^DH diffusion-based denoiser at layer 18 of Depth Anything 3's 40-layer ViT encoder (hidden dim 1536), operating directly in the geometry-aware feature space rather than pixel or compressed latent space. The denoiser uses global cross-view attention for multi-view feature aggregation. Training combines an interpolated flow matching loss (using degraded features as the source distribution with noise perturbation α=0.3) and an attention alignment loss (λ_attn=1.0) that regularizes denoiser attention maps to geometric correspondences. Refined features pass through the remaining encoder layers and feed both geometry and RGB auxiliary decoders.

Key results:

• Pose AUC30: HiRoom 67.22, ETH3D 74.68, DTU 92.37, 7Scenes 84.73, ScanNet++ 87.45 (outperforms single-view and VAE baselines across all five)
• F-Score reconstruction: ETH3D 45.79, 7Scenes 36.08, ScanNet++ 35.77
• PSNR image restoration: HiRoom 21.89 dB; LPIPS 7Scenes 0.249

Why it matters / caveats: Operating in the geometry-aware feature space of a pretrained reconstruction model allows the denoiser to exploit cross-view correspondences that pixel-space methods cannot access. The approach is validated only under motion-blur degradation on the Depth Anything 3 benchmark; generalization to other degradation types (rain, haze, low-light) is not demonstrated.

Problem: Diffusion-based large language models (D-LLMs) lack effective safety monitors that exploit their multi-step denoising trajectories, leaving a key signal unused compared to autoregressive approaches.

Method: D²-Monitor is a cascade architecture consisting of a linear probe base monitor that computes step-wise decision margins |d_s| across denoising steps, and a hesitation signal n_τ counting steps where the margin falls below threshold τ. When n_τ exceeds routing threshold λ, an advanced MLP or TimeAttn probe processes only the hesitation windows for refined classification; otherwise the base probe outputs a majority-vote decision. Training uses a three-stage pipeline: out-of-fold scoring to identify hesitation trajectories, separate base and advanced probe training, and test-time cascade routing. The method is evaluated on WildguardMix, ToxicChat, and OpenAI-Moderation datasets across four D-LLMs including LLaDA-8B-Instruct and LLaDA-2.0-mini-16B.

Key results:

• LLaDA-8B-Instruct on WildguardMix: D²-TimeAttn 89.4 F1 with ≤0.54M parameters vs. full TimeAttn baseline 88.7 F1 with 1.59M parameters
• LLaDA-2.0-mini-16B: 83.0 F1 (0.26M params) vs. LSTM baseline 80.9 F1 (1.51M params)
• Cross-dataset (trained on WildguardMix, tested on ToxicChat): LLaDA-8B-Instruct 74.4 F1 at 0.85M expected parameters
• Margin-based routing produces 13.5% accuracy gap between base and advanced probes vs. 7.2% for entropy-based routing

Why it matters / caveats: The hesitation signal provides a principled difficulty proxy for routing computation in D-LLMs, achieving better safety F1 with far fewer parameters than sequence-based baselines. The approach is configurable for resource-constrained deployment. The work is limited to text-only D-LLMs; multimodal diffusion models are not addressed.

Problem: Existing video generation benchmarks (VBench, EvalCrafter) focus on 5–10 second clips, leaving minute-scale audio-visual generation across text, image, and video conditioning modalities without a unified evaluation framework.

Method: LongAV-Compass is a benchmark of 284 test cases (128 T2AV, 115 I2AV, 41 V2AV) organized by scenario (Vlog, Content-Creator, Performance Ads, Brand Ads) and complexity level L1–L4. Evaluation spans 20+ dimensions using MLLM-assisted scoring (Gemini) for event fulfillment, visual quality, long-form continuity, transition stability, holistic presentation, and text-video alignment (CLIP), plus audio metrics (synchronization, quality, coherence). Task-specific I2AV metrics use DINO-v2 for identity, ArcFace for face similarity, and ImageBind for audio-visual alignment. The benchmark covers 11 systems including Seedance 2.0, Kling 3.0, and Veo 3.1.

Key results:

• Proprietary models maintain stable performance L1→L4 (70.6 to 73.9) while open-source systems degrade (57.9 to 51.4)
• Event fulfillment drops from 59.4 to 52.0 (open-source) vs. 76.0 to 74.4 (proprietary) as event chain length increases beyond 4
• Human alignment correlation: 0.917 (content fidelity), 0.935 (visual quality), 0.867 (long-video stability) on 40-case pilot
• Seedance 2.0 leads overall in I2AV and V2AV; Kling 3.0 leads T2AV event fulfillment (0.9274)
• Veo 3.1 scores 71.8 on I2AV vs. only 57.4 on V2AV, showing large input-format sensitivity

Why it matters / caveats: This is the first benchmark to unify T2AV, I2AV, and V2AV evaluation for minute-scale generation, revealing that proprietary models maintain coherence over longer sequences while open-source models degrade substantially. The benchmark is limited to 284 test cases and relies heavily on MLLM judges whose own biases may affect scores.

Problem: Current video generation benchmarks assess only prompt-following correctness ("rightness") and cannot evaluate nuanced cinematic quality ("goodness") across aesthetic, physical, and production dimensions.

Method: EvalVerse organizes evaluation into a pipeline-aware taxonomy of 7 cinematic aspects, 18 dimensions, and 45 sub-dimensions aligned with pre-production, production, and post-production filmmaking stages. A "Real-to-Gen" data engine transforms professional videos into test cases using Gemini 3.1 Pro for prompt synthesis. Expert calibration proceeds in two fine-tuning stages: first preference alignment via Bradley-Terry ranking loss on pairwise comparisons, then score calibration on pointwise expert annotations with Chain-of-Thought reasoning. Deterministic perception operators (DINO for identity, InsightFace for face recognition, YOLO for semantic anchoring, SyncNet for audio-visual sync, Whisper for speech emotion) provide evidence grounded in the VLM CoT pipeline. 34 professional film experts provided the calibration annotations.

Key results:

• Spearman Rank Correlation Coefficient (SRCC) with human experts: Visual Concept Design 0.7472–0.8082; Acting 0.7636–0.8276; Cinematography 0.7517–0.8018; Aesthetics 0.7460–0.8174; Multi-Shot Logic and Rhythm 0.9000; Sound Design 0.9487
• All SRCC values statistically significant (p < 0.05)
• Top-ranked model: Seedance 2.0 (strongest in soundscape fidelity, identity preservation, visual quality, camera control)

Why it matters / caveats: By combining professional expert calibration with structured CoT reasoning and deterministic perception operators, EvalVerse makes scalable automated cinematic quality assessment feasible without repeated costly human studies. The reliance on 34 expert annotators introduces potential occupational and cultural biases in the aesthetic calibration.

Problem: Parallel test-time scaling methods keep reasoning branches isolated, causing redundant exploration and wasted compute when multiple branches independently rediscover the same intermediate results.

Method: Collaborative Parallel Thinking (CPT) is a training-free inference framework that extracts compact intermediate information from active search branches and maintains a deduplicated query-level information pool, broadcasting it through each branch's input context so subsequent steps can reuse findings from sibling branches. The method is evaluated on HMMT (Harvard-MIT Mathematics Tournament) and AIME (American Invitational Mathematics Examination) benchmarks across multiple rollout budgets and model scales.

Key results:

• CPT establishes a stronger accuracy–latency Pareto frontier than strong baselines across all evaluated rollout budgets and model scales on HMMT and AIME
• Specific per-benchmark accuracy numbers not stated in the available text

Why it matters / caveats: CPT is plug-and-play (training-free) and directly applicable to any parallel sampling framework, making it broadly deployable. The information pool design raises questions about whether sharing intermediate steps could introduce confirmation bias or cause branches to prematurely converge on incorrect shared conclusions; ablations on this are not detailed in the available text.

Problem: Deploying capable frontier-scale language models in long-horizon agentic workflows (software engineering, web research, office automation) requires both high task performance and inference efficiency at 192K-token context lengths.

Method: MiniMax-M2 is a 229.9B-parameter sparse Mixture-of-Experts transformer (62 layers, 9.8B activated per token) using 256 fine-grained experts with sigmoid gating and per-expert bias terms for load balancing, full multi-head attention with GQA (48 query / 8 KV heads), RoPE, and a Multi-Token Prediction module (K=1 during pre-training, K=3 during continued pre-training via weight copying). Pre-training uses 29.2T tokens with upsampled code/math/STEM content in two phases (constant 19.9T + decay 9.3T tokens), with progressive context extension from 8K to 192K. Post-training combines agent-driven SWE data pipelines, an Agent-as-a-Verifier (AaaV) framework, and the CISPO RL algorithm (asymmetric importance-sampling clipping) within the Forge infrastructure, which supports white-box and black-box agents, prefix-tree merging (up to 40× training speedup), and MTP-based speculative decoding. M2.7 adds an autonomous self-evolution loop where the model debugs its own training runs.

Key results:

• SWE-bench Pro: M2.7 56.2% vs. Opus 4.6 57.3%, GPT 5.4 57.7%, Gemini 3.1 Pro 54.2%
• SWE-bench Multilingual: M2.7 76.5% vs. Opus 77.8%
• AIME 2026: M2.7 94.2%
• GPQA-Diamond: M2.7 89.8%
• BrowseComp: M2.7 77.8%
• Prefix-tree merging: up to 40× RL training speedup
• Self-evolution: 100-round autonomous iteration yielding 30% gain on internal programming scaffold benchmark

Why it matters / caveats: MiniMax-M2 demonstrates that sparse MoE models with ~10B activated parameters can match dense frontier models on agentic benchmarks when paired with specialized post-training pipelines. The self-evolution capability in M2.7 is an early but concrete step toward autonomous model improvement. Evaluation relies partly on internal benchmarks, and third-party replication of the 100-round self-evolution result has not been reported.

Problem: Transforming full film episodes into localized or stylized versions (actor replacement, style transfer) requires maintaining character identity, scene consistency, and narrative coherence across hundreds of shots — a temporal scale at which current generative models exhibit identity drift and semantic erosion.

Method: Soap2Soap is a three-agent pipeline: a Video Understanding Agent (Gemini 3 Flash) that extracts a scene-aware JSON screenplay (Sjson) and assigns visual reference anchors; a Video Generation Agent that performs batch keyframe generation as 2×2 or 3×3 grids (Nano Banana 2) for intra-batch identity consistency, followed by shot-level image-to-video synthesis (Veo 3); and a Verification Agent (Gemini 3 Flash) that executes a Critique–Correct–Verify loop across four dimensions (generation quality, identity, scene stability, plot consistency). The Dual-Bridge Consistency mechanism maintains long-horizon coherence through the language bridge (Sjson) and a visual memory ℳ with scene- and shot-level anchors, with dynamic contextual memory allocation providing each shot only its minimal sufficient context. SoapBench is introduced as the evaluation benchmark, covering 10 movies with up to 42 consecutive shots per segment.

Key results:

• ID-VLM: Soap2Soap 9.17 vs. Kling O1 8.11, Runway Gen4 7.48, Mocha 6.21
• Scene-VLM: Soap2Soap 8.84 vs. Kling O1 8.37
• Plot-VLM: Soap2Soap 8.67 vs. Kling O1 8.60
• CLIP-I (ID): Soap2Soap 0.842 vs. Kling O1 0.632
• CLIP-I (Scene): Soap2Soap 0.819 vs. Kling O1 0.751
• Ablation: removing dynamic memory allocation drops F1 from 0.936 to 0.618; removing verification loop drops F1 to 0.887

Why it matters / caveats: The Dual-Bridge memory architecture shows that structured semantic + visual memory can substantially outperform commercial video APIs (Kling, Runway) at multi-shot long-form consistency. The system depends on proprietary foundation models (Veo 3, Gemini 3 Flash, Nano Banana 2), which limits reproducibility and may raise cost barriers for adoption.

Problem: Existing vision-language models use uniform frame sampling that allocates equal token budgets to static and dynamic video regions, failing to capture fine-grained temporal events such as high-frequency repeated motions.

Method: LLaVA-OneVision-2 introduces codec-stream tokenization, which decomposes video into adaptive Groups of Pictures (GOPs) using H.264/H.265 codec signals (P/B-frame packet sizes) rather than fixed time intervals. Within each GOP, motion vectors and luma residuals form a saliency map that selects 2×2 patch blocks for canvas packing with frame-level allocation weights. The architecture uses OneVision-Encoder with codec patchification and windowed spatial attention, a two-layer MLP projector, and Qwen3-8B as the language decoder. Training proceeds in four stages: image-text pairs, LLaVA-OneVision-1.5 instruction data + 60-frame video captions, 384-frame long-form captions, and a final stage with variable-length-GOP codec pipeline (768 frames) plus the new LLaVA-OneVision-2-Spatial-4M dataset for 2D/3D spatial supervision. A new JumpScore benchmark (189 jump-rope videos, decimal-second cycle annotation, mAP metric) is introduced for fine-grained temporal grounding.

Key results:

• Video benchmark average (18 tasks): LLaVA-OV-2 62.5 vs. Qwen3-VL-8B 58.2 (+4.3)
• Spatial benchmark average (11 tasks): 63.5 vs. 58.2 (+5.3)
• Tracking J&F (4 tasks): 48.0 vs. 32.4 (+15.6)
• JumpScore mAP: 74.9 vs. Qwen3-VL-8B 30.1 (+44.8)
• Temporal grounding: Charades-STA +5.2, ActivityNet +7.0, QVHighlights +7.0
• Codec-stream vs. uniform frame sampling (matched budget): temporal grounding +9.7, JumpScore +17.3
• CrossPoint spatial reasoning: +35.0 over Qwen3-VL-8B

Why it matters / caveats: Leveraging codec structure for token allocation is a practical and zero-cost signal that substantially improves temporal grounding and spatial reasoning at the 8B scale. However, codec-stream tokenization underperforms uniform frame sampling on motion prediction and trajectory estimation tasks where dense temporal continuity matters more than event-level evidence.

Problem: Reinforcement learning with verifiable rewards (RLVR) frameworks for multi-source visual reasoning (e.g., infrared + RGB, multi-view) do not explicitly model the performance interaction between single-source and multi-source inputs, causing naive multi-source fusion to underperform the best single source when one modality dominates.

Method: MARS (Mono-Anchored Advantage Normalization) generates both multi-source and mono-source rollouts during training and normalizes advantage estimates using the union of rewards from both sets: A^hy = (r_multi − mean(G^multi ∪ G^mono)) / std(G^multi ∪ G^mono). This formulation is applied as a drop-in modification to GRPO and DAPO. A theoretical result (Theorem 3.2) decomposes the gradient into standard reward optimization plus a multi-source information gain regularization term ΔIG measuring the reward increment of multi-source over mono-source trajectories.

Key results:

• Average improvement: +3.2% on GRPO, +4.9% on DAPO across evaluated datasets
• Infrared modality: +3.8% improvement
• Multi-view modality: +7.0% improvement
• Robustness under visual degradation: only 0.5% drop under severe Gaussian noise

Why it matters / caveats: MARS provides a theoretically grounded and lightweight modification to standard RLVR training that consistently improves multi-source visual reasoning without architectural changes. The evaluation covers infrared and multi-view settings; generalization to other modality combinations (depth, audio-visual) is not yet demonstrated.

Problem: Existing LLM agent benchmarks focus on explicit single-turn instructions and do not evaluate whether agents can infer, maintain, update, and proactively act on implicit user preferences extracted from fragmented long-term interaction histories.

Method: VitaBench 2.0 organizes tasks as temporal sequences for 56 manually curated user profiles, each with 2,000+ fine-grained preferences across daily life domains (dining, travel, shopping) that dynamically shift through addition, deletion, and modification events. Interaction histories consist of multi-turn conversation dialogues and user behavior logs mixing signal and noise. Two memory implementations are compared: Agentic Memory (agent-controlled retention/retrieval) and RAG Memory (vector-based retrieval). Evaluation metrics are Avg@4 (average score across 4 runs) and Pass@4 (fraction of tasks where all 4 runs succeed). The benchmark also includes proactive tasks requiring agents to recognize when additional information is needed before acting.

Key results:

• Top performer (Claude Opus 4.6) under full context: 0.503 Avg@4, 0.337 Pass@4
• DeepSeek-V4-Pro: 0.456 Avg@4, 0.267 Pass@4
• Claude Opus 4.6 with agentic memory: drops to 0.454 Avg@4 (performance degradation vs. full context)
• Proactive tasks consistently underperform standard personalization (Claude: 46.0 vs. 27.4)
• Even with ground-truth preferences provided, most models score below 50, indicating difficulty in prioritizing and applying preferences

Why it matters / caveats: VitaBench 2.0 surfaces a critical capability gap: even frontier models fail to reliably personalize behavior over long interaction histories, and enabling chain-of-thought "thinking" modes provides inconsistent gains, suggesting personalization requires capabilities qualitatively different from general reasoning. The benchmark is limited to daily life domains, and the 56-user profile set may not capture the diversity of real-world user preference distributions.

Generated 2026-05-27 from HuggingFace Daily Papers API.