AI papers — 2026-05-13

Problem: Existing multimodal systems treat visual understanding and image generation as separate pipelines; SenseNova-U1 unifies them in a single architecture that operates directly on pixels without pretrained vision encoders or VAE decoders.

Method: The NEO-unify architecture uses a native Mixture-of-Transformers (MoT) backbone with full parameter decoupling between understanding and generation streams, a near-lossless two-layer convolutional visual interface (32×32 patches with MLP decoding), hybrid attention enabling causal masking for text while allowing bidirectional image token interaction, resolution-adaptive noise scaling (σ_R), and flow matching for pixel-space generation. Two variants are released: SenseNova-U1-8B-MoT (dense) and SenseNova-U1-A3B-MoT (MoE, 30B total / ~3B active). Training spans six stages: understanding warmup, three-phase generation pre-training scaling from 256² to 2048², unified mid-training (84K steps), SFT (9K steps), RL post-training with aesthetic/style rewards, and Distribution Matching Distillation (100 → 8 inference steps).

Key results:

• MMMU: 74.78% (8B), outperforming Qwen3VL-8B (74.10%)
• MMMU-Pro: 67.69% vs. Qwen3VL's 60.40%
• MathVista: 84.20% vs. Qwen3VL's 81.40%
• GenEval overall: 0.91 (both variants), vs. Qwen-Image 0.87, BAGEL 0.82, Janus-Pro 0.80
• DPG-Bench (A3B): 88.14 overall, 94.19 Global score (highest among open-source models)
• TIIF-Bench short instructions: 89.74 (8B), outperforming FLUX.1 and Emu3.5-32B
• CVTG-2K text rendering: 0.940 average word accuracy (8B); LongText-Bench: 0.979 English, 0.962 Chinese
• Per-step latency for 2048×2048 generation: 0.415 s on RTX 5090

Why it matters / caveats: SenseNova-U1 provides the first encoder-free, VAE-free architecture that meaningfully closes the gap between unified and specialized models on both understanding and generation benchmarks at 8B–30B scale. The paper notes residual grid artifacts in the MLP patch decoder and that the A3B variant's RL post-training (200 epochs) has room for improvement compared to the 8B variant (1,600 epochs).

Problem: Cloud-hosted LLM agents with personalized memory systems expose sensitive user information to cloud providers during memory processing.

Method: MemPrivacy runs a lightweight on-device model (0.6B–4B parameters, trained with SFT then GRPO with F1-score rewards) that identifies privacy spans according to a four-level taxonomy (PL1 generic preferences → PL4 exploitable secrets), replaces them with typed structured placeholders (e.g., <EMAIL_1>), and locally stores the mapping for downstream restoration. The cloud receives only desensitized text with semantically meaningful placeholder types, and responses are restored locally via string substitution. A new benchmark, MemPrivacy-Bench, contains 200 user profiles, 52,000+ privacy instances, and ~1M dialogue tokens (50% Chinese, 50% English), annotated to 98.08% accuracy.

Key results:

• MemPrivacy-4B-RL: 85.97% F1 on MemPrivacy-Bench vs. GPT-5.2 at 68.99% and OpenAI Privacy Filter at 35.50%
• Out-of-distribution (PersonaMem-v2): 94.48% F1 vs. Gemini-3.1-Pro at 92.18%
• Utility loss on LangMem (GPT-4.1 backend): −1.30% vs. −26.67% for irreversible masking
• Utility loss on Mem0: −0.73% vs. −41.87% for irreversible masking
• Edge inference latency: under 2 seconds per message even on privacy-dense data
• Protecting only PL4 (secrets): less than 0.25% utility loss across all memory systems

Why it matters / caveats: The framework demonstrates that typed placeholders preserve enough semantic structure for cloud memory systems to function effectively, with utility losses under 1.6% even when protecting all four privacy levels. No formal differential privacy bounds are provided, and the evaluation covers only three memory systems (LangMem, Mem0, Memobase), leaving adversarial scenarios with auxiliary information unaddressed.

Problem: Full-attention LLMs lack efficient mechanisms to retain and recall information beyond their fixed context window without retraining or replacing the backbone.

Method: δ-mem augments a frozen full-attention LLM backbone with a compact online associative memory state. Using delta-rule learning, it compresses historical context into a low-rank state matrix (e.g., 8×8) and generates low-rank attention corrections that are injected into each layer — requiring no full fine-tuning, no backbone replacement, and no explicit context extension. Only the lightweight memory module is trained.

Key results:

• With 8×8 memory state: achieves 1.10× the performance of the frozen backbone
• Outperforms the strongest non-δ-mem memory baseline by 1.15×
• MemoryAgentBench: 1.31× over backbone
• LoCoMo benchmark: 1.20× over backbone
• General capabilities are largely preserved

Why it matters / caveats: δ-mem offers a plug-in memory upgrade for deployed frozen LLMs without the cost of full fine-tuning or architectural changes, which is attractive for continual learning and long-context tasks. The HTML full paper was unavailable at time of processing, so architectural details and dataset descriptions above rely on the abstract; specific hyperparameters, dataset names, and ablation numbers are not stated.

Problem: Training deep research agents via reinforcement learning requires dense reward signals, but long-form research tasks lack verifiable intermediate rewards.

Method: RubricEM decomposes research trajectories into four explicit stages (Plan, Research, Review, Answer) conditioned on self-generated rubrics. Stage-Structured GRPO (SS-GRPO) assigns separate rubric-scored credit to each stage rather than broadcasting a single terminal reward. A reflection meta-policy shares the backbone with the task policy; it generates and scores repair reflections asynchronously, storing the highest-scored ones in a rubric bank for future retrieval. The base model is Qwen3-8B distilled via SFT from Gemini-3.1-Pro teacher trajectories, then trained with 1,400 RL steps.

Key results:

• RubricEM-8B average across HealthBench / ResearchQA / DeepResearchBench / ResearchRubrics: 55.5 vs. DR Tulu-8B at 53.6 (1,900 RL steps) and Tongyi DeepResearch-30B at 50.8
• HealthBench: 49.3; ResearchQA: 74.5; DRB: 47.8; ResearchRubrics: 50.3
• Short-form OOD transfer (SimpleQA / 2Wiki / WebWalker / DSQA): 73.5 average vs. DR Tulu-8B at 49.0
• SimpleQA: 92.3%; WebWalker: 70.0%
• RL adds 6.3 points over the SFT-only baseline (49.2 average)

Why it matters / caveats: RubricEM shows that structured stage decomposition and rubric-as-interface training can produce competitive long-form research agents at 8B scale, with strong OOD generalization to short-form retrieval tasks. Limitations include reliance on a strong proprietary teacher (Gemini-3.1-Pro), potential LLM judge biases in rubric scoring, and a different search backend from the DR Tulu baseline, complicating direct comparison.

Problem: Vision-Language-Action models for robotic control and embodied AI treat action generation separately from environment dynamics prediction, limiting generalization and planning capability.

Method: The paper introduces World Action Models (WAMs) as a unifying framework that jointly models the distribution over future states and actions. It proposes a taxonomy organizing existing approaches into Cascaded WAMs (where a world model and action policy are chained) and Joint WAMs (where state prediction and action generation share a common learned representation). The paper surveys data ecosystems spanning robot teleoperation, human demonstrations, simulation, and egocentric video, and proposes evaluation protocols covering visual fidelity, physical commonsense, and action plausibility.

Key results:

• Not stated (survey/position paper with no original experimental benchmarks).

Why it matters / caveats: WAMs provide a conceptual unification of world models and action models that could guide future embodied AI research toward more sample-efficient and generalizable policies. As a position paper, it does not report new experimental results; claims about the superiority of the WAM framing remain to be validated empirically.

Problem: Enterprise agents must predict cascading state transitions triggered by business rules, but learned world models trained offline degrade when deployment configurations shift.

Method: The paper introduces CascadeBench, a synthetic benchmark with 27,243 verified transition samples across 64 worlds, 6 industries, and 3 organizational sizes, stratified into three complexity tiers (schema-determined T1, rule-composable T2, execution-inferred T3). It compares three agent strategies on ServiceNow instances: prompted frozen models, LoRA fine-tuned world models (rank 16, trained 2 epochs), and a discovery agent using the SyGra ReAct-style framework that retrieves live business rules via a single snow_query tool (15-call recursion budget). Models evaluated include Claude Sonnet/Opus 4.6, GPT-5, Gemini 3 Pro, Qwen-3.5/3.6-27B, and Gemma-4-31B.

Key results:

• Fine-tuned models (in-distribution): 82–91.6 IoU; on CascadeBench (out-of-distribution): collapse to ~40–41 IoU
• Discovery agent with oracle rules: 38–50 IoU (reasoning ceiling)
• Discovery agent without rules: 12–32 IoU vs. prompted baseline 9–10 IoU
• T2 (rule cascades): direct prompting drops to 0.0 IoU; discovery agent recovers to 0.63–0.65
• T3 (execution-dependent): both methods plateau at ~0.55, indicating a fundamental ceiling
• Multi-step rollout (k=1–5): discovery consistently outperforms matched baselines by 0.05–0.13 IoU

Why it matters / caveats: The work demonstrates that runtime retrieval of inspectable configuration is more robust than offline-learned dynamics under deployment shift, particularly for rule-driven enterprise transitions. The approach is evaluated only on ServiceNow and assumes configuration is readable at inference time; when access controls block rule inspection, the discovery agent degenerates to the prompted baseline.

Problem: Unified multimodal models (UMMs) that perform both text reasoning and image generation lack a reinforcement learning signal that can jointly optimize text and diffusion-based image generation for self-reflective refinement.

Method: AlphaGRPO applies GRPO to BAGEL (a native UMM backbone) by formulating unified trajectories that combine autoregressive reasoning text with diffusion-based image tokens, propagating shared advantage signals to both. The reward function, DVReward, decomposes image prompts into atomic binary questions across 10 semantic dimensions (entity existence, attributes, spatial relations, counting, style, etc.) and 8 quality dimensions (geometry, texture, lighting, aesthetics, anatomy, etc.), scoring answers via "Yes"/(Yes+No) token probability ratios and combining results as a geometric mean. False-Positive Rectification (FPR) prevents degraded refinements from receiving positive advantages. Training uses LoRA (r=32, α=64) on 19,500 prompts across 39 compositional tasks for 380 steps on 64 A100 GPUs.

Key results:

• TIIF-Bench short (with self-reflective refinement): 83.86 vs. BAGEL baseline 75.21 (+8.65)
• GenEval (with refinement): 88.2 vs. BAGEL 84.0 (+4.2%)
• DPG-Bench (with refinement): 87.86 vs. BAGEL 85.07 (+2.79)
• GEdit-Bench (image editing, T2I-trained model): +0.52 overall vs. BAGEL
• Human preference on 200 real prompts: 40.5% overall preference, 43.0% perceptual quality preference for AlphaGRPO over BAGEL
• DVReward outperformed VIEScore, PickScore, HPSv3, and UnifiedReward in ablations

Why it matters / caveats: AlphaGRPO is the first demonstration that GRPO with decompositional verifiable rewards can jointly optimize autoregressive and diffusion generation in a single UMM, enabling inference-time self-reflective improvement. Current limitations include noise artifacts at 512-resolution, reliance on outcome-only rewards (no process rewards), and evaluation limited to single-turn refinement rather than multi-turn conversation.

Problem: Visual tokenizers that rely solely on the final encoder layer discard fine-grained texture and edge information that is attenuated as residuals in deeper layers, limiting reconstruction and generation quality.

Method: DRoRAE adds a 29M-parameter fusion module to a frozen DINOv2-B encoder (86M parameters) paired with a ViT-XL decoder (335M parameters). The module consists of per-layer two-layer MLP experts, an energy-constrained router using L2-norm normalization (rather than softmax, enabling negative weights for active feature suppression), and an incremental correction formulation (β=0.2) that adjusts base features to maintain decoder compatibility. Training follows a three-phase decoupled strategy: (1) train decoder with frozen encoder, (2) train fusion module only with frozen decoder, (3) fine-tune decoder with frozen fusion.

Key results:

• ImageNet-256 reconstruction rFID: 0.57 → 0.29; PSNR: 18.8 → 24.32 dB; LPIPS: 0.256 → 0.134; SSIM: 0.483 → 0.701
• Class-conditional generation (DiT-XL) with AutoGuidance (scale 1.5): gFID 1.74 → 1.65
• GenEval (CC12M text-to-image): 0.60 vs. RAE baseline 0.56
• Expert capacity scaling follows log-linear law (R²=0.86); layer count scaling R²=0.49

Why it matters / caveats: DRoRAE establishes multi-layer fusion as a predictably scalable axis for visual tokenizer quality, analogous to vocabulary size scaling in NLP. Experiments are limited to DINOv2-B (12 layers); generalization to larger encoders and video tokenization remains untested.

Problem: Computer use agents given both GUI actions and API tool calls tend to overuse or underuse tools, failing to learn the optimal switching policy between the two action spaces.

Method: ToolCUA trains Qwen3-VL-8B-Instruct through a three-stage pipeline. First, an Interleaved GUI-Tool Trajectory Scaling Pipeline converts 10,000 existing pure-GUI trajectories into hybrid-action training data by synthesizing a library of 4,350 tools from recurring GUI procedures. Second, Tool-Bootstrapped GUI Reinforcement Fine-Tuning (RFT) applies SFT warmup on synthetic interleaved data (180K steps) followed by single-turn offline RL on 5K critical switching-point steps. Third, Online Agentic RL optimizes a Tool-Efficient Path Reward combining tool appropriateness (R_tool) and path efficiency (R_length) using GRPO with rollout size 32.

Key results:

• OSWorld-MCP task accuracy: 46.85% (+18.62 pp over baseline, ~66% relative improvement); SOTA among 8B models
• Tool Invocation Rate (TIR): 24.32%; Average Completion Steps: 14.93
• Hybrid training (+3.9 pp) outperforms pure-GUI training (42.05%)
• Cross-platform: 23.9% on unseen Linux multi-app tasks; 33.8% on WindowsAgentArena (unseen Windows)
• Without interleaved pretraining, online RL reaches only ~15% TIR; without path reward, GRPO drops ~7 pp

Why it matters / caveats: ToolCUA demonstrates that synthesizing tools from existing GUI trajectories and training with a switching-aware reward can meaningfully improve computer use agents without manual tool engineering. Synthesized tools lack concrete runtime implementations, creating dependency on tool maturity, and the benchmark coverage is primarily OSWorld-MCP.

Problem: Pixel-space diffusion models struggle to reach the quality of latent diffusion models (LDMs), while LDMs are memory-bottlenecked by VAE decoders and cannot natively generate at ultra-high resolutions such as 4K.

Method: L2P transfers a pre-trained LDM (Z-Image) to pixel space by discarding the VAE, applying large-patch tokenization (16×16 patches at 1024×1024; 64×64 at 4K), freezing intermediate DiT transformer layers, and training only shallow input/output blocks plus a lightweight U-Net Detailer Head using a Flow Matching objective aligned to the source LDM. A synthetic data pipeline generates 20,000 training images from the source LDM itself using a hierarchical taxonomy of 1,000+ categories. 4K capability is added via dynamic patch size expansion and increased noise shift (optimal range 4–5) using 10K prompts from UltraHR-100K.

Key results:

• DPG-Bench (1024×1024): 86.00, exceeding the source Z-Image-turbo at 84.86
• GenEval: 0.76 overall (~93.6% of source model)
• 4K UltraHR-eval4k: FID 33.46 (best among compared methods), FID-patch 21.77, IS 12.28
• 4K inference: 97.67% faster than source LDM; 38.81% less GPU memory
• Performance saturates at only 20K synthetic training samples

Why it matters / caveats: L2P enables VAE-free native 4K generation by transferring learned priors from an LDM using only synthetic data from the source model itself, avoiding costly real-data curation. Performance is bounded by the source LDM's capabilities, and the framework deliberately omits task-specific pixel-level losses (perceptual, physics-based) to remain architecture-agnostic.

Problem: Existing video generation models cannot produce multi-shot cinematic sequences in real time with interactive per-shot prompting and without regenerating prior footage.

Method: CausalCine adapts the Wan2.1-T2V-14B bidirectional diffusion model into a causal generator through three stages: (1) Long Multi-Shot Causal Tuning using 2N-segment packing with block-sparse causal attention on 100K long multi-shot videos (~15 s clips); (2) Content-Aware Memory Routing (CAMR) that dynamically retrieves historical KV cache entries by semantic relevance (mean-pooled query/key descriptors, top-k selection) with Block-Relative RoPE re-anchoring; (3) Few-Step Causal Distillation via Distribution Matching Distillation (DMD) compressing 100 steps to 4 steps with adversarial GAN regularization. Trained on 64 H800 GPUs at 832×480 resolution; runs at 16 FPS on 8 H200 GPUs.

Key results:

• Aesthetic score: 0.6261 (highest among autoregressive baselines including Self-Forcing, Infinity-RoPE, LongLive, MemFlow, ShotStream)
• Text alignment: 0.1980 (highest)
• Shot-Cut Accuracy (SCA): 0.9732 (highest)
• Subject consistency: 0.9717; Background consistency: 0.9675
• Vs. bidirectional HoloCine (15 s): Aesthetic 0.6194 vs. 0.5842; SCA 0.6608
• CAMR raises SCA from 0.5832 (no memory) to 0.7530

Why it matters / caveats: CausalCine is the first real-time interactive multi-shot video generator, enabling directors to steer narratives shot-by-shot without full regeneration. The system requires 8 H200 GPUs, well beyond consumer hardware, and struggles with fine-grained physical object-state continuity (e.g., precise liquid positions, hand poses) across shot boundaries.

Problem: In tool-integrated reasoning (TIR) systems, natural language reasoning and code execution are interleaved in ways that allow undetected NL arithmetic errors and treat code as post-hoc verification rather than the primary reasoner.

Method: ThinC (Think in Code) trains models to channel all computation through sequential code blocks: an initial NL planning phase establishes strategy, then multiple code blocks connected by interpreter execution outputs carry out all reasoning, with the final answer extracted directly from interpreter output. Trajectories are distilled from Qwen3.5-27B using few-shot prompting (12,200 problems, filtered for ≥3 code blocks and <50% planning-phase tokens), followed by SFT (3 epochs) and three-stage GRPO with verifiable rewards and increasing context budgets (16K→32K tokens). Training data comes from Skywork-OR1 and OpenMathReasoning (English, positive-integer answers only).

Key results:

• ThinC-4B avg@16 across AIME 2024/2025/2026, HMMT 2025, BeyondAIME: 78.1%, surpassing all interleaved TIR baselines including ReTool-32B and rStar2-Agent-14B, and outperforming Qwen3-235B-A22B-Thinking on 4 of 5 benchmarks
• AIME 2024: 88.3%; AIME 2025: 85.8%; AIME 2026: 86.0%; HMMT25: 74.0%; BeyondAIME: 56.1%
• ThinC-1.7B avg: 42.8% (+10.6 pp over base model)
• 99.2% of final answers grounded in interpreter output
• Average 349 lines of code per trajectory vs. ASTER's 102; fewer tool calls (6.1 vs. 11.1) and shorter responses (13.5K vs. 15.4K tokens)
• Recovery rate through 3 consecutive code failures: 64–69%

Why it matters / caveats: ThinC demonstrates that forcing mathematical reasoning entirely through executable code — using NL only for strategic planning — yields higher accuracy and fewer tool calls than interleaved TIR, even at 4B scale. Experiments are limited to 1.7B and 4B parameters and competition-level mathematics; applicability to other tool-integrated domains (search, symbolic solvers) is not established.

Problem: Training multimodal search agents on static datasets cannot adapt to evolving policy capabilities or the compositional visual reasoning required when intermediate tool outputs are images rather than text.

Method: The paper introduces two components. A Visual-Native Agent Harness registers every tool-produced image into a persistent image bank (referenced as <image:N>) so that downstream tools can consume intermediate visual evidence; the workspace includes 9 tools (web/image/scholar search, browser, Google Lens visual search, zoom, rotation, flip, Python execution). On-Policy Data Evolution (ODE) is a closed-loop pipeline with a forward stage (seed proposal → web exploration collecting 6 nodes → multimodal evidence graph construction → verifiable QA task synthesis) and a backward stage (rollout execution → 7-dimension rubric scoring → configuration update). SFT and RL training targets Qwen3-VL-8B and Qwen3-VL-30B.

Key results:

• Qwen3-VL-8B: 24.9% → 39.0% average accuracy across 8 benchmarks (+14.1 pp), surpassing Gemini-2.5 Pro (37.9%)
• Qwen3-VL-30B: 30.6% → 41.5% average accuracy (+10.9 pp)
• Per-benchmark (8B): VDR +16.2%, MMSearch +18.7%, FVQA +20.0%, MMSearch+ +14.9%
• Removing image-bank reusability costs up to 4.9 pp on MMBC
• ODE difficulty distribution: easy tasks drop from 41.5% (static SFT) to 0.4% (evolved RL), hard+expert rise to 61.9%

Why it matters / caveats: ODE shows that iteratively evolving training data against the current policy's failure distribution is significantly more effective than static synthesis for multimodal search agents. Computational overhead of iterative evolution is not quantified, scalability to larger model families is untested, and generalization beyond the 8 evaluated benchmarks is unknown.

Problem: Video relighting methods that rely on intrinsic decomposition (G-buffers) lose complex lighting effects like transparency and transmission, while pose-based approaches require camera pose estimation that accumulates errors.

Method: Relit-LiVE is built on Wan 2.1-T2V-1.3B and uses an RGB-Intrinsic Fusion Renderer that concatenates raw reference frames (preserving complex lighting effects) with grouped G-buffer intrinsic properties (metallic/roughness, depth/normal) as complementary conditioning. The model jointly generates relit video frames and per-frame environment maps warped to match camera viewpoint, eliminating explicit pose estimation. Training proceeds in three stages: supervised on synthetic (8,000 videos, 5,700 PBR maps, 2,241 HDR environment maps) and real datasets (13,809 clips); Intrinsic Perception Enhancement interpolating with/without raw image conditioning; and self-supervised illumination consistency via cycle-consistent reverse-video relighting pairs.

Key results:

• Synthetic video benchmark: PSNR 25.39, SSIM 0.807, LPIPS 0.205 (outperforms Diffusion Renderer, UniRelight, NeuralGaffer)
• MIT Multi-illumination benchmark: PSNR 21.86, SSIM 0.849, LPIPS 0.132
• Real-world temporal consistency (Motion Preservation): 0.1692 vs. 0.2405–0.4557 for baselines
• Environment map angular error in sunlit scenes: 20.35° mean vs. 54.88° (DiffusionLight), 66.12° (StyleLight)
• Ablation: removing environment video costs −5.94 PSNR; removing raw image costs −2.79 PSNR

Why it matters / caveats: Jointly generating relit video and environment maps in a pose-free framework is a meaningful step toward practical video production relighting. Inference is slow (~10 minutes for 57 frames at 832×480 on A800 GPU), and the frame-channel concatenation design forces resolution/frame-rate trade-offs that preclude real-time use.

Problem: LLM agents face safety failures distributed across entire trajectories (unsafe tool calls, prompt injection compliance, harmful instruction following) that scalar safety rewards cannot address without inducing broad over-refusal.

Method: FATE (Failure-Aware Trajectory Evolution) operates in iterative self-evolution rounds: the current policy executes tasks, on-policy failures are identified by verifiers, the same policy generates K repair candidates per failure, repairs are re-scored across four objectives (security, task utility, over-refusal control, trajectory validity), Pareto-front selection retains repairs that balance all objectives, and the policy is updated first via SFT on selected repairs then via Pareto-Front Policy Optimization (PFPO) on newly sampled completions. All repair proposals come from the current policy itself to match its actual failure distribution.

Key results:

• AgentDojo attack success rate: reduced from 0.812 to 0.540 (−33.5%) for Qwen3-8B
• Task success under attack (AgentDojo): 0.132 → 0.392 (+26.0 pp)
• AgentHarm harmful compliance rate: reduced from 0.719 to 0.125 (−82.6%) across model families
• Valid refusal rate: +51.9%
• ATBench classification accuracy: 77.8% (FATE) vs. 64.0% (AgentDoG baseline) (+13.8 pp)
• Results consistent across Llama, Ministral, Gemma, Phi, and Qwen families; all sizes from 0.6B to 32B

Why it matters / caveats: FATE shows that multi-objective Pareto optimization over on-policy failure repairs can substantially reduce agentic safety violations without sacrificing task utility, avoiding the over-refusal collapse seen with scalar safety rewards. The approach depends on benchmark-specific verifiers that may not transfer to new domains, requires environment-reset capability, and carries computational overhead from repair generation and re-scoring.