AI papers — 2026-08-19

Problem: Existing evaluations of LLM agent "skills" only measure aggregate task success, leaving unexplained when skills help, why they work, and where they fail.

Method: The authors build a contrastive trajectory-analysis pipeline that compares matched agent executions under Raw, Workflow Memory, and Skill (SKILL.md) conditions constructed from the same fixed-budget composition grid of successful/failed trajectories (5s0f through 0s5f), evaluated with Codex+GPT-5.3-Codex and Gemini CLI+Gemini-3.1-Pro-Preview on Terminal-Bench-2, SkillsBench, and Terminal-Bench-Pro. They normalize 8,135 trial records, open-code 240 sampled trajectories into 238 validated unique labels, and consolidate them into a 12-mode taxonomy grouped into three Skill-use Categories (SC1 procedural anchoring, SC2 execution/verification failure, SC3 invocation/budget failure). A separate retrieval study (RQ4) uses Qwen3-Embedding-0.6B for embedding-based ranking, explicit agent selection, and full-pool execution to test skill-pool scaling (5 to 100) and distractor confusability (random/similar/dissimilar).

Key results:

• Skill injection reaches 61.9% oracle-status success vs. 59.1% raw and 55.9% workflow memory, improving over matched Workflow Memory by 6.06 percentage points (95% bootstrap CI [+0.76, +11.36]).
• Procedural anchoring accounts for 65.7% of skill mechanisms vs. only 4.5% for explicit knowledge injection (human validation: 95.8% exact agreement, Cohen's κ=0.952 on 238 labels).
• SC2 execution-layer failures drop to 23.5% of skill-arm labels vs. 37.3% (raw) and 33.3% (workflow); environment_infrastructure_failure falls from 5.3% (raw) to 0.2% (skills).
• skill_guidance_misapplied_or_ignored occurs in 10.0% of skill-arm cases vs. 0.8% (raw) and 0.4% (workflow); workflow memory instead suffers 10.6% timeout/budget exhaustion vs. 1.7% (raw) and 4.4% (skills).
• As skill-pool size grows from 5 to 100, actual-use precision (Arm 3) collapses from 29.6% to 3.3% while downstream success stays roughly flat (36.4%→39.3%); offline embedding top-1 precision on similar pools falls from 70.5% to 53.4%.
• Lightweight baselines (instruction-derived short plan 47.7%, workflow-derived test-first template 59.2%) underperform Workflow Memory (62.3%) and Skill injection (79.2%) on 26 Terminal-Bench-2 tasks, showing gains are not from mere compact hints.

Why it matters / caveats: Skills primarily work by stabilizing execution (setup, tool sequencing, verification) rather than injecting facts, but they introduce a new invocation/applicability failure surface and do not fix algorithmic logic errors or weak runtime verification (algorithmic_logic_error stays 8.3-11.0% across arms); retrieval accuracy alone does not predict downstream success, so building reliable self-evolving agents requires attention to the full generate-retrieve-adapt lifecycle, not just producing more skills.

Problem: Existing benchmarks mainly test whether AI systems can produce correct answers from learned knowledge or execute tasks under extensive human methodological guidance, leaving it unclear how far AI can autonomously explore, select methods, and execute project-level scientific research when that guidance is withdrawn.

Method: ASI-Bench comprises 60 project-level research tasks across 11 scientific domains (mathematics, physics, chemistry, biology, astronomy, materials science, earth science, medicine/biostatistics, computer science, robotics, electrical engineering), each with a fixed scientific objective, data, executable sandbox environment, and verifiable artifacts. Autonomy is measured via a B1–B4 guidance gradient that progressively withdraws methodological information while holding the task and scoring fixed: B1 gives the complete method/procedure, B2 gives only the method name, B3 gives only the research goal and data (agent must determine the method), and B4 adds task-irrelevant distractor information to the B3 setting. Tasks were built by 40+ experts over 31,000+ human hours, from 1,300+ candidate ideas, with five review rounds, 1,100+ review assignments, 2,000+ task revisions, and 1,500+ sandbox validation runs; 18 Agent×Model configurations (harnesses: Codex, Claude Code, Kimi Code, MiMo Code, OpenHands; backbones including GPT-5.5/5.6, Claude Opus 4.8/5, GLM-5.2/5.3, DeepSeek V4, Kimi K2.7/K3, MiniMax M3, MiMo V2.5 Pro) were evaluated on it.

Key results:

• Average score across 18 Agent×Model configurations drops from 50.91 (B1, full guidance) to 29.10 (B2, method only) to 26.62 (B3, no method given) to 26.99 (B4, with distractors).
• Best system, Codex + GPT-5.6 Sol (ultra), reaches B3=51.60 — the only configuration to exceed 50 without prescribed methodology; raising reasoning effort from xhigh to ultra lifts B3 from 40.86 to 51.60 (+10.74 points).
• B1→B2 drop is 21.82 points versus only a further 2.48-point drop from B2→B3, identifying "method operationalization" (turning a method into a full procedure), not method selection, as the primary bottleneck.
• B2 is the most expensive setting (6.91M tokens, 49.7 min/task — 59% more tokens and 32% more time than B1), while B1 is cheapest (4.35M tokens, 37.8 min/task); B3/B4 use 25%/30% more tokens and 22%/18% more time than B1.
• Same backbone model varies sharply by harness: MiMo V2.5 Pro scores 16.17 (MiMo Code) vs. 23.25 (Claude Code); Kimi K2.7 scores 19.72 (Kimi Code) vs. 27.34 (Claude Code).
• Cost and performance are decoupled: Codex+GPT-5.6 xhigh hits B3=40.86% at ~$684/run, matching Claude Opus 5+Claude Code's 40.70% at ~$2,728/run (about 4x the cost).

Why it matters / caveats: The sharp B1→B2/B3 decline shows current frontier systems remain heavily dependent on human-specified procedures and are far from autonomous end-to-end scientific research, giving the field a concrete, unsaturated (B3 average 26.62) metric for tracking progress toward this capability; the benchmark is intentionally open for community task contributions to prevent saturation and gaming over time.

Problem: Reinforcement learning fine-tuning of long-horizon LLM agents is hampered by the heavyweight backpropagation, activations, and optimizer states needed for full-parameter updates, and by increasingly unreliable credit assignment as trajectories grow longer and more branching.

Method: Agentic ESOpt is a full-parameter evolution-strategies (ES) framework that, at each generation, samples Gaussian parameter perturbations around the current LLM (storing only noise seeds and applying in-place addition/subtraction so memory stays at inference level), rolls out the perturbed agents in the environment, z-score normalizes their trajectory-level rewards, and applies an online reward-weighted parameter update analogous to Salimans et al.'s ES pseudo-gradient. It adds a cosine decay schedule for the perturbation scale σ (nonzero terminal σT for train-time regularization, decayed to 0 for test-time compute) to balance exploration and adaptation, and its black-box interface lets parameter updates be interleaved with prompt-space methods like Trace2Skill and EoH for prompt-parameter co-evolution.

Key results:

• On long-horizon Sudoku (H*=15) with Qwen3.5-4B, Agentic ESOpt reaches 53.13% success vs. 40.63% for the strongest GRPO baseline (+12.50 points), while requiring only 8.41GB GPU memory vs. 58.88GB for GRPO (85.7% reduction).
• Across ReAct-style Math (DAPO/AIME 2026) and DocVQA with Qwen3.5-4B, Agentic ESOpt improves over the base model by an average of 13.7 points and over Agentic GRPO by 8.3 points.
• On WebArena-Lite with full-parameter Qwen3.5-27B, Agentic ESOpt improves the No Skill baseline from 29.47% to 36.16% (+6.69 points), and combined with Trace2Skill raises it from 33.94% to 36.36% (+2.42 points).
• In automatic heuristic design (test-time compute), Agentic ESOpt improves 28 of 36 matched Sample/EoH comparisons across 12 test sets (e.g., TSP N=20 gap reduced by 22.96% vs. Sample at T=1000).
• Wall-clock training on Sudoku H*=15 was faster with Agentic ESOpt (9.4h) than Agentic GRPO (19.0h) despite a larger population (G=32 vs. 8 rollouts), with comparable model FLOPs (9.4 vs. 10.9 EFLOPs).
• A population-sensitivity study found doubling G from 8 to 16 improved final test success by +677.0% for a 4B backbone but 0.0% for a 9B backbone, suggesting stronger backbones need fewer sampled directions.

Why it matters / caveats: By avoiding backpropagation and horizon-wise credit assignment, Agentic ESOpt enables full-parameter fine-tuning of large agents (e.g., 27B) on modest hardware (4×H100) and composes naturally with prompt/skill-space optimization; however, results are based on models up to 27B and specific benchmarks (Sudoku, Math/DocVQA, WebArena-Lite, AHD), and the population-scaling law across model sizes is described by the authors as preliminary, with universal scaling and frontier-model validation left to future work.

Problem: Frontier-scale open-weight MoE models remain effectively unusable on personal hardware because existing edge serving engines (llama.cpp, Ollama, KTransformers) use static expert placement and lack principled policies for prefill transfer, decode-time cache misses, and fluctuating edge resources, causing severe throughput loss and multi-second-to-minute latency spikes under agentic workloads.

Method: FreeToken introduces a two-level CPU-resident/GPU-cache expert-memory hierarchy with three co-designed mechanisms: full-layer double buffering that pipelines PCIe expert transfer with GPU computation during prefill; a semantic-aware state cache that anchors recurrent-state checkpoints at special-token boundaries (thinking segments, tool calls) so agentic context edits only trigger re-prefill of the new suffix; and a bandwidth-adaptive q⋆ policy during decode that analytically splits each step's cache misses between GPU cache-fill (PCIe transfer) and direct CPU execution based on the machine's measured PCIe bandwidth (B_P) and host expert-processing bandwidth (B_H). All routing-dependent cache control (LRU eviction, victim selection, miss classification) runs inside a statically captured CUDA Graph, and an elastic memory manager allows the GPU expert cache to be resized at runtime without restarting the engine.

Key results:

• On an RTX 5090: 77–83 tok/s decode on Qwen3.6-35B-A3B (BF16) and 22–25 tok/s on DeepSeek-V4-Flash (284B, MXFP4), 1.5–2.3x higher decode throughput than the strongest baseline across four agentic workloads.
• Decode rate stays within 12% of single-turn performance across agentic workloads, versus KTransformers losing 31% of its rate by the second workload.
• Worst-case TTFT stays below 44s in every workload, while baselines exceed 150s (llama.cpp 232s, Ollama 179s, KTransformers 946s) in at least one setting.
• Across five consumer GPUs (RTX 3090/4090/5090/5090-desktop/4060-laptop), decode throughput improves 1.3–2.1x over baselines; the 8GB RTX 4060 laptop serves a 35B model at 39.3 tok/s (92% of the RTX 4090 rate), exceeding Codex's 33 tok/s median production decode speed.
• On a single RTX PRO 6000 (96GB) workstation GPU, serves GLM-5.2 (753B parameters, 40B active, NVFP4) at 14.9 tok/s versus llama.cpp's 7.3 tok/s (2.0x), with comparable mean TTFT (7.5s vs 7.8s); KTransformers cannot serve this model at all on the same hardware.
• At equal cache capacity on an RTX 5090, FreeToken's LRU expert cache misses only 16%/39% of decode-time expert reads (Qwen3.6/DSV4-Flash) versus 41%/59% for KTransformers and 62%/89% for llama.cpp.

Why it matters / caveats: The system demonstrates that consumer GPUs (from an 8GB laptop to a single workstation GPU) can interactively serve MoE models from 35B up to 753B parameters — previously assumed to require datacenter clusters — by treating bandwidth as a scheduling signal rather than a fixed bottleneck; results depend on specific quantized checkpoints (MXFP4/NVFP4) and measured per-machine bandwidth profiling, and comparisons exclude cross-engine wall-clock totals since agent trajectories diverge across engines.

Problem: Deploying Vision-Language Models directly for embodied navigation is hampered by a geometric mismatch between VLMs' 2D pre-training priors and 3D action spaces, rigid always-on or fixed-interval Chain-of-Thought reasoning schedules, and memory mechanisms that either overflow context or discard critical historical information.

Method: The paper proposes TAMP-Nav, built on Qwen2.5-VL-7B and trained on a newly curated 90k-trajectory MultiNav-CoT dataset (derived from VLN-CE with Gemini 2.5 Flash-generated CoT annotations via spatiotemporal key-node mining). It combines a Pixel-to-3D Action Formulation (the VLM points to a 2D pixel on egocentric views, projected into 3D via depth/camera intrinsics for a SLAM controller), a Selective Reasoning and Anchor-Trajectory Memory mechanism (CoT triggered only at key topological nodes, stored as high-fidelity Explicit Anchors, with intervening path segments compressed into lightweight RoPE-based Space-Time Indicator tokens), and a Two-Level GRPO alignment paradigm that superimposes global trajectory-level rewards (success, SPL, reasoning density) with fine-grained local step-level rewards (target approach, collision avoidance, stop correctness, reasoning value, format) using an annealed guided-sampling rollout scheme.

Key results:

• Achieves state-of-the-art 66.2% SR / 58.8% SPL on R2R-CE Val-Unseen and 65.7% SR / 56.9% SPL on RxR-CE Val-Unseen, surpassing DualVLN (64.3%/58.5%) and NavFoM (61.7%/55.3%).
• Reaches this performance using only 90k training trajectories (~700k interactions), versus 763k trajectories for DualVLN and 3.37M interactions for NavFoM.
• Reduces inference cost to ~9 interaction steps per trajectory and 16.58s per task on an A800 GPU, versus ~30 steps and 37.47s (StreamVLN) / 41.46s (DualVLN).
• On long-horizon trajectories (>12.5m, 5,927 samples), TAMP-Nav's Anchor-Trajectory Memory achieves 49.8% SR versus 41.9% (DualVLN) and 30.9% (StreamVLN).
• Emergent selective reasoning uses only a 26.3% CoT trigger ratio while nearly matching the 100%-CoT upper bound (66.2% vs. 66.8% SR), and reasoning at trivial corridor locations drops from 38% (SFT) to 11% (after RL).
• Ablations show Pixel-to-3D action formulation adds +24.8/+25.2 SR points over a NavFoM-style metric-waypoint baseline, and Two-Level GRPO with annealed guidance adds +6.9/+7.3 SR points over trajectory-only GRPO advantage.

Why it matters / caveats: The pixel-pointing action space, selective reasoning, and dense two-level RL rewards together deliver both higher accuracy and much greater sample/inference efficiency than prior VLM-based navigators, and the approach is validated with real-world deployment; robustness tests show only a 2.8-point SR drop under 20% multiplicative depth noise, and CoT supervision quality (teacher model choice) is shown to materially affect downstream navigation performance.

Problem: Video creation agents cannot learn filmmaking knowledge from high-quality human films because no existing video representation is simultaneously agent-readable, agent-editable, and faithful enough to preserve the cinematic information needed for reconstruction and generation.

Method: AVA-Encoder is an Agentic Video Auto-Encoder that maps a film into a text-centered Film Knowledge Graph (Story–Event–Shot hierarchy plus Character, Scene, Object, Style, Camera, and Audio state nodes linked to a separate multimodal asset layer) via a multi-level Agentic Video Encoder (film-, shot-, and keyframe-level understanding with inter-level context injection), then reconstructs video through a fixed decoder pipeline. A dual-loop textual-gradient evolution framework, following the TextGrad formulation, drives self-improvement: an outer loop performs Data-Agnostic Encoding Policy Pseudo-Training (rewriting the shared 𝑃shot/𝑃kf prompts) guarded by an Anti-Forgetting Gate, and an inner loop performs Data-Dependent KG Representation Refinement (editing the input-specific graph) guarded by an Anti-Degradation Gate, with QA-based fact-checking rewards (𝑅reward) separated from an independent final evaluation protocol (𝑅eval).

Key results:

• AVA-Encoder reaches an Overall reconstruction fidelity of 49.0%, a 20.7-point absolute (73.1% relative) improvement over the strongest external baseline, soap2soap at 28.3% (Video 57.8% vs 36.7%, KF 73.7% vs 39.5%).
• The two optimization loops together add 6.6 points (15.6% relative) over removing both (49.0 vs 42.4 Overall); hierarchical encoding alone improves 27.5 (naive single-level) to 45.8 (policy-only), an 18.3-point/66.5% relative gain.
• In the controlled policy-only setting, the pseudo-trained encoder policy (45.8%) beats a human-tuned policy (44.4%) by 1.4 points (3.2% relative) while cutting shot-level prompt tokens by 74.3% (31,336→8,052) and keyframe-level tokens by 70.1% (13,574→4,062).
• Acceptance gates contribute 5.5 points (12.6% relative), improving from 43.5% (no gates) to 49.0% (full gates).
• The automatic evaluation benchmark agrees with human judgments on 710 of 730 blinded triples (97.3%) across 18 clips, 129 shots, and 246 keyframes.
• Injecting the released Film KG representation as a single textual input improves Overall generation quality (1–4 scale) for every tested downstream framework, e.g., MovieAgent 2.47→3.30 and FilmAgent 1.85→2.83.

Why it matters / caveats: The Film KG representation and its editing framework provide a text-native, agent-operable substrate that both faithfully reconstructs cinematic video and generalizes to boost existing agentic video-generation systems without framework-specific tuning; results depend on fixed proprietary foundation models (Gemini-3.1-Pro-Preview, Qwen-3.7-Plus, Nano Banana Pro, HappyHorse 1.0) and a relatively small evaluation set of 18 non-overlapping clips.

Problem: Diffusion-based image editing models are capped below 1K resolution by quadratic attention cost, and the common two-stage workaround (low-resolution edit followed by independent super-resolution) causes information divergence (hallucinated details that contradict the original high-resolution source) and texture degradation (over-smoothed or over-sharpened artifacts).

Method: EditBridge, built on Qwen-Image-Edit, reformulates high-resolution refinement as a diffusion bridge (Brownian bridge) that translates the upsampled low-resolution edited image directly to the high-resolution output, conditioned on the original HR source, rather than regenerating from noise. To make HR source conditioning tractable, it introduces Prior-Guided Block-wise Sparse Attention (PG-BSA), which extracts a correspondence prior (via argmax matching of cross-domain attention maps from the first-stage LR edit) and uses it to restrict target-to-source attention to a local k×k window around each semantically aligned anchor, reducing cross-domain attention complexity from O(N²) to O(N·k²); the sparse attention mechanism is implemented via the VMoBA framework with FlashAttention kernels, and the model is fine-tuned with LoRA (r=128, α=128) using the Prodigy optimizer.

Key results:

• Achieves 3.6–8.4× speedup at 2K resolution versus diffusion-based SR baselines, completing refinement in 4.08s (vs. 14.82–662.50s for competitors).
• Enables practical 4K editing in 61.10s, an 11× speedup over direct high-resolution inference (695.23s) and 1.8–2.1× faster than SR methods, while achieving the best mPSNR (24.14) and mSSIM (0.8347) among all compared methods at 4K.
• At 1K resolution, single-step inference (N=1) outperforms 5-step and 10-step sampling (HaarPSI 0.4991 vs. 0.4131/0.4020) while running in 0.84s.
• Ablation shows the proposed sparse attention avoids the ghosting/contextual-bleeding artifacts of full attention while being faster (0.84s vs. 0.73s at 1K is comparable, but 4.08s vs. 4.71s at 2K) and yielding better mSSIM (0.871 vs. 0.705 at 2K).
• Trained on 5,000 image pairs at 1K/2K resolution plus 1,500 pairs at 4K, curated from Aesthetic-4K and Aesthetic-Train-V2 with Gemini 3-generated instructions and Nano Banana Pro-synthesized targets.
• In a 30-participant, 50-image four-alternative forced-choice user study, EditBridge won highest preference across Detail Preservation, Realism, and Aesthetics against PiSA-SR, HiFlow, and direct inference.

Why it matters / caveats: The diffusion-bridge formulation plus semantically-routed sparse attention makes faithful 4K instruction-based editing practical in about a minute rather than tens of minutes, directly addressing the hallucination/texture problems of the standard edit-then-upscale pipeline. Authors note two caveats: the method depends on a pre-computed correspondence-index prior (not yet automated/learned), and the iterative bridge sampling still carries non-negligible overhead at very high resolutions.

Problem: Modern agents run inside deploy-time harnesses (context construction, tool execution, control flow) rather than as standalone LLMs, and existing RL frameworks that connect these harnesses to training through an LLM proxy ("harnessed agentic RL") make inconsistent, underspecified design choices around retokenization, sample merging, advantage calculation, and loss normalization that can destabilize training.

Method: The authors formalize harnessed agentic RL as a POMDP where the harness (not the trainer) owns the latent execution state, and characterize four implementation challenges: retokenization/sample merging (chat-template non-compositionality, decode-retokenize drift, output transformation breaking token-prefix continuity), advantage calculation, loss normalization, and training-backend scheduling under dynamic sample counts. They build Agent Lightning v1.0, a ~3,500-line disaggregated framework with an API Gateway, Rollout Controller (Kubernetes or local), and a VERL-based Customized Trainer, adopting best-effort sequence merging, rollout-level (not sample-level) advantage and loss normalization (rollout-level token-mean loss), and a "collocated async RL" scheme that time-shares rollout and weight-update phases on the same GPU pool.

Key results:

• Coding agent (Qwen3.5-9B + mini-SWE-agent on SWE-smith-derived data): RL improves SWE-bench Verified from 41.8% to 56.4% (+14.6 absolute points) using only ~6K training examples.
• Rollout-level Advantage + Rollout-level Norm reaches 38.2% validation reward at step 128 vs. 35.0% (sample-level baseline) and 33.1% (rollout-advantage-only), with more stable entropy growth.
• Search agent (Llama-3.2-3B-Instruct + GRPO on HotpotQA): validation exact-match reward improves from 25.1% to 41.7% (+16.6 points).
• General instruction-following agent (Qwen3-4B-Instruct-2507 + RLOO): validation reward improves from 51.9% to 70.2% (+18.3 points).
• Collocated async RL achieves roughly a 2x end-to-end speedup over synchronous RL while using fewer GPUs than standard async RL.
• On coding-agent rollouts, only 36% remain a single merged training sample on average, with each rollout yielding 2.41 samples on average — evidence for why rollout-level (not sample-level) statistics matter.

Why it matters / caveats: The paper gives the first systematic account of the correctness/stability pitfalls (off-policy stitching from naive token-buffer replacement, sample-count-dependent gradient weighting) that arise when RL training is run through real deployment harnesses rather than a trainer-owned environment loop, and ties these directly to reward-hacking risks (e.g., agents fetching source via git history or network calls) that required explicit safeguards; results are demonstrated only on three specific agent/harness/model combinations with modest compute, not a broad benchmark sweep.

Problem: Existing instruction-guided video editing datasets and models mostly support only single, isolated editing operations, and fail to handle compositional instructions that require jointly understanding, localizing, and executing multiple editing intents (e.g., replacing a person, removing an object, and stylizing the background) within the same video.

Method: The authors build CoinVE-200K via a three-stage pipeline: Qwen3.6-27B parses source videos (from filtered OpenVid-HD clips) into subjects/objects/background taxonomies and generates compositional multi-instruction prompts by sampling and combining atomic edit templates (replace, add, remove, stylize); SAM3 and SAM2 produce key-frame and temporally-propagated region masks used to guide HunyuanImage-3.0/Nano Banana key-frame editing followed by a fine-tuned Wan2.2-Animate-14B/VACE video synthesizer; Gemini 2.5 Pro then performs strict multi-perspective quality filtering (per-instruction accuracy, semantic/temporal consistency, physical plausibility). On top of this data, they train CoinVE-Edit, a 22B model combining Qwen3-VL-8B-Instruct with a Wan2.1-T2V-14B DiT backbone, using a Mask Predictor and GateNet to produce spatial-temporal masks and local/global gates that drive a novel "Q-Blending Cross-Attention" mechanism to disentangle and route per-instruction attention without adding new DiT parameters, trained via a three-stage curriculum (feature alignment, single-instruction, then compositional-instruction fine-tuning).

Key results:

• CoinVE-200K contains 200K video-editing pairs (1080p, 81–201 frames, avg. 2.55 instructions/video: 57.7%/31.7%/8.5%/2.1% with 2/3/4/5 instructions) with a reported average editing quality score of 4.85, higher than existing single-instruction datasets.
• On CoinVE-Bench (compositional editing), CoinVE-Edit scores SA 87.97 / SPA 89.45 / EP 89.60, beating proprietary Seedance 2.0 (SA 85.34, SPA 87.71) and Kling O3 (SA 86.91, SPA 80.93), with scope accuracy (SPA) exceeding the best competitor (Seedance 2.0) by 1.74 points.
• On single-instruction OpenVE-Bench, CoinVE-Edit achieves the top overall score (3.41) among open-source methods, ahead of SAMA (3.33), KiwiEdit (3.17), and VINO (2.91).
• Ablation: adding the Mask module raises SPA from 84.43 (Instr.-Concat.) to 85.49 (Q-Bias), and adding Q-Blending further raises it to 89.45, with SA improving from 82.61 to 87.97 overall.
• Offline-trained auxiliary modules reach a Mask Predictor IoU of 0.71 and GateNet binary classification accuracy of 0.95.
• Training used ~2M filtered image-editing samples, ~0.9M filtered video-editing samples, plus CoinVE-200K, run on 32 NVIDIA H200 GPUs.

Why it matters / caveats: This is the first large-scale dataset and benchmark specifically targeting multi-intent, region-aware compositional video editing rather than single-operation edits, and the region-aware Q-Blending attention offers a parameter-free way to reduce cross-instruction interference; however, results rely on MLLM-based (Gemini) automatic judging rather than large-scale human evaluation, and video-quality metrics (AQ, TQ, CQ, TS) show CoinVE-Edit only roughly on par with, not clearly superior to, baselines despite its accuracy gains.

Problem: Text-to-image models remain vulnerable to "benign adversarial" prompts—inputs that pass language-level safety checks (e.g., after LLM rewriting) yet still reliably trigger unsafe (NSFW) generations because the harm arises from the model's learned visual output distribution rather than from explicit unsafe text.

Method: DiSCO is a training-free, strictly black-box, prompt-level defense that first builds model-specific safe/unsafe reference image pools by querying the target generator on I2P prompts and filtering with NudeNet+Q16 classifier consensus, then optimizes a prompt via autoregressive suffix expansion using a LLaMA-3-8B suffix generator with beam search (width K=4, T=16 steps), scoring each candidate's generated image with a distribution-guided contrastive objective in frozen CLIP embedding space that maximizes cosine similarity to sampled safe references (R=8) while minimizing similarity to unsafe references. It is applied selectively only when a prompt's initial generation is flagged unsafe, and can be prepended to both undefended generators and existing defenses (SLD-Max, SAFREE, RECE, ESD) without modifying the underlying model.

Key results:

• Across 32 system–attack settings and five seeds, DiSCO reduces average ASR from 23.6% to 2.4% under NudeNet and from 8.3% to 1.7% under Q16.
• On undefended models under Ring-A-Bell, NudeNet ASR drops from 84.2%→7.8% (SD 1.4), 75.4%→3.9% (SD 2.0), and 89.7%→5.0% (Flux).
• Averaged over four attacks, NudeNet ASR falls by 42.6 points (SD 1.4), 33.0 (Flux), 26.5 (SD 2.0), 11.6 (SD 3); combined with existing defenses, SLD-Max+DiSCO improves by 21.1 points and SAFREE+DiSCO by 20.8 points (average NudeNet ASR).
• Generation quality improves alongside safety: CLIP alignment increases by +0.065 to +0.086 and ImageReward by +1.67 to +2.22 across undefended backbones.
• Ablations show a reference pool of only ~670 images (25% of full 2613/84 pool) suffices for near-identical ASR, and the full contrastive objective (avg 15.6% ASR) outperforms safe-only (21.6%) and unsafe-only (20.6%) single-pool variants.
• Default beam settings (K=4, T=16) achieve 15.5% ASR with CLIP 0.281, the best quality/efficiency trade-off among tested K∈{2,4,8}, T∈{8,12,16,32} configurations.

Why it matters / caveats: DiSCO offers a practical, architecture-agnostic safety layer usable on proprietary/closed-source models since it requires no gradients, weights, or internal access—only query access to the generator; the main cost is inference-time overhead from the T×K beam search requiring multiple candidate image generations per optimized prompt.

Problem: Video autoencoders are optimized almost exclusively for pixel-level reconstruction, but a reconstruction-optimal latent space is not necessarily well suited for generative modeling, especially given video's added need for temporal compression and coherence.

Method: V-RAE builds a generative latent space directly on frozen vision foundation model features (DINOv3, SigLIP2, EUPE, or V-JEPA 2.1) rather than training a VAE from scratch: a lightweight temporal attention pooling module compresses the temporally dense frozen features while preserving semantic structure, and a spatiotemporal Transformer decoder with 3D RoPE and a multi-frame unpatchify layer reconstructs coherent video from the compressed latents. Only the pooler and decoder are trained (with L1, LPIPS, adversarial, and Gram-matrix losses), and a DiT with rectified flow and a dimension-dependent noise-schedule shift is trained on top for generation. The paper also introduces tFVD, a temporal-coherence diagnostic based on decoding latent-midpoint interpolations.

Key results:

• V-RAE (V-JEPA 2.1) reaches 2.13 rFVD on K600, a 40.5% improvement over the strongest video VAE baseline (3.58); DINOv3-L variant gets 6.12 rFVD on UCF101 (vs. 6.05 for Wan2.1 VAE).
• Semantic probing: V-RAE reaches 89.13% top-1 accuracy on UCF101 (DINOv3-L) vs. 30.83% for the strongest VAE baseline (AToken); also 72.91% on SSv2 and 83.12% on K400.
• Class-conditional generation: best V-RAE variant (V-JEPA 2.1) achieves gFVD of 117.86 on UCF101 and 19.16 on K600, improving over the strongest non-V-RAE baseline by 25.14 and 22.50 points respectively.
• Converges up to 6× faster than Wan2.2 VAE latents under matched training settings (e.g., EUPE-B matches Wan2.2's 180K-step K600 gFVD in only 30K steps).
• tFVD correlates far more strongly with generation quality (gFVD) than rFVD does: Pearson r = 0.919 vs. 0.473 on K600, and r = 0.621 vs. 0.200 on UCF101.
• On Cityscapes future video prediction (40K steps), V-RAE (EUPE-B) improves gFID from 15.02 to 11.52 and gFVD from 144.47 to 111.36 versus Wan2.2 VAE, despite worse rFVD (29.29 vs. 7.03).

Why it matters / caveats: The results show reconstruction fidelity alone is a poor proxy for generative utility, and that reusing frozen semantic representations (rather than training a dedicated reconstruction-focused VAE) yields latent spaces that are both more semantically informative and easier/faster for diffusion transformers to learn, with benefits extending beyond generation to future-frame prediction. A caveat is that V-RAE underperforms large-scale video VAEs on pixel-level metrics like LPIPS/PSNR/SSIM since the encoder is frozen and not optimized for low-level fidelity.

Problem: Existing evaluations of memory-augmented LLM agents are benchmark-concentrated, efficiency-blind, and backbone-narrow, so it remains unclear which memory substrate (the underlying medium in which memory is represented and stored) should be used under which operating regime.

Method: The authors build a controlled harness that isolates the memory substrate as the sole experimental variable, implementing 11 methods (M1–M11) across seven substrate families spanning external memory (Flat Index, Text Record, Structural, Hierarchical, Refinement) and internal memory (Weight, Activation) — e.g., dense/sparse vector indices, a gist index, evolving notes, a dual-level entity-relation graph, RAPTOR-style hierarchical trees, distilled-strategy and skill-bundle refinement, LoRA adapter tuning, and full-context vs. episode-clustered re-prefill. All substrates are run under identical interaction histories on three backbones (Qwen3-8B, Qwen3-32B-AWQ, Gemma-4-26B-A4B-IT) across four benchmarks (LoCoMo, MemoryAgentBench, ALFWorld, BigCodeBench-Hard), instrumented with 26 performance and efficiency metrics, plus a top-k retrieval-breadth sweep and an attention-probing analysis.

Key results:

• Landscape survey of 52 prior memory systems: 62% of benchmark/system pairs use only LoCoMo and LongMemEval, only 21% report any efficiency metric, and 81% use GPT-family backbones exclusively.
• On LoCoMo/LME-S, the Dual-Level Graph (M5) wins across model bands but costs 10–100× the latency of flat baselines (e.g., M5 27.84s–29.22s per query vs. M2 at 0.33s–2.37s).
• On ALFWorld, Episode-Clustered Re-prefill (M11) roughly doubles NoMem task success on Qwen3-8B (11.9% vs. 5.7%), and Distilled Strategies (M7) reaches the table's peak 32.1% task success rate on Qwen3-32B-AWQ.
• On BigCodeBench-Hard, M5 leads with 15.5%–20.9% Pass@1 but at 2–6× the inference cost of flat substrates, which nonetheless lift Pass@1 above NoMem on every backbone.
• Retrieval-depth sweep: increasing k monotonically raises LLM-judge accuracy on LoCoMo, but on ALFWorld it flips the sign — M7 drops from 32.1% (k=1) to ~25% (k=5) and flat retrievers fall below the 22.4% NoMem baseline.
• Attention probing shows retrieval depth funnels attention mass from task-context tokens into the retrieved block (LoCoMo: 0.34→0.10 context mass vs. 0.05→0.66 retrieved mass; ALFWorld: 0.19→0.17 vs. 0.34→0.44), explaining why more retrieval helps QA but hurts agentic decision-making; a scalability sweep (6K–262K tokens) shows M8 improving 0.32→0.51 P4 and M5 improving 0.28→0.48 in quality but paying steep latency growth at 262K tokens.

Why it matters / caveats: The findings argue against a single universal memory substrate, showing that optimal choice reverses between user-centric QA and agent-centric decision-making and shifts again with history length, motivating substrate routing (a multi-substrate or multi-agent composition) as a necessary design principle; the design rule proposed is to trade read breadth for write depth (retrieve fewer entries, invest more in write-time distillation/structuring).

Problem: Conventional data pipelines for image generation curate task-specific corpora (T2I, editing, etc.) in isolation, ignoring the dependency structure among generative capabilities that a generalist model must acquire.

Method: The authors build a capability-driven data infrastructure with three interoperable data engines — a T2I engine for text-image grounding, an image-editing engine for inter-image transformation (combining operation-specific pair construction via SAM3-based masking, mining of natural visual associations from co-occurring/multi-panel images, and expert self-distillation from fine-tuned Qwen-Image-2512/FLUX.2/Wan 2.2 models), and a knowledge-grounded engine using Wikidata PageRank-guided entity retrieval — unified by VLM-based caption experts (initialized from Qwen3.5-27B) that align T2I captions and editing instructions across granularities. On top of this, a five-stage capability-aligned curriculum (256px T2I pre-training → 256/512px complex T2I → joint 512px T2I&Edit pre-training → 512/1024px continual training → 1024px SFT) jointly evolves task composition, concept distribution, data quality, and resolution, closed by a capability-gap-driven active feedback loop that retrieves/constructs data for failure modes and applies gap-aware resampling.

Key results:

• Curates a 440M-image T2I corpus (from a billion-scale pool), 120M image-editing pairs, and over 27M image-entity pairs.
• Trains MM-DiT models from scratch at 3B and 6B parameter scales.
• On CPI-General-Bench (2,039 examples, 30 tasks) and CPI-Practical-Bench (558 examples, 51 application types), the 3B model scores 3.95/3.91 (Overall 3.93) and the 6B model scores 3.96/3.92 (Overall 3.94), on a 1–5 VLM-assessed scale.
• Knowledge engine distills ~3M high-salience entities from over 100M Wikidata entities via PageRank filtering plus VLM assessment.
• Taxonomy for rebalancing uses over 280K fine-grained semantic tags organized into a 4-level hierarchy (15/74/331 categories at the top three levels).

Why it matters / caveats: The paper reframes data curation itself as a scaling axis organized around capability dependencies rather than static corpora, showing measurable transfer (e.g., text-rendering learned from T2I data transferring to editing); however, quantitative evaluation is limited to two CPI-Bench subsets from the same author group (with scores clustered tightly around ~3.9–3.96), and no comparison table against baselines (e.g., Qwen-Image, FLUX.2) on these metrics is reported despite qualitative comparisons being shown.

Problem: Existing agent benchmarks are mostly researcher-defined and evaluated with coarse holistic judgments, leaving it unclear whether AI progress translates into reliably completing the end-to-end professional workflows that real users actually delegate to AI products.

Method: The authors build StartupBench through a survey- and interview-driven pipeline: they identify market-validated AI-native startups (funding above USD 1M plus evidence of real adoption), interview deep users of these products to recover authentic workflows, and have domain experts reconstruct these into benchmark tasks, each formalized as a triple T=(q, E, R) of a natural-language request, an input workspace, and a set of weighted fine-grained rubrics. Deliverables are then scored using an Agent-as-a-Judge protocol (AgentJudge) that evaluates each rubric item independently via its own judging session rather than a single holistic pass, run under a unified "Nanobot" agent harness with GPT-5.5 as the judge model.

Key results:

• StartupBench contains 97 tasks across 6 domains (Medical & Healthcare, Finance, Legal, Business & Management, STEM & CS, Education & Humanities), with an average of 25.3 fine-grained rubrics per task.
• The best model (Kimi-K3) reaches only a 73.67 average score but just a 29.55% success rate (score ≥90); GPT-5.6-sol scores 73.61 average / 31.27% success rate, showing large gaps between partial progress and full completion.
• Weakest domain is Finance (lowest average score, 54.48%), while Business is easiest (all models score above 60).
• Domain-Specific Compliance and Calculation Precision are the weakest rubric dimensions across all models, while Structure & Completeness and Output & Presentation are strongest.
• Rubric satisfaction rate drops as importance increases: 68.67% (Auxiliary) vs. 65.89% (Important) vs. 63.45% (Core), showing models often satisfy peripheral requirements while missing core task-defining ones.
• The rubric-wise AgentJudge achieves 92.78% rubric-level and 92.84% task-success-level agreement with expert human annotations, versus only 83% agreement for a holistic single-pass judge; on the 56 tasks with explicit output-format requirements, compliance ranges from 86.3% to 97.6% across models.

Why it matters / caveats: By deriving tasks from commercially validated startup products rather than researcher intuition, the benchmark argues that even frontier general-purpose agents remain far from producing professionally usable deliverables without further human verification, particularly due to weak complex-instruction-following and domain-specific expertise—explaining continued demand for specialized vertical AI agents.

Problem: Existing agent safety benchmarks target individual attack mechanisms or a limited subset of operational settings, making it hard to compare how safety failures emerge across the different responsibilities of an agent harness (the system that manages tools, extensions, state, permissions, and actions).

Method: The authors introduce a lifecycle taxonomy organizing agent harness safety into six operational phases—Harness Configuration, Capability Extension, Runtime Operation, State Persistence, Action Control, and Incident Recovery—and build HarnessRisk, a benchmark of 128 sandboxed cases (roughly evenly split across the six phases) where each case pairs a benign user objective, delivered over three sequential owner turns, with an adversarial instruction embedded in an untrusted workflow artifact. Every trajectory is scored on four binary metrics—Utility, Attack Success Rate (ASR), Persistence, and Detection—computed by a GPT-5.4 evaluator using the full interaction transcript, tool calls, state changes, and mock-service observations, and validated against deterministic predicates and human annotations. Evaluations run on three agent harnesses (OpenClaw, Hermes, Nanobot) with six language models across 14 model–harness configurations, each case run under three sampling seeds (384 trajectories per configuration).

Key results:

• Across 14 model–harness configurations, ASR ranges from 12.6% (GLM-5.2 on Nanobot) to 80.9% (Kimi K2.6 on OpenClaw), while Utility stays between 75.0% and 97.6%.
• The same model can vary >4x in safety by harness: GLM-5.2 has 54.7% ASR on OpenClaw vs. 12.6% on Nanobot; DeepSeek-V4-Pro ranges from 37.3% (Nanobot) to 65.4% (Hermes).
• Useful-but-unsafe trajectories account for 59% of runs on OpenClaw, 38% on Nanobot, and 43% on Hermes.
• Detection correlates strongly with lower ASR (Pearson r = −0.71, Spearman ρ = −0.77), but detection alone is insufficient: MiniMax M3 on OpenClaw detects risk in 97.9% of runs yet still has 31.2% ASR; GLM-5.2 reaches 92.2% Detection with 54.7% ASR.
• Harness Configuration is the most vulnerable of the six lifecycle phases on every harness tested.
• Evaluator agreement with independent references: 92.5% (κ=0.83) for Utility, 89.7% (κ=0.77) for ASR, 84.3% (κ=0.65) for Persistence, and 85.7% (κ=0.69) for Detection.

Why it matters / caveats: The results show safety is a joint property of the deployed model+harness configuration rather than the model alone, and that high task utility or explicit risk recognition can mask (or fail to prevent) substantial attack success—implying harness-level safeguards (provenance preservation, persistent-state protection, action gating, recovery verification) are needed beyond model-level alignment; all evaluations were conducted in isolated sandboxes with simulated services rather than real-world deployments.

Problem: Existing uncertainty quantification (UQ) methods for LLM agents rely on local signals (token probabilities, entropy, per-step confidence) or model trajectories as linear sequences, so they fail to capture how errors accumulate and propagate through the relational dependencies among reasoning steps, tool calls, and environment feedback across long-horizon agent execution.

Method: RUPA (Relational Uncertainty Propagation for Agents) represents an execution trajectory as a directed dependency graph with nodes for reasoning states, tool invocations, and observations, connected by seven relation types (sequential, latest, repetition, progression, parallel, feedback, goal alignment). Edge weights are learned from relation reliability, relation strength, and temporal decay, and uncertainty is propagated over the graph (with an exponentially decayed momentum term) and combined additively with each node's local uncertainty (entropy-based for assistant nodes, signal-based for environment nodes) to produce a trajectory-level risk score.

Key results:

• Improves average AUROC over the strongest baseline across 6 LLMs: 0.608→0.656 (Qwen3.5-27B), 0.629→0.645 (Qwen3.6-35B), 0.761→0.780 (Gemma4-26B), 0.842→0.861 (Gemma4-31B), and 0.694→0.718 (MiniMax-M2.7).
• In uncertainty-guided sampling on Terminal-Bench-2, RUPA raises Qwen3.5-27B task accuracy from 0.105 (random) to 0.213, and on GAIA from 0.261 to 0.297.
• Ablation: removing graph modeling drops AUROC from 0.718 to 0.678 and AUPRC from 0.805 to 0.642; a random graph topology gives similarly degraded AUROC of 0.681, confirming gains stem from meaningful dependency structure, not just added graph features.
• In entropy-matched low-uncertainty bins (Q1) where token-level confidence is uninformative (~0.5 AUROC), RUPA still achieves ~0.85 AUROC and ~0.93 AUPRC on GAIA/MiniMax-M2.7 trajectories.
• Preliminary study shows traditional UQ near-random on τ-2: sequence probability AUROC of only 0.205 (Airline) and 0.301-AUPRC-adjacent figure in the Retail domain; verbalized confidence reaches 0.485–0.523.

Why it matters / caveats: Demonstrates that explicitly modeling relational (not just temporal) dependencies is key to reliable, early failure detection in long-horizon LLM agents, with graph parameters calibrated in an outcome-blind way from unlabeled trajectories; however, absolute AUROC/F1 scores remain modest on harder benchmarks (e.g., ~0.5–0.6 on Terminal-Bench-2 for several models), and the approach relies on hand-designed lexical/embedding heuristics (via bge-m3) for edge-type detection rather than learned graph construction.

Problem: Standard flow matching for pixel-space image generation interpolates noise toward a fixed clean-image endpoint, leaving the coarse-to-fine spectral evolution (global structure before fine texture) to be learned implicitly, which makes high-dimensional pixel-space generation harder to optimize.

Method: Energy-Guided Flow Matching (EG-FM) replaces the fixed clean endpoint with a moving, heat-kernel-filtered endpoint y_t(x) that evolves from a low-pass version of the image to the full-spectrum image; the heat-kernel response R(h,ρ)=exp(-ahρ²) is controlled by a sample-adaptive heat-time h(x,t) solved via bisection so that each image's recovered spectral energy matches a shared global release clock q(t) (quintic smootherstep by default); the resulting endpoint motion t·∂t y_t(x) is added to the standard transport velocity y_t(x)-ε to form the regression target v_t, requiring no backbone or training-data changes.

Key results:

• ImageNet 256×256: PixelDiT-XL+EG-FM reaches FID 1.55 at 200 epochs (vs. 1.61 baseline at 320 epochs) and FID 1.45 at 600 epochs (vs. 1.54 baseline at 800 epochs) — roughly 4x faster convergence.
• DeCo-XL/16+EG-FM reaches FID 1.63 at 440 epochs vs. 1.69 baseline at 600 epochs; HyperDiT-H+EG-FM reaches FID 1.51 at 220 epochs vs. 1.56 baseline at 600 epochs.
• ImageNet 512×512 (40 epochs of continued training): PixelDiT+EG-FM gets FID 1.68/IS 295.5 (vs. baseline FID 1.81/IS 278.6 after 530 more epochs); HyperDiT+EG-FM reaches FID 1.58.
• Text-to-image (512×512): EG-FM-T2I achieves GenEval 0.85 (vs. 0.78 for PixelDiT-T2I) and DPG-Bench 83.9, best DPG among compared methods and second-best GenEval (0.01 below DeCo-XXL/16's 0.86).
• Ablations: sample-level adaptive heat-time gives FID 1.99 vs. 2.48 for shared-linear schedule; optimal σ0=3.5 (FID 1.99 vs. 2.36 baseline at 80 epochs); best CFG scale 2.55 giving FID 1.45.

Why it matters / caveats: The method is a drop-in trajectory/loss modification (no architecture or data changes) that consistently improves FID and convergence speed across three distinct pixel-space backbones (DeCo, HyperDiT, PixelDiT) and transfers to higher resolution and text-to-image settings; results rely on a specific heat-kernel/Parseval-based formulation and numeric bisection to solve for sample-adaptive heat time, adding some implementation complexity and requiring per-sample spectral computation during training.

Problem: Compute-optimal scaling laws are well established for large language models but remain largely unexplored for text-to-image diffusion models.

Method: The authors build Abra, a controlled family of dense flow-matching (v-prediction) diffusion transformers from 60M to 2B parameters (19 to 33 blocks, width 384 to 1792), using double-stream blocks followed by single-stream blocks, a frozen Qwen3-4B text conditioner, and the FLUX VAE latent space; hyperparameters are transferred across the ladder with µP (Complete-P parameterization) and validated via spectral coordinate checking. Models are trained on DataComp-1B (re-captioned with VLMs) across 10^19 to 10^22 FLOPs, deliberately overtrained up to 400 tokens-per-parameter (TPP) so the compute-optimal point falls inside the search range, and fit with power-law scaling analysis (iso-FLOP profiles, PCHIP interpolation of EMA loss).

Key results:

• Compute optimality occurs at approximately 200 image tokens per parameter (TPP), about 10x the Chinchilla-optimal 20 TPP for LLMs, with the optimum stable within ±17 TPP across different fitting procedures.
• Diffusion models tolerate 2x overtraining with less than 0.5% loss penalty, versus substantial penalties for LLMs (Gemstones suite, 48M–2B) at similar overtraining ratios.
• Generative metrics (FID, KID, CLIPScore, CMMD) all follow clean power laws in compute, but their compute-optimal TPP allocations diverge (FID/KID favor more data, CLIPScore/CMMD favor more parameters).
• Linear-probe (ImageNet top-1) representation quality also scales predictably, peaking at middle layers and diffusion timestep t≈0.7, but is compute-optimal at a much lower TPP than generation quality; the authors estimate a "double-point" where generation and understanding are jointly optimal at about 5×10^22 FLOPs and a 6B-parameter model.
• Compute-optimally trained models exhibit scaling collapse: rescaled training loss curves from different model sizes (60M/120M/250M, 4 seeds each) converge below the noise floor within the first 15% of training.
• Higher training resolution (256 to 768px, tested on 120M–500M models) requires more image tokens to reach compute optimality, consistent with reduced information density per patch at higher resolution.

Why it matters / caveats: The results give a concrete, actionable rule (~200 TPP) for allocating compute in text-to-image diffusion pretraining and show overtraining smaller models is a safe strategy for cheaper inference, unlike in LLMs where undertraining a large model is worse than overtraining a small one; findings are based on a single controlled architecture family up to 2B parameters and 10^22 FLOPs, so extrapolation to substantially larger frontier-scale models is not directly verified in the paper.

Problem: Autoformalizing natural-language math into Lean 4 requires more than translation—models must correctly map concepts onto Mathlib's type/definition hierarchy—but existing pipelines rely on parametric memory for library knowledge and use single-pass Best-of-N filtering that cannot diagnose or repair semantic errors.

Method: MathForm is a data-construction and training pipeline with a Retrieval Planner (driven by gpt-oss-120b) that issues targeted queries to LeanExplore to fetch relevant Mathlib definitions/theorems before generation, followed by a Formalization Generator that produces Lean 4 candidates; candidates pass through Format Check, Lean 4 compilation, and a semantic Consistency Check (QwQ-32B judge), with failures triggering up to three rounds of verification-guided iterative refinement rather than post-hoc rejection. The verified NL-FL pairs undergo trajectory reconstruction (removing proof/tactic content) to form the FormalVerse dataset, and MathForm-8B (Qwen3-8B base) is trained via SFT followed by RL using DAPO with a binary reward combining compilation success and semantic consistency (judged by gpt-oss-20b).

Key results:

• FormalVerse dataset contains ~367K verified Lean 4 NL-FL pairs; refinement rounds 2–3 recover an additional 31% of pairs beyond single-pass generation (69% round 1, 20% round 2, 11% round 3).
• MathForm-8B achieves average Pass@8 of 88.06% (Syntax Check) and 72.37% (Consistency Check) across six benchmarks, beating the strongest specialized baseline ReForm-32B (81.61%/68.41%) by 6.45/3.96 points.
• On FATE-M/H/X (frontier algebra benchmarks), MathForm-8B attains CC pass rates of 97.33%/63.00%/37.00%, exceeding the strongest baseline by 6/10/12 percentage points, with the gap widening at higher abstraction levels.
• RL improves MathForm-8B-SFT's average SC from 84.38% to 88.06% and CC from 66.53% to 72.37%; on FATE-H, Mean@3 pass rate rises from 0.30 to ~0.40 (about 33% relative gain) during RL training.
• Ablation on FATE series (gpt-oss-120b generator): full MathForm pipeline reaches 49.67% SC / 48.00% CC average vs. 27.33%/26.43% for single-pass generation, and beats the best single-component variant (BoN or retrieval-only) by 7.00/6.90 points; similar gains (+30.14 SC points) hold with a different generator family (Qwen3-235B-A22B-Thinking-2507).
• Trained on 100K-example controlled subsets, FormalVerse yields 60.32% average CC, beating FineLeanCorpus by 13.79 and NuminaMath-LEAN by 18.83 percentage points, while SC scores are comparable—indicating the advantage lies in semantic fidelity, not just compilability.

Why it matters / caveats: The results show that an 8B model with retrieval-augmented, verification-refined training data can outperform 32B specialized autoformalizers, especially on advanced algebra requiring deep library knowledge, suggesting data quality/construction methodology can substitute for parameter scale; caveats include reliance on LLM judges (gpt-oss-120b/20b, QwQ-32B) for semantic consistency, whose reliability tops out at F1≈0.89 on a 200-example human-annotated test set, and the RL stage uses a relatively small curated set of 3,000 examples.

Problem: DeepSeek Harness (DSH), an open-source plugin-based agent framework, needs a controlled, runtime-level assessment of how well it resists indirect prompt injection delivered through the untrusted content it reads (web pages, files, emails, skills) rather than through direct user prompts.

Method: The authors use A.I.G (AI-Infra-Guard) to build a source-to-sink test pipeline: it constructs attacker payloads, embeds them in 16 indirect content channels via a taint router, drives DSH's real TypeScript agent loop through a DSHRealHarnessAdapter (fixture source tools and locally-logged sink tools with no external side effects), and collects normalized traces. Each trace is scored by two evaluators — JR (RuleJudge), a deterministic evidence-based checker, and JL (LLMJudge), a semantic LLM-based judge — yielding full success, partial compliance, or failure/not-reached verdicts.

Key results:

• 14,560 controlled executions across 1,120 base cases (16 channels x 2 carrier modes x 35 payload objectives), with 1 naive baseline and 12 attack methods.
• Overall full-success rate: 5.6% (819/14,560) under JR, 5.3% (772/14,560) under JL; partial compliance is 2.0% under JR vs 7.3% under JL, giving broad-influence rates of 7.6% (JR) and 12.6% (JL).
• Strongest attack results: fake-completion reaches 16.6% JR / 17.0% JL full success in text mode (vs. 5.5%/5.7% for the naive baseline); hidden Unicode reaches 25.5% JR full success in file mode but 0.0% in text mode; the skills channel reaches 14.3% (text) and 16.0% (file) JR full success, and load_skill as a source tool reaches 15.2% JR / 13.7% JL.
• File mode overall outperforms text mode: 6.2% vs 5.1% JR full success, with more sink calls fired (387 vs. 254).
• Output-only canary objectives reach 35.7% JL full success versus only 2.5% for sink-required objectives, showing output compliance and actual sensitive-action success are very different risks.
• Obfuscation shows the largest judge disagreement: 13.6% JR full success in both modes but only 9.1%/8.8% under JL, indicating mechanical rule matches (sink call, token, argument) don't always reflect true semantic task completion.

Why it matters / caveats: The study shows indirect injection risk in DSH is not one bad prompt but a full pipeline issue spanning tool-result ingestion, carrier parsing/representation, and tool-call authorization — with skills/plugins and file-format parsing (e.g., hidden Unicode) as especially high-risk, underexplored vectors requiring provenance and privilege controls. All sinks in the experiment are simulated local fixtures (no real emails, commands, or funds moved), so reported percentages describe this controlled DSH configuration only and are not claimed as universal vulnerability rates.

Problem: Pre-optimized 3D Gaussian Splatting reconstructions are unordered, spatially irregular, and vary widely in primitive count (raw 200 m × 200 m aerial tiles can exceed 3.0 million primitives), so they cannot be fed directly into the fixed-channel structured spatial tensors that scalable latent 3D generators require.

Method: GS-Voxel deterministically voxelizes a compatible pre-optimized SH0 3DGS reconstruction into sparse active voxels without per-scene optimization, retaining sub-voxel positions and attributes via top-Kα primitive selection and 8-bit quantization within a fixed local slot budget (Kin) per voxel. A two-stage factorized VAE — a Geometry VAE (hierarchical subdivision/occupancy) and a Local Attribute VAE (per-voxel Gaussian attributes, Kout slots) — then encodes these into compact sparse latents at (R/8)³ resolution. Image-conditioned generation uses a three-stage flow-matching pipeline built on TRELLIS.2 (sparse-structure, geometry, and attribute flows with 0.3B-parameter DiTs), with overlap-aware BFS-spiral tiled inference and repaint-style inpainting enabling large-area synthesis beyond a single training crop.

Key results:

• At default settings (R=256, Kin=16), direct GS-Voxel conversion achieves PSNR 40.04, SSIM 0.98, LPIPS 0.023 on 10 validation tiles, retaining ~1,602K of ~3M+ primitives across ~377K active voxels.
• Local Attribute VAE ablation: Kout=4 gives best reconstruction (PSNR 23.09, SSIM 0.62, LPIPS 0.331) vs. Kout=1 (22.81/0.61/0.350) and Kout=8 (22.12/0.58/0.354).
• Factorized VAE beats a single-stage joint baseline: Geometry VAE reaches 0.99 Shape IoU vs. 0.76 for the single-stage model; Local Attribute VAE PSNR 23.09 vs. 21.01 for single-stage.
• Tile-level generation FID of 28.0 and KID of 0.020 (15K generated vs. 15K held-out ground-truth images), compared to cross-paper references CityDreamer (FID 97.3), GaussianCity (86.9), and EarthCrafter (69.5) — though protocols differ and are not directly comparable.
• Dataset: ~18K training samples (200×200 m² each) from large-scale aerial 3DGS scenes, with a virtual camera rig generating ~3,900 candidate views per window, filtered via alpha coverage (τα=0.9) and Qwen-VL-Plus scoring down to ~500–1,000 reliable views per window.
• Large-area synthesis demonstrated up to 1,400 m × 800 m, well beyond the 200 m × 200 m training crop.

Why it matters / caveats: The fitting-free approach avoids expensive per-scene refitting or global Optimal Transport rearrangement used by prior methods (e.g., GaussianCube, Can3Tok), letting latent capacity scale naturally with occupied voxels rather than a fixed scene-wide primitive budget — enabling generation of aerial scenes with millions of unevenly distributed Gaussians. Caveats acknowledged by the authors: the current implementation is restricted to SH0 (view-independent color) aerial scenes; top-Kin capacity can discard primitives in extremely dense voxels and struggle with very thin structures; robustness to real vs. rendered vs. generated conditioning images is untested; and the paper does not yet report entropy-coded bitrates or rate–distortion analysis for storage efficiency.

Problem: State-of-the-art AI co-scientist and auto-research systems are researcher-agnostic, producing the same research package for a given goal regardless of who the requesting scientist is, which erases the tacit knowledge, prior work, and network position that drive genuinely novel ideas.

Method: The paper formalizes "personalized auto-research," conditioning every stage of the research pipeline on a graph-derived researcher context c_u = Φ(S_u, z_u), where z_u is a representation learned from the researcher's position in a heterogeneous research graph G (co-authorship, citation, publication, affiliation, usage, and topic-assignment edges). It proposes Algorithm 1, a personalized agentic tree-search loop that modifies the SOTA auto-research procedure (experiment manager, code execution, vision-language figure review, manuscript writing) by adding personalized retrieval R_u(g), a user-conditioned hypothesis utility U(h|g,u) = α·Nov(h,I,c_u) + β·Rel(h,g,c_u) + γ·Feas(h,W0,c_u), and optional in-loop researcher feedback that updates c_u across runs. The framework is organized around three components: graph-grounded researcher representation, personalization across the full pipeline (retrieval, hypothesis search, experimentation, writing, citation, review), and evaluation grounded in the individual, and it is extended to team settings by replacing u with a team T and aggregating member contexts via c_T = Ψ({(S_u, z_u)}).

Key results: Not stated (this is a position/framework paper with no implemented system, experiments, or benchmark numbers reported).

Why it matters / caveats: The paper's contribution is conceptual — a formal problem definition, an algorithmic template, and an open-problems agenda (creativity collapse/monoculture at population scale, cold-start dependence on career stage, team personalization, and evaluation without ground truth) rather than an empirical system; it explicitly notes that building benchmark infrastructure for feasibility, expert-assessed quality, and longitudinal impact evaluation remains a major missing piece.

Problem: LLM-agent workflows for authoring 3D content as programs remain brittle because existing programmatic interfaces force agents to reason about fragile low-level numeric coordinates instead of the semantic structure and spatial relations LLMs are actually good at.

Method: The authors jointly design aDSL, a Python-embedded domain-specific language built on a hierarchical Asset container with named part attachments, CSG primitives/booleans/transforms, and declarative spatial-reasoning operators (e.g., align_anchors, align_centers, distribute) that expose AABB-based relational queries instead of raw coordinates; on top of this they run a training-free, role-specialized multi-agent system (Planner, Coder, Executor, Debugger, Critic) following a Plan-Execute-Critic self-refinement loop, where the Planner emits a hierarchical decomposition plus a verifiable constraint checklist, the Coder synthesizes the aDSL program, an Image Critic and Code Critic jointly verify renderings against the same program structure, and a selective memory mechanism separates persistent constraints from pruned transient state. The backbone LLM for all agents is Gemini 3 Pro.

Key results:

• On text-to-shape (100 objects x 2 prompts = 200 prompts from ShapeNet/ABO/Objaverse), aDSL achieves the best code-generation CLIP/VQA/success scores, e.g. ShapeNet: 29.63 CLIP / 65.34 VQA / 1.00 success, versus Scene Language (28.35/59.13/0.97) and ShapeCraft (27.70/57.26/1.00).
• On image-to-shape (30 Toys4K objects), aDSL reaches 84.42 CLIP / 184.71 FID / 1.00 success, best among code-generation baselines (ShapeCraft 79.34/187.62/1.00, Scene Language 78.68/206.21/0.93).
• Human study (38 participants, 20 cases vs. Scene Language): aDSL preferred 85.39% of the time for prompt alignment and 86.84% for geometric/visual quality.
• Ablations: removing spatial-reasoning utilities raises average refinement rounds from 4.25 to 4.67 (VQA 65.34→63.75); swapping aDSL for raw Blender scripting raises rounds to 6.08; removing the Planner raises rounds to 5.58; disabling self-correction drops success rate to 0.98 and VQA to 61.53; removing both spatial utils and refinement drops results furthest to 28.20 CLIP / 59.25 VQA / 0.97 success.
• Efficiency: ~190s per refinement round and ~4.7 rounds on average (~889s total per object) with >95% of runtime spent on LLM calls; memory reuse cuts a follow-up "cyber-punk motorcycle" edit from 5 rounds/845s to 1 round/164s.

Why it matters / caveats: The joint agent-language co-design—giving agents relational/declarative operators instead of raw coordinates—directly targets a known failure mode in programmatic 3D generation and also enables downstream uses like articulated (URDF-exportable) assets, localized shape editing, and scene composition combined with external generators (e.g., SpaceControl+Trellis) for higher fidelity. Caveats stated by the authors: output quality is bounded by the DSL's primitive expressiveness, the Critic relies on 2D renderings and can suffer perspective ambiguity, and the system depends on strong proprietary LLMs (Gemini 3 Pro), limiting accessibility until distilled into open models.

Problem: Engram-style hashed external memory stores learned knowledge in an addressable table read through a small learned interface, but it is unclear whether that frozen table is genuinely portable knowledge or just a co-adapted extension of the specific backbone that trained it.

Method: The authors formalize cross-model frozen-memory extraction: a source model trains an Engram memory table (deterministic n-gram hashing into an external table), the table is frozen and attached to a different target backbone, and only a lightweight target-side reader is trained via next-token prediction while both memory and backbone stay frozen. A tokenizer-agnostic canonicalization scheme (NFKC normalization, lowercasing, accent stripping, word-boundary n-gram hashing) keeps memory addresses stable across differing tokenizers, and the reader family is a shared-value, multi-branch, multi-layer design with context-aware sigmoid gating injected via residual connections (best configuration: dual-layer injection at layers {2, 10} with R=4 branches).

Key results:

• Full 3×3 source–target transfer matrix (Pythia-160M/Qwen3.5-0.8B/Qwen3.5-9B → Pythia-410M/Qwen3.5-4B/TinyLlama-1.1B) shows relative perplexity reductions in all 9 cells, ranging from 1.6% to 15.7% (strongest: Qwen3.5-0.8B → TinyLlama-1.1B).
• On Mistral-7B-v0.3 QA (NQ, WebQA, TriviaQA, TruthfulQA, HotpotQA), the strongest reader (LLaMA {2,10}-R4) reaches 38.5 average accuracy vs. 32.1 base, 33.4 for RAG, 32.6 for kNN-LM, and 36.1 for MLP Memory — a new SoTA among compared methods.
• Reader placement alone (single-layer {10}-R1 → dual-layer {2,10}-R1) raises average accuracy from 34.2 to 37.5; adding branches (R1→R4) raises it further to 38.5, without changing the transferred memory.
• Cross-model transfer nearly matches same-model reuse under the strongest reader (38.5 vs. 38.5 average accuracy), and scaling training budget from 10/10 to 30/30 tokens (millions) raises average accuracy to 38.8.
• Memory-only (no trained reader) and permuted-key controls collapse performance (e.g., Mem-only Mistral drops to 7.0 average, -78.1% vs. base), confirming the reader, not raw memory presence, drives usefulness.
• Downstream gains are task-selective: RTE improves +0.7 to +3.7 points and SciQ up to +3.7 points across model scales, while TruthfulQA consistently degrades by -0.2 to -0.8 points.

Why it matters / caveats: The work shows external memory tables can be genuinely reused across model families and tokenizers, but only when paired with a sufficiently expressive target-side reader — reader design, not memory content alone, is the first-order determinant of transfer quality. A key caveat is that benefits are task-dependent: transferred memory helps factual/evidence-oriented QA but is neutral or mildly harmful on calibration-sensitive tasks like TruthfulQA.

Problem: Byte-level hierarchical language models reduce sequence-processing cost by compressing bytes into latent tokens, but still decode one byte at a time at inference, leaving generation slow.

Method: The paper introduces Latent Causal Attention (LCA), a multi-byte prediction (MBP) method that ties parallel byte generation to the variable-length segments already produced by a hierarchical LM's boundary predictor (built on the FlexiTokens architecture), using a single multi-byte decoder head with a boundary-aware "LCA mask" that lets each byte attend to itself, earlier bytes in its own segment, and all bytes of the immediately preceding segment, while masking later positions—preserving causality without needing one prediction head per future token as in standard multi-token prediction (MTP). At inference, bytes are accepted left-to-right either via a probability threshold τ or via speculative-decoding verification (against LCA's own next-byte head or an external model), and the number of speculative candidates n can be varied at test time without retraining.

Key results:

• 373M-parameter models pretrained on 50B bytes from FineWeb-edu (3× compression rate); LCA-MBP lies on the Pareto front (performance vs. throughput) in 3 of 4 downstream tasks (IFEval, CoQA, CNN/DailyMail).
• LCA-MBP has a lower byte acceptance rate than MLP-MBP (e.g., 45.95% vs 61.91% on IFEval) but achieves better downstream quality at comparable throughput.
• Lowering acceptance threshold τ from 0.9 to 0.7 raises acceptance rate from 50.1% to 56.7% and throughput by ~10%, at a cost of ~3 points average performance.
• Under speculative-decoding verification, increasing candidates n from 3 to 7-8 yields +29% to +37% throughput gains with performance held constant (guaranteed identical to next-byte-only output).
• Mean of 3.05 out of 6 speculative bytes accepted per decoding step, with 15% of steps accepting the entire candidate window.
• Using LCA as a drafter with an external FxT model as verifier matches FxT's downstream performance while achieving 1.4-1.7× speedup (e.g., 1.74× on DailySum, 1.56× on es-en, 1.42× on fr-en), despite both models being the same size.

Why it matters / caveats: LCA removes the per-token parameter overhead of standard MTP heads and shows any hierarchical byte-level model can pair with an LCA counterpart as a drop-in accelerator without a smaller draft model. Caveats: experiments are limited to a single 373M-parameter scale and one training corpus, evaluation covers only English-centric/English-paired tasks, and the optimal speculative window n is task-dependent under threshold acceptance.

Problem: Scaling dense CLIP-style vision encoders improves vision-language model performance but increases compute cost and inference latency, and prior Mixture-of-Experts (MoE) attempts at vision encoders have not closed the gap to state-of-the-art dense models.

Method: MoE-ViE replaces the MLP blocks in a CLIP vision tower (except the first transformer block) with fine-grained MoE layers—32 experts per layer at 1/4 the hidden width of the dense MLP, with shared experts always active and sigmoid-based top-k routing—balanced via a z-score variant of loss-free balancing (replacing the sign(·) bias update with a normalized magnitude update) and served through a custom Triton kernel using Grouped GEMM and kernel fusion; a frame-level distillation loss against a frozen image-pretrained teacher plus freezing of MoE experts and text-tower MLPs is used during video finetuning to prevent catastrophic forgetting of image capabilities.

Key results:

• On zero-shot video benchmarks, MoE-ViE-H/14 (3.5B total / 1.1B activated params) matches or exceeds PEcore G/14 (1.9B activated params), e.g. 76.5% vs 76.0% on K400 classification.
• On ImageNet-A (adversarial), MoE-ViE shows +5.6% over PEcore B, +0.2% over PEcore L, and +0.6% over PEcore G despite PEcore G having 1.7× the activated parameters.
• The custom Triton MoE kernel gives >2.5× speedup over a vanilla MoE implementation, bringing MoE-ViE-H/14 latency to ~76% of PEcore G/14's latency (e.g. 82.59ms vs 101.21ms at batch size 16 for 576 tokens) while matching accuracy.
• Aligned with Llama 3.1 Instruct 8B, MoE-ViE-H (1.1B activated params) outperforms baselines including PEcore G/14 (1.9B) and InternViT2.5/14 (5.5B) on average image (75.2 vs 69.2/68.1) and video (58.9 vs 48.6/48.7) scores.
• The proposed magnitude-aware (z-score) loss-free balancing outperforms both auxiliary-loss methods and the original loss-free method (e.g. 63.2% vs 62.6% avg. classification accuracy for MoE-ViE-B/32).
• Across scales (B/32, L/16, H/14), MoE-ViE consistently outperforms both dense counterparts and conventional (non-fine-grained) MoE under matched compute budgets.

Why it matters / caveats: The results suggest fine-grained MoE combined with careful load balancing and a dedicated kernel can deliver dense-level or better accuracy at meaningfully lower latency for vision encoders, though total parameter counts remain much larger than activated counts (e.g. 3.5B total vs 1.1B activated for MoE-ViE-H), and captioning results trail the best dense baselines (second only to SigLIP2-g-opt among sub-2B-activated-parameter models).

Problem: Graph neural network architectures are described through family-specific notation (message passing, graph networks, spectral filters, attention, geometric equivariant models, etc.) that obscures which computations are shared and which are genuinely different, making architectures hard to compare or systematically search.

Method: The authors propose a common layer equation, H̄^(ℓ+1) = φℓ(Bℓ(H^(ℓ), H^(0)), ⊞_{k∈K} C_k^(ℓ)), factored into seven fixed components: update domain X, channel set K, propagation bank {Pk} (where information moves), per-channel message maps {Ψk} (what moves), a channel-fusion operator ⊞, an ego/residual map Bℓ, and an update map φℓ. A fixed "slot discipline" assigns every elementary operation to one of these seven roles, and the equation is validated through worked reductions of GCN, GraphSAGE, GAT, and GIN plus representative fillings spanning seven architectural families (spatial, attention, spectral, Graph Transformer, heterogeneous, higher-order, and geometric).

Key results:

• Organizes more than 200 catalogued architectures drawn from over 400 cited works into the seven-component taxonomy (Table 1 illustrates 21 canonical methods across the seven families).
• Derives 17 component-level theoretical results (16 with proofs in the appendix), including: under endpoint-local messages and node-local updates, operator support in Pk bounds one-layer dependencies, and a full effective operator row is necessary for one-layer global mixing.
• In the linear additive regime, raw/displayed channel count is proven non-identifiable, whereas the minimum Kronecker separation rank of the summed layer operator is an invariant.
• Using a language-model controller to search the structured design space (3 generation levels: filler recombination, new filler, or equation expansion), 6 generated architectures survive the validity audit — 5 at Levels 1–2 (e.g., Spectrally-Gated Propagation, Relation-Equivariant Network) and one Level-3 architecture (Sequential Co-Lifted Node–Edge Layer).

Why it matters / caveats: The framework gives a shared vocabulary for locating exactly which component two architectures differ in and for connecting propagation choices to oversmoothing, oversquashing, heterophily, and expressivity, but it is a representational/theoretical unification rather than a new trained model — no new architecture is empirically benchmarked, and the paper explicitly states that selecting the best component filling for a given dataset remains an open, unresolved empirical inverse problem.

Problem: Existing cardiac foundation models are trained separately for a single sensing modality (ECG, PPG, or PCG), so the shared underlying cardiac physiology across these modalities is never exploited despite temporal offsets that make naive cross-modal alignment misleading.

Method: CardioState-JEPA is a physiology-aware joint-embedding predictive architecture that maps ECG, PPG, and PCG waveforms into a common token space via lightweight modality-specific stems (Conv1d multi-scale depthwise stems) and processes them with a single shared ViT-B Transformer encoder (12 layers, 12 heads, d=768). It trains in two stages: Stage I performs intra-modal masked latent prediction (JEPA-style, EMA teacher, Smooth-L1 loss) on abundant unimodal data; Stage II adds delay-aware cross-modal JEPA using a learned delay aligner (a delay head producing a bounded per-token offset via tanh, Gaussian-kernel target gathering, and entropy-based confidence weighting) supervised by physiological anchors such as R-peak-to-S1 interval and pulse arrival time, plus auxiliary VICReg-based state-alignment and cardiac-phase-prediction losses.

Key results:

• On PPG: average classification AUROC rises from 72.2 (best baseline AnyPPG) to 80.4, and average regression MAE drops from 10.9 to 9.1.
• On ECG (18 settings across 6 datasets/label fractions): average AUROC of 84.1 vs. 68.5 for the strongest self-supervised-only baseline (MoCo-v3), a 15.5-point gain, and it is competitive with label-supervised (ECGFounder, 83.5 avg) and text-supervised (D-BETA 85.9, ECG-FM 84.0) models at 84.1 avg despite using no privileged labels/text.
• On PCG: CirCor murmur detection AUROC of 97.9±0.1 (vs. 79.1±2.7 for AudioMAE, an 18.8-point gain) and CinC2016 abnormal heart-sound detection of 66.8±0.4 (vs. 62.4±0.7 for AudioMAE).
• Modality silhouette score of pooled codes falls from 0.121 after Stage I to −0.006 after Stage II, showing cross-modal training removes sensor-specific clustering while downstream performance still improves.
• Ablations: removing cross-modal prediction drops ECG avg to 83.8, PPG classification avg to 63.9, and PCG avg to 78.4; removing delay modeling drops ECG avg to 85.3 and PPG classification avg to 68.9.
• Pretraining on all three modalities together (82.3 PCG avg) outperforms PCG-only pretraining (60.2 avg), showing cross-modal supervision benefits even the modality with the least data.

Why it matters / caveats: It demonstrates that heterogeneous cardiac sensors can mutually supervise one shared foundation model rather than requiring per-sensor pretraining, with a single frozen encoder transferring across 25 downstream tasks; however, on ECG it still slightly trails the best text-/label-supervised models on some PTB-XL subsets (e.g., 90.1 vs. 89.1 on PTB-XL Super 100%), indicating privileged clinical annotations still offer a small edge in some settings.

Problem: Unified image restoration methods that adapt pretrained text-to-image latent diffusion models suffer from VAE-induced loss of fine details and open-ended synthesis priors that introduce content-inconsistent artifacts.

Method: PixRestore is a VAE-free pixel-space Diffusion Transformer (DiT), built on the LightningDiT backbone, trained from scratch with flow matching directly on patchified RGB pixels (patch size 8) instead of a latent space. It conditions on multi-layer DINOv2 features fused by a learned adaptive layer router (trained via LQ-HQ feature-similarity supervision) injected through cross-attention, with a hierarchical feature loss that supervises less-reliable layers more strongly. The multi-step model is then distilled into a single-step generator using DINO-based adversarial (GAN) finetuning.

Key results:

• Pixel DiT-S beats latent DiT-S baselines (SD2VAE/FluxVAE/QwenVAE) with 26.62 dB PSNR / 0.1593 LPIPS / 54.32 MUSIQ vs. best latent baseline 22.80 dB / 0.2181 / 51.87, using only 23.41M params.
• Full PixRestore (~53.7M params incl. frozen DINO encoder, 1 NFE) runs at 44 ms/image and 658 GFLOPs, about 7-23x faster than PromptIR/DiffUIR/FoundIR/FAPE-IR, and only ~1/181 the FLOPs of FoundIR-v2 and ~1/1209 of Flux-IR.
• On 8 public UIR benchmarks, PixRestore/PixRestore-B rank first or second on most PSNR/SSIM/LPIPS/DISTS/DR-Score metrics against PromptIR, DiffUIR, DA-CLIP, FoundIR(-v2), UniRestore, Flux-IR, and FAPE-IR (e.g., de-rainstreak: 32.28/32.85 dB PSNR vs. best baseline FAPE-IR* 31.91 dB).
• Single-step finetuning (A10) improves PSNR from 28.07 to 28.49 dB, LPIPS from 0.1202 to 0.1120, and MUSIQ from 53.34 to 55.52 over the 1-NFE multi-step ablation, while keeping 1-step inference.
• Flow pretraining + one-step finetuning outperforms direct regression training under equal iteration budget: 28.49 dB PSNR / 0.8589 SSIM / 0.1120 LPIPS / 55.52 MUSIQ vs. 27.00 / 0.8179 / 0.1494 / 52.00.
• Adaptive hierarchical DINO guidance (multi-layer, learned weighting) improves over uniform 6-layer averaging: PSNR 27.66 vs. 27.36, LPIPS 0.1209 vs. 0.1239.

Why it matters / caveats: Shows that a compact (~50M parameter), VAE-free pixel-space diffusion transformer trained from scratch can outperform much larger T2I-pretrained UIR models (some with over 16B-21B parameters) in fidelity, perceptual quality, and speed; the authors also note standard no-reference metrics (MUSIQ, AFINE-NR) fail to reliably measure degradation removal, motivating their VLM-based DR-Score, and larger PixRestore variants (B/L/XL) show the design scales further.