AI papers — 2026-07-13

Problem: Existing terminal-agent benchmarks (e.g., Terminal-Bench, SWE-Bench) evaluate only short tasks with binary pass/fail outcomes, which understates the difficulty of real long-horizon workflows and gives sparse reward signals that can't distinguish an agent that nearly finishes from one that fails immediately.

Method: The authors build Long-Horizon-Terminal-Bench (LHTB), 46 containerized terminal tasks across nine categories (experiment reproduction, software/reverse engineering, multimodal analysis, interactive games, scientific computing, etc.), following the Terminal-Bench/Harbor task format (natural-language instruction, Docker image, task config, oracle solution) but decomposed into fine-grained graded subtasks with weighted, deterministic checks (binary, continuous/thresholded, and episode-aggregating) to compute a normalized dense reward R. Tasks were constructed by seeding real professional workflows (some adapted from APEX-Agents) as deliberately broken projects with public low-weight checks plus hidden high-weight stress tests (schema variations, noise, rotated/cropped frames, etc.), calibrated for difficulty by repeatedly running DeepSeek-V4-Pro under a 1.5-hour budget and filtering 120 candidates down to 46. The authors evaluate 15 frontier models (GPT-5.5, GPT-5.4, GPT-5.3 Codex, DeepSeek V4 Pro, Gemini 3.1 Pro, GLM 5.1/5.2, Kimi K2.6/K2.7 Code, MiniMax M3, Qwen3.7 Max/3.6 Plus, Doubao Seed 2.1 Pro, Hy3, Grok 4.20) using the Harbor framework with the Terminus-2 agent scaffold (Codex harness for GPT-5.3).

Key results:

• Average per task across 15 models: 231 episodes, 9.9M tokens, 85.3 minutes wall-clock (90-minute timeout), ~$10.2 (cost table shows average 228 episodes, 85.1 min, $10.21/task).
• Best model GPT-5.5: 15.2% pass@1 (7/46) at R≥0.95, 10.9% (5/46) at R≥1.0, 17.4% (8/46) at R≥0.9.
• Mean pass rate across all 15 models: 4.3% at R≥0.95, 1.7% at R≥1.0.
• MiniMax M3, Kimi K2.7 Code, DeepSeek V4 Pro: 6.5% (3/46) each at R≥0.95.
• Mid tier (Qwen3.7 Max, Doubao Seed 2.1 Pro, Gemini 3.1 Pro, GLM 5.1, GPT-5.3 Codex): 4.3% (2/46).
• GLM 5.2, Qwen3.6 Plus, GPT-5.4, Hy3: 2.2% (1/46) each; Kimi K2.6 and Grok 4.20: 0% (0/46); Grok 4.20 lowest mean reward (R=0.08).
• Across 15×46=690 runs: only 30 runs (4.3%) pass R≥0.95; 227 runs (32.9%) show no meaningful progress (R<0.05); 433 runs (62.8%) achieve partial reward, with 180 runs (26.1%) reaching R≥0.5.
• Near-misses (0.75≤R<0.95) occur more than twice as often as passes: 73 vs. 30.
• Pass rate and mean reward correlate only moderately (Spearman ρ=0.56).
• 79% of unresolved runs (518/660) are timeouts (mean reward 0.10–0.35); 19% are early exits; 3% are harness errors.
• 14 "false finish" runs identified (early exit at R≥0.75), e.g., Kimi K2.7 Code stops at R=0.92 on duckdb-optimizer-closure; GLM 5.2 stops at R=0.90 on apex-ib244-matter.
• Cost range ~$2.5 (Hy3) to ~$28/task (GPT-5.4, despite lower pass rate than GPT-5.5 at ~$21/task); Pareto frontier includes GPT-5.5, MiniMax M3, Doubao Seed 2.1 Pro, Hy3.
• GPT-5.4 needs 302 episodes/task vs. GPT-5.5's 208.

Why it matters / caveats: Dense subtask-level grading reveals substantial partial progress and distinct failure modes (timeouts vs. premature/overconfident stopping) that binary pass/fail evaluation would completely obscure, showing that long-horizon task completion and self-verification — not just local step correctness — are the primary bottlenecks for current frontier agents; the paper notes the benchmark and evaluation harness are being released to support further research.

Problem: Current foundation-model pretraining discards the visual information in scientific documents (figures, equations, tables, page layout) by converting them to plain text before training, causing a loss of reasoning-relevant structure.

Method: The authors propose Visual Pretraining (VP), which renders raw document pages as images, extracts frozen ViT features (27-layer ViT-style encoder, hidden size 1152, 16 heads, patch size 16, spatial merge size 2), filters out blank/background patches via a foreground mask based on pixel variance and luminance, and orders remaining patches in raster scan. These sparse visual tokens are projected into the LLM's hidden space and the shared autoregressive backbone (e.g., Qwen3.5, Qwen3, Llama3.2 Vision, Llama3.1) is trained with a causal "next visual latent prediction" objective using an InfoNCE contrastive loss (temperature τ=0.07) against frozen visual targets, jointly with standard text next-token cross-entropy (λ_vis=0.1) during continued pretraining (CPT), followed by identical SFT. VP is compared against a matched Text Pretraining (TP) baseline where the same PDFs are parsed to text via MinerU2.5.

Key results:

• VP uses only ~20B visual tokens vs. ~80B text tokens (TP) from the same PDF corpus (total CPT budget: 120B for VP vs. 180B for TP) — roughly 25% of the token budget.
• GPQA Diamond improves up to 3.22 points over TP (e.g., 76.24 → 79.29 on Qwen3.5); MMLU-Pro improves up to 2.1 points; HLE improves only marginally (up to 0.97 points).
• Qwen3.5: MMLU-Pro 83.91→85.09, GPQA 76.24→79.29, AIME 50.60→51.52, HLE 30.24→33.08 (TP→VP).
• Normalized scaling gains over TP-vs-base: 1.27x on MMLU-Pro, 2.02x on GPQA, 2.88x on AIME-25.
• Multimodal benchmarks (Qwen3.5): MMMU-Pro 72.14→73.87, ChartQAPro 56.42→61.80 (+5.4 pts vs TP), MathVista 85.50→86.70; Llama3.2 Vision ChartQAPro 22.23→27.67, MathVista 39.60→44.40 (+4.8 pts vs TP).
• Cross-modal alignment (100 image-text pairs): centroid separation drops 1.665→0.661 (-60%), cosine similarity 0.631→0.907, Linear CKA 0.657→0.745, mutual k-NN@1 0.140→0.310.
• PPL-based image-to-text retrieval: R@1 64.0%→99.0%, MRR 78.2→99.5.
• Optimal visual token budget: 8,192 foreground tokens (1x setting) per batch offers best trade-off; lower resolution at fixed budget preserves performance.
• A generative-decoder variant (MAR decoder + frozen VAE, ~300M extra params) gives only marginal gains (e.g., GPQA 75.44 vs 75.95 decoder-free) while costing ~1.4x training time and 30-40% lower throughput — so the main method uses the decoder-free formulation.
• Main Qwen3.5-35B-A3B CPT run took ~36 hours plus ~10 hours SFT on a 128-accelerator cluster.

Why it matters / caveats: VP offers a more efficient and effective alternative to text-only continued pretraining for scientific reasoning and also improves multimodal alignment/transfer without any image-text pair supervision, but the authors note it is not fully independent of language pretraining (it's layered on top of text CPT), was tested mainly on high-knowledge-density scientific PDFs, and it remains open whether the approach generalizes to natural images or video.

Problem: Computer vision still lacks a unified, general-purpose foundation model analogous to LLMs — existing systems like Segment Anything and Depth Anything remain task-specific, requiring custom architectures, heads, and losses for each vision problem.

Method: The paper introduces GenCeption, which repurposes a pretrained text-to-video diffusion backbone (WAN 2.1, 1.3B and 14B variants) as a feed-forward perception model: the DiT is fed clean (noise-free) video latents with timestep fixed at t=0, its Rectified Flow velocity output is negated to recover x0-aligned features, and a single forward pass replaces the original 50-step iterative denoising. Dense tasks (depth, normals, segmentation, DensePose, camera raymaps) are unified by representing all outputs in a shared 3-channel RGB latent space (using a "Rothko" raymap layout for camera pose), while sparse tasks (2D/3D keypoints) are handled via learnable per-frame tokens decoded by an MLP; training uses a single unified L2 loss with task specification handled entirely through text prompts and data formatting rather than architecture changes. Training data is a synthetic corpus of 7,500 human-centric videos built from 800 RenderPeople assets animated with 200 CMU mocap motions, rendered in Blender for depth/normal/segmentation ground truth, supplemented with TartanAir, Virtual KITTI, MVS Synth (depth/camera) and MeViS, Ref-COCO, YouTube-VOS (expression-referring segmentation).

Key results:

• Trained on 480×832, 81-frame/24fps video, batch size 64 on 256 v6e TPUs, Adam lr=5e-5, 15,000 steps.
• Inference: 1.3B model runs at 13.6 FPS (5.92s, 15.3GB VRAM); 14B model at 8.0 FPS (10.03s, 42.8GB VRAM) on a single v6e TPU — eliminating WAN's standard 50-step diffusion.
• Matches or beats specialized SOTA models: surface normals beat NormalCrafter and Lotus-2 (e.g., Sintel mAE 29.3 vs Lotus-2's 30.3); depth/camera pose competitive with DepthAnything V3, D4RT, VGGT-Ω (e.g., Sintel AbsRel 0.130 for Ours-L vs 0.145 for DepthAnything V3); outperforms SAM3 on expression-referring segmentation on MeViS (69.0 J&F vs 64.5 for SAM3+Gemini 3.5 Flash); beats Genmo and TRAM on 3D keypoints (EMDB MPJPE 71.8mm vs 73.0mm for Genmo).
• Data efficiency: achieves comparable depth-estimation performance to D4RT and VGGT-Ω using 7x to 500x less training data (8.08K videos / 1.23M frames vs. VGGT-Ω's ~3M videos / ~600M frames).
• Pretraining comparison: WAN 2.1-based backbone substantially outperforms same-size V-JEPA and VideoMAE V2 backbones under identical fine-tuning data (e.g., AbsRel 0.201 for WAN-S vs 0.422 for V-JEPA-H on one benchmark split).
• Scaling: performance improves with both model size (1.3B → 14B) and data volume (1 → 7.5K videos), e.g., depth δ1 accuracy rises from 37.3% to 76.8% with more pretrained layers/scale.
• Ablation: training the DiT from scratch (no pretraining) yields a nearly flat learning curve, confirming pretraining is essential; joint multi-task training helps segmentation but degrades 3D keypoint estimation.
• Emergent behaviors: a model trained exclusively on synthetic single-human videos generalizes zero-shot to real multi-object scenes and out-of-distribution categories (animals, anthropomorphic characters), sometimes producing finer detail (e.g., cat whiskers, hair) than its synthetic Blender-rendered training data.

Why it matters / caveats: The results suggest that generative video pretraining itself (not just data/model scale) instills transferable spatiotemporal and vision-language priors, pointing toward a single unified backbone replacing today's zoo of task-specific vision models; however, the training data and evaluation are heavily human-centric/synthetic, joint multi-task training shows mixed-to-negative effects on some tasks (especially 3D keypoints), and the authors describe the scaling behavior as only "preliminary."

Problem: Standard On-Policy Distillation (OPD) for post-training LLMs is theoretically appealing (dense rewards, on-policy, avoids catastrophic forgetting) but suffers from severe optimization instability because the token-level reward (log probability ratio between teacher and student) diverges to negative infinity whenever the teacher assigns near-zero probability to a student-generated token.

Method: The authors propose Trust Region Policy Distillation (TOP-D), which (1) constructs a dynamic "proximal teacher" by interpolating teacher and student probability distributions with coefficient α (π̃ = απ + (1−α)πθ), which mathematically transforms the unbounded OPD reward log ρk into a smooth, lower-bounded reward r̃k = log(αρk + 1 − α); and (2) adds internal trust-region iterations (a PPO/GRPO-style clipped surrogate objective with importance ratios and token-level normalized advantages) to allow off-policy data reuse across multiple mini-batch epochs. The method requires no explicit teacher construction and is trained on the DAPO-Math-17k dataset using Qwen3-1.7B-Base and Qwen3-8B-Base as students and Qwen3-14B and Qwen3-30B-A3B-Instruct-2507 as teachers, compared against GRPO, DAPO, and standard OPD baselines.

Key results:

• With Qwen3-8B-Base student and Qwen3-30B-A3B-Instruct-2507 teacher: TOP-D reaches 50.42% avg@32 on AIME24 vs. 24.58% for standard OPD (+25.84 absolute), vs. 30.10% for GRPO and 32.92% for DAPO.
• On AIME25: TOP-D 34.06% vs. OPD 23.33% (+10.73); on AIME26: TOP-D 44.06% vs. OPD 25.42% (+18.64).
• On AMC23: TOP-D 88.13% vs. OPD 76.88% (+11.25); MATH-500: 91.23% vs. 87.98% (+3.25); OlympiadBench: 64.67% vs. 59.29% (+5.38).
• TOP-D exceeds DAPO by 17.5 absolute percentage points on AIME24.
• With smaller Qwen3-1.7B-Base student (30B teacher): TOP-D AIME24 avg@32 = 20.31% vs. OPD 8.96% (+11.35). Standard OPD actually falls behind RLVR baselines (GRPO 10.52%, DAPO 12.29%) at this smaller scale, while TOP-D still dominates.
• Ablations: setting α=1.0 (removing the proximal teacher) reproduces unstable OPD-like training; removing internal trust-region iterations causes much slower convergence; performance is robust across α ∈ {0.1, 0.2, 0.3}.
• Gradient variance is proven bounded and a convergence bound is derived, plus a monotonic improvement guarantee for the internal iterations.
• Training used 32 NVIDIA H200 GPUs (4 nodes × 8 GPUs); TOP-D adds zero additional computational overhead versus standard OPD.

Why it matters / caveats: TOP-D offers a theoretically grounded, drop-in fix for OPD's instability that yields large empirical gains on math reasoning benchmarks with no extra compute cost. Caveats: experiments are limited to student models up to 8B parameters, training runs were relatively short (~200–400 update steps, no observed saturation), and the theoretical bounds rely on idealized assumptions.

Problem: Existing compensation-based post-training quantization methods for LLMs (e.g., GPTQ, GPTAQ) build their quantization objective only from input activation covariance, implicitly treating all output channels as equally important and ignoring output-side gradient sensitivity.

Method: KronQ is a PTQ framework built on the Kronecker-factored Hessian approximation H ≈ H_X ⊗ H_G, where H_G is the gradient covariance estimated from a single backward pass over the calibration set; it keeps the GPTAQ solver exactly (with H_G canceling algebraically in the column-wise OBS update) while adding (1) bidirectional incoherence processing (BiIP) that extends QuIP#-style randomized Hadamard rotations and diagonal rescaling to the output dimension via H_G, and (2) an inter-layer mixed-precision allocation score tr(H_G)·tr(H_X). It is evaluated on LLaMA-2-7B/13B/70B, LLaMA-3-8B/70B, Gemma-3-12B, DeepSeek-R1-Distill-Llama-8B, and Phi-4-mini-instruct, using GPTQ/GPTAQ as primary baselines, WikiText-2 perplexity, QuaRot for W4A activation quantization, and seven zero-shot reasoning benchmarks plus GPQA-Diamond, MMLU, AIME-2024, and LiveCodeBench.

Key results:

• On LLaMA-3-70B at 2-bit weight-only quantization, GPTQ and GPTAQ diverge/degenerate (GPTQ: 2.6e3 PPL; GPTAQ: NaN), while KronQ achieves 7.93 WikiText-2 perplexity.
• LLaMA-2-7B W2 group quantization (g=128): GPTQ degrades to 274.00 PPL, GPTAQ to 23.19, while KronQ achieves 7.61 (avg zero-shot accuracy 53.7% vs. 39.3%/40.3%).
• LLaMA-2-7B W2A4: KronQ reduces PPL from GPTQ's 36.74 to 9.38 (GPTAQ: 10.91).
• Newer models (W2, WikiText-2 PPL): Gemma-3-12B-PT 60.10 (KronQ) vs. 1.3e4 (GPTQ)/269.0 (GPTAQ); DeepSeek-R1-Distill-Llama-8B 27.72 vs. 73.53/110.34; Phi-4-mini-instruct 19.26 vs. 111.42/31.07.
• At W4A16, DeepSeek-R1-Distill-Llama-8B LiveCodeBench: 37.3 (KronQ) vs. 16.3 (GPTAQ)/19.9 (GPTQ); Gemma-3-12B-IT MMLU: 70.02 vs. 65.21/64.82.
• Mixed-precision: KronQ's joint score achieves lower WikiText-2 PPL than W3 baselines (SliM-LLM, SliM-LLM+, CMPQ) using only ~2.6 average bits on LLaMA-2-7B.
• Inference: peak VRAM reduced 3.5–3.9× at W4 and 4.0–7.5× at W2 vs. bf16; a 70B model fits on a single A100 at W4 (35–39 GB) instead of two 80GB A100s; decoding latency improves 1.25–2.51× on 7B–13B models.
• Calibration overhead: +8–11 s/layer latency vs. GPTAQ, released after H_G cancels in the quantization loop.

Why it matters / caveats: By incorporating output-side gradient covariance (previously discarded) into both incoherence processing and mixed-precision allocation, KronQ substantially improves stability and accuracy at ultra-low bit-widths (2–3 bit) where prior activation-covariance-only methods diverge or degrade severely, while adding only modest calibration-time overhead and no extra inference cost.

Problem: Panoramic (equirectangular) world models struggle to maintain geometric, radiometric, and temporal consistency over long-range camera memory in complex real-world outdoor environments, because existing memory mechanisms (3D points, KV caches) inherit perspective assumptions and fail under panoramic distortion and rotation-induced viewpoint shifts.

Method: PanoWorld is a diffusion-based panoramic video generation framework built on the Wan2.2-5B backbone (LoRA fine-tuned), which exploits the rotation-equivariant property of equirectangular projections to decouple camera rotation from translation, using two new modules: Dense Panoramic Ray-Conditioning (DPRC), which unprojects pixels to spherical rays and injects translation-induced motion manifolds via Projective Positional Embeddings (PRoPE) into the DiT blocks, and Geometry-Aware Memory Augmentation (GMA), which retrieves historical features from a memory bank in a shared PRoPE ray space with confidence-guided gating/fusion. Training follows a three-stage pipeline (panoramic LoRA fine-tuning with a latitude-aware reconstruction loss, view-dependent motion learning, then memory-anchored coherence). The authors also build World360, a 120K-clip dataset (70K real UAV clips + 50K AirSim360 simulated clips), and extend the model to real-time generation via Causal Forcing/DMD distillation.

Key results:

• World360 dataset: 120,000 total sequences, larger/more diverse than prior panorama datasets (e.g., Argus 283k, 360-1M 1,076k, Matrix-Pano 116k).
• At 480p, PanoWorld achieves FID 27.64 (vs. Imagine360 81.18, Matrix-3D 34.63, OmniRoam 60.77) and best perceptual quality score among compared methods.
• At 720p, PanoWorld achieves FID 16.93 (vs. Matrix-3D 35.35, OmniRoam 44.89).
• Trajectory control (PSNR at various temporal windows, 480p): PanoWorld 22.83±3.73 down to 20.92±3.73, vs. Matrix-3D (20.47→18.02) and OmniRoam (18.51→17.02).
• Ablation on GMA: full model gives +1.07 PSNR gain over w/o GMA at the longest window; a Random Memory variant collapses to ~13.3-15.5 PSNR.
• Real-time extension via Causal Forcing/DMD distillation generates a 161-frame panoramic video in 8 seconds on a single NVIDIA H20 GPU, versus ~4 min 48 s for the full model, ~16.5 min for Matrix-3D, and ~31 min for OmniRoam.
• Real-world Scene-Reality dataset: 124 long-duration 360° panoramic videos with synchronized 6-DoF INS poses, over 6 million frames.

Why it matters / caveats: PanoWorld demonstrates that exploiting panoramic rotation-equivariance to simplify motion modeling and structure long-range memory retrieval yields substantially better geometric/radiometric consistency and trajectory control than prior panoramic world models, while enabling near-real-time generation; evaluation is based on the authors' own newly constructed World360 benchmark, and comparisons rely on baselines being adapted to the same protocol.

Problem: Existing generative/editing approaches to dense prediction (depth, normals, matting, segmentation, pose) reuse text-to-image pretraining by encoding task targets into an RGB-trained VAE latent space and decoding them back through a target-side generative interface, which the paper argues is unnecessary and indirect for tasks that need pixel-correct, task-native fields rather than rendered RGB-style content.

Method: The paper introduces ReChannel, built on the frozen FLUX-Klein text-to-image DiT backbone (4B and 9B variants). It keeps the pretrained VAE encoder only for encoding the input RGB image, runs the DiT in deterministic zero-noise (σ=0) mode with lightweight task-specific LoRA adapters inserted into the frozen backbone, and drops the target-side VAE decoder entirely. Each adapted spatial token is mapped directly to its pixel-space target patch via a shared, token-local linear readout head (~33K parameters, no spatial mixing or decoder), trained with per-task pixel-space regression/mask losses (depth, surface normals, alpha matting, referring segmentation, saliency, pose heatmaps). Referring segmentation reuses the referring expression as the backbone's text condition.

Key results:

• Monocular depth: ReChannel-9B achieves best absRel on KITTI (0.063) and ScanNet (0.047); trails Edit2Perc only on NYU (0.051 vs 0.044).
• Surface normals: ReChannel-9B best mean angular error on NYU (15.6°), ScanNet (13.9°), iBims (15.1°).
• Trimap-free matting: ReChannel-9B sets new SOTA on P3M-500-P (SAD 5.69) and P3M-500-NP (SAD 6.67); zero-shot AIM-500 SAD of 34.90 vs GenPercept's 75.5 (less than half).
• Referring segmentation: ReChannel-4B beats 7B–8B LLM-based baselines (LISA, GLaMM, Text4Seg) (80.3 vs 79.4 avg cIoU); ReChannel-9B reaches 82.0 average cIoU, best on all 8 RefCOCO/+/g splits.
• Pose: ReChannel-9B reaches 79.2 AP on COCO val, +0.9 AP over ViTPose-L.
• Saliency: ReChannel-9B best Fmax (DUTS-TE 0.944, ECSSD 0.968) and MAE (0.018, 0.017).
• Ablations: head-only frozen body collapses normals to 44.30°/43.55°/48.16° and matting SAD to 180.97/170.92; a 13× larger conv head and full fine-tuning both underperform the linear readout.
• Efficiency: ReChannel runs at 47.7 ms/image on a single L40S GPU at 512² (same as a bare backbone forward pass); the edit-paradigm baseline runs at 118.1 ms (2.48× slower) at matched accuracy.

Why it matters / caveats: The results suggest generative T2I pretraining can be exploited for dense prediction purely through its organized spatial token field, without inheriting the costly target-side VAE/generative rendering interface, yielding both higher accuracy and up to 2.48× speedup; the authors note the approach was validated only on FLUX-Klein and pixel-aligned targets.

Problem: Large language models with extended context windows still show degrading accuracy as input length grows because they struggle to identify and use the evidence most relevant to a question, and applying test-time training (TTT) naively (on the full context or on randomly sampled spans) is either too expensive or too noisy to reliably help.

Method: The authors propose Self-Guided TTT (S-TTT), a two-stage test-time adaptation framework: Stage 1 prompts the base LLM itself to read the full context and question and output up to 8 verbatim "evidence spans" it judges relevant (falling back to uniformly sampled spans if it fails); Stage 2 performs LoRA-based test-time training (rank r=16, alpha=32, AdamW, query-projection layers only, 16 gradient steps) with a next-token-prediction objective restricted to those selected spans, after which the adapted model generates its final answer conditioned on the original full context. It is evaluated on Qwen3-4B-Thinking-2507 and Llama-3.1-8B-Instruct against LongLLMLingua, qTTT, QRHead Span TTT, Random Span TTT, and Full Context TTT, on LongBench-v2 and LongBench-Pro (English) with contexts up to 128k tokens.

Key results:

• Preliminary diagnostic on LongBench-v2 (Qwen3-4B-Thinking-2507): base model 40.4% accuracy; Random Span TTT drops it to 38.9%; TTT on GPT-5.5-annotated oracle spans raises it to 45.9%.
• Main results, Qwen3-4B-Thinking-2507 on LongBench-v2: <64k bucket, S-TTT reaches 47.7 vs. base model 46.7 and Random Span TTT 43.6; 64k–128k bucket, S-TTT reaches 35.3, best among all methods.
• Llama-3.1-8B-Instruct on LongBench-v2: S-TTT improves base model from 36.9→38.4 (<64k) and 26.3→28.2 (64k–128k).
• LongBench-Pro, Qwen3-4B-Thinking-2507: S-TTT reaches 42.0 in the 64k–128k bucket, strongest of all TTT baselines.
• Abstract-level headline: up to 15% relative improvement across both models on the two benchmarks.
• Span-selector ablation: model annotation (47.7 / 35.3) beats perplexity-based selection (46.7 / 31.9) and entropy-based selection (45.1 / 33.0).
• Annotation coverage: fallback rate (model fails to produce valid spans) ranges 8.2%–39.9% depending on model/benchmark.
• Efficiency: S-TTT becomes cheaper than Full Context TTT from 64k tokens onward, and cheaper than (or comparable to) Random Span TTT at 64k+.

Why it matters / caveats: The results support the paper's claim that training-data quality, not the adaptation mechanism itself, is the key bottleneck for long-context TTT, and show a self-supervised way to obtain higher-quality adaptation signal that also becomes cheaper at longer contexts. A caveat is that the model's span-annotation reliability varies substantially by benchmark and model (fallback rates up to ~40%), and in a few settings other TTT baselines slightly outperform S-TTT.

Problem: Traffic simulators for autonomous driving are increasingly optimized for realism benchmarks like WOSAC, which measure fit to a single logged future and therefore fail to capture or reward the diversity of plausible traffic behaviors.

Method: Flow-ERD combines two components: Agent-Type Aware Flow Matching (AFM), a flow-matching backbone (affine optimal-transport path) that generates continuous H-step kinematic action sequences via a SMART-style scene encoder and a DiT-style chunked-attention flow decoder, then executes actions through agent-type-specific transitions (holonomic for pedestrians, non-holonomic/bicycle-style for vehicles and cyclists); and Entropy-Regularized Distillation (ERD), a closed-loop fine-tuning stage minimizing a tempered reverse-KL objective (temperature β = 1/(1+γ)) implemented via Distribution-Matching Distillation (following Self-Forcing) to reduce covariate shift while preventing mode collapse. Evaluation uses the WOSAC 2025 benchmark (test and 4% validation splits) with the Realism Meta-Metric (RMM) plus a proposed log-free diversity metric, Cross-Pair Diversity (CPD), and an intent-entropy analysis over 1,048 validation scenes.

Key results:

• On WOSAC 2025 test split, AFM backbone (no fine-tuning) achieves RMM 0.7845 and the best kinematic score (0.4955) among all baselines including fine-tuned ones.
• Flow-ERD (AFM+ERD) ranks first overall with RMM 0.7878, kinematic 0.5062, and best minADE 1.2721, beating fine-tuned baselines CAT-K (0.7846), RoaD (0.7847), RLFTSim (0.7857), SMART-R1 (0.7858), DecompGAIL (0.7864).
• On the validation split, AFM backbone attains the highest CPD (0.1858) among pretrained baselines (vs. SMART 0.1655, TrajTok 0.1587, UniMM 0.1514) at RMM 0.7836.
• ERD fine-tuning: β=1.0 gives RMM 0.7876 with CPD 0.1684 (ΔCPD = -0.0174 vs. backbone); β=0.99 gives RMM 0.7869 with CPD 0.1828 (ΔCPD = -0.003), while other fine-tuned baselines lose at least ΔCPD = 0.007 in diversity.
• Intent-diversity analysis: SMART concentrates almost all mass on the dominant "straight" maneuver; Flow-ERD increases intent diversity and recovers rare U-turn maneuvers as β decreases.

Why it matters / caveats: The results argue that realism-only benchmarks (like RMM/WOSAC) can be gamed by dominant-mode collapse, and that diversity must be measured and optimized explicitly — Flow-ERD shows realism and diversity can be improved jointly rather than traded off; a caveat is that CPD is a pure spread measure that can conflate genuine multimodality with closed-loop drift/error, so the authors only compare CPD at matched RMM.

Problem: Phone segmentation and recognition are typically modeled with separate, gradient-trained systems even though the phonetic structure needed for both is already latent in self-supervised speech model (S3M) representations.

Method: The authors introduce SPAM (S3M-based Phonological Activation Mapping), which projects each frame of a WavLM-large representation onto phonological vectors (difference-of-means directions for each PanPhon phonological feature, e.g., voicing, backness) to produce a time-aligned activation map; two gradient-descent-free heads then operate on SPAM — a recognition head doing nearest-neighbor matching against canonical PanPhon feature vectors, and a segmentation head ensembling seven boundary-detection signals (multi-scale cosine-distance differences, backward-contrast signals from a least-squares regressor, and a mel-spectrogram cosine-distance signal) via prominence-based peak detection. Phonological vectors are estimated from as little as ~1 minute (18 utterances) of TIMIT training data, with no fine-tuning of the underlying S3M.

Key results:

• Segmentation (R-value): SPAM achieves the best average R-value (72.0) among TIMIT-trained baselines (CTC 61.9, FCE 69.2, BCE 68.5), and best results on GTIMIT-S (75.2), TORGO (72.2), SSNCE (60.3), Voxangeles (75.1), GTIMIT-Thai (75.8).
• SPAM outperforms MFA-based toplines using ground-truth transcripts on SSNCE (60.3 vs 52.6–52.7) and GTIMIT-Thai (75.8 vs 66.9).
• Recognition (PFER, PRiSM benchmark): SPAM uses only 47K learnable params vs 316M for CTC/FCE; SPAM degrades only ~2% from TIMIT to accented English while CTC/FCE degrade 60–117%.
• Sample efficiency: little to no performance degradation down to 1/256 of TIMIT training data (18 utterances, under a minute of audio).
• Oracle ablation: with ground-truth segmentation, PFER drops to 11.1 on TIMIT and 8.4 on VoxAngeles, showing the segmenter (not the recognizer) is the main bottleneck.
• Unseen phones: PFER is 6.4 on phones seen in TIMIT training vs 9.4 on unseen phones.

Why it matters / caveats: SPAM offers an interpretable, near-zero-training alternative to standard CTC/FCE/BCE phone segmentation-recognition pipelines that generalizes far better to out-of-domain, multilingual, and atypical (dysarthric/accented) speech and to unseen phones, using orders of magnitude less labeled data and trainable parameters. The paper notes segmentation remains the main bottleneck, and the S3M frame rate is relatively coarse for fine-grained boundaries.

Problem: Fine-tuned LLMs can memorize newly injected facts almost perfectly yet fail to use that knowledge in downstream multi-hop reasoning, a dissociation the paper terms the "Knowing–Using Gap."

Method: The authors construct fact-injection datasets from STaRK-Prime (biomedical, PrimeKG) and STaRK-MAG (academic, Microsoft Academic Graph) knowledge bases, fine-tuning Qwen-2.5 (1.5B/3B/7B) and LLaMA-3.2/3.1 (1B/3B/8B) models via both LoRA (rank 16, alpha 32) and full fine-tuning on "chaining" and "intersection" multi-hop QA tasks. They introduce "self-patching," an activation-patching variant that copies a hidden-state representation from an anchor token position in a source run/layer into a target run/layer and measures the resulting change in answer correctness, scanning all layer pairs to build permeation maps across training checkpoints. Based on these maps they design a fixed two-layer-pair heuristic (source ≈0.8L and ≈0.1L both targeting ≈0.5L, L = number of layers) as a practical, non-oracle remedy.

Key results:

• Base models score only ~3.8–6.0% zero-shot on memorization tasks pre-finetuning, confirming knowledge novelty.
• Under LoRA, chaining tasks show near-perfect memorization (~10.4 epochs) but generalization accuracy of only 0.303 with a temporal lag of 4.6 epochs; intersection reaches 0.910 generalization accuracy with lag of only 0.6 epochs.
• Oracle self-patching at convergence lifts chaining accuracy 1.5–6× across all models/domains (e.g., Qwen-2.5-1.5B chaining: 0.078 → 0.793 on STaRK-Prime); intersection gaps are nearly eliminated (up to 0.994–1.0).
• Token-position ablation: patching the entity token gives mean effect 0.6408 (p<0.0001) versus 0.1359 for random position and 0.0485 (not significant) for the BOS token.
• CoT prompting only partially helps (e.g., Qwen-2.5-7B chaining: 0.124 → 0.390) versus self-patching's 0.774.
• The fixed non-oracle heuristic recovers 58–75% of oracle headroom on average: chaining average rises from 0.121 (no patch) to 0.357 (fixed) versus 0.444 (oracle).

Why it matters / caveats: The findings reframe the generalization failure as a routing/alignment problem rather than a knowledge-capacity problem — facts are stored but not routed to computation-effective (mid) layers — suggesting alignment-aware training as a path forward. The heuristic remedy is a proof of practicality rather than a full solution, leaving a 25–42% gap to oracle performance.

Problem: Existing open language models are either not genuinely open (weights released without reproducible data/training details), under-serve German relative to English, or use dense Transformer architectures whose KV-cache grows with context and concurrency, making long-context, high-concurrency serving expensive.

Method: The authors train Soofi S 30B-A3B, a 52-layer hybrid Mixture-of-Experts Mamba-Transformer that reuses the openly published Nemotron 3 Nano reference architecture unmodified (23 Mamba-2 sequence-mixing layers, 23 granular MoE layers with shared experts, 6 Grouped-Query Attention layers), totaling ~31.6B parameters with only ~3.2B active per token and just 6 layers maintaining a KV cache. Training uses Megatron-Bridge with AdamW under a Warmup-Stable-Decay schedule across three data phases (diverse pretraining ~20T tokens, high-quality annealing ~6.58T tokens, long-context extension to 1M tokens ~0.10T tokens), totaling ~26.68T consumed tokens drawn from Nemotron-CC, Dolma 3, FinePDFs/FineWiki, HPLT, German-Commons, KletterMix, and the commercially licensed Genios corpus, with German up-weighted to 7.2%/15.3% versus a ~5% baseline.

Key results:

• Trained on up to 512 NVIDIA B200 GPUs (64 DGX B200 nodes), consuming ~253,000 B200 GPU-hours from 24 March to 13 May 2026.
• vs. large open-source models (Alia 40B, EuroLLM 22B, Apertus 70B, Olmo 3 32B): highest English aggregate 70.1 (+2.8 over Olmo 3 32B), German aggregate 79.1 (+6.3 over Apertus 70B).
• Code: HumanEval 73.8 (+10.8), MBPP 70.2 (+7.4), HumanEval-DE 65.5 (+3.0), MBPP-DE 84.2 (+13.4) over next-best open-source baseline.
• Math: GSM8K-Platinum-DE 87.1 (+9.7 over Olmo 3 32B), Minerva-500 79.4 (+24.2).
• Knowledge/reasoning: GPQA-Diamond 43.4 (+10.1 over Olmo 3 32B), INCLUDE-DE 61.2 (+10.1, tied with Qwen3.5 35B-A3B).
• vs. Nemotron 3 Nano 30B-A3B (identical architecture, different data recipe): +1.8 English aggregate, +4.2 German aggregate — isolating the effect of the German-English data curriculum.
• Long-context serving: measured aggregate decode throughput 8-9x that of dense 14-24B models at 40K context/batch 32, staying flat from 4K to 256K tokens where dense models degrade.
• ~99% of the training mixture is reconstructable from public sources; only the commercially licensed Genios corpus (1.3% of Phase 1 tokens) is not open-data-redistributable, satisfying OSI's OSAID 1.0 openness standard.

Why it matters / caveats: Soofi S demonstrates that a sparse MoE hybrid Mamba-Transformer can match dense 14-27B models on English/German benchmarks while activating far fewer parameters and sustaining much higher long-context throughput; presented as the strongest fully open (data-transparent) German-English model to date. The paper notes residual gaps versus larger dense baselines in contamination-aware code evaluation, open-domain factual recall, extractive reading comprehension, and German competition mathematics.

Problem: Development of medical multimodal foundation models is constrained by the scarcity of large-scale, high-fidelity clinical image-text data, since existing PubMed Central (PMC)-derived datasets suffer from poor medical relevance, undecomposed multi-panel figures, and weak image-caption alignment.

Method: MedPMC is a five-stage automated pipeline (initial screening, multi-panel figure detection, multi-panel figure separation, caption separation and alignment, medical figure classification) applied to 6.1 million PMC articles, using a PubMedBERT text classifier for screening, a Vision Transformer for multi-panel detection and figure classification, a YOLOv10-based detector for figure separation, and a supervised InternVL-2.5-4B MLLM for caption separation/subfigure-subcaption alignment. The resulting 11M curated image-text pairs were used to train MedPMC-CLIP (OpenCLIP ViT-L/14 initialization, following BMC-CLIP's contrastive training protocol) and, via the LLaVA-Med instruction-tuning pipeline, as a replacement vision encoder for a multimodal LLM.

Key results:

• Curated 11 million medical image-text pairs from 6.1M PMC articles (5.1M permissively licensed) via a pipeline funneling 3.7M candidate figures down to 7.3M final medical subfigures.
• Component performance: initial screening F1=93.2 vs. keyword baseline F1=61.7; multi-panel detection F1=96.5; figure separation mAP=89.8; caption separation/alignment F1=81.4, ROUGE-L=85.3 (vs. PMC-OA F1=48.5/ROUGE-L=44.4).
• Manual review (5 annotators): 95.3% of MedPMC images medically relevant vs. 19.7% for BIOMEDICA.
• MedPMC-CLIP vs. BMC-CLIP across 26 benchmarks/11 specialties: +7.1 pp accuracy, +10.5 pp F1, +7.1 pp AUC, despite using <half the training pairs; won in 10/11 specialties.
• As vision encoder in LLaVA-Med: +1.9 pp on MMMU (CI crosses zero) and +16.9 pp on OmniMedVQA; disease diagnosis/lesion grading subcategory: 34.2%→45.4% (+11.1 pp).
• Clinical validation on 10,524 YNHHS dermatology images (morphology-to-image retrieval): +3.4 pp Recall@1, +11.7 pp Recall@5, +10.7 pp Recall@10 vs. BMC-CLIP.

Why it matters / caveats: MedPMC demonstrates that systematic, reproducible literature curation (not just scale) can yield stronger medical vision-language representations that transfer to real clinical data; the authors note MedPMC is not a replacement for clinical datasets, is smaller than broader (but noisier) PMC collections like PMC-15M and BIOMEDICA, and the MMMU improvement's confidence interval crossed zero.

Problem: Vision-language models used as digital museum guides for ancient Greek pottery struggle to ground open-ended questions in reliable curatorial evidence and often give confidently wrong answers instead of expressing uncertainty when evidence is incomplete or disputed.

Method: VaseMuseum is an inference-time, training-free multimodal agent framework built around VaseAgent, a Qwen3-VL-8B-based reasoning core that runs a DeepResearch-style iterative tool-use loop for web/museum retrieval, followed by two lightweight reliability controls: source control (validity filtering via an accessibility/text-sufficiency score, MMR-based diverse hit selection, capped at 5 hits) and response control (claim-evidence lexical-overlap matching, a confidence score combining claim coverage/consistency/multi-source support/conflict penalties that triggers hedged answers below threshold). An additional training-free GRPO-style selector samples K independent controlled trajectories and reranks them via reliability verifiers without updating the VLM backbone. Evaluation data is built from VaseVQA-3D's 3,000+ vase image collection, using 518 vases linked to the LIMC database, enriched with museum catalogues and iconography sources.

Key results:

• Three balanced evaluation splits of 100 instances each (Visual-only, Visual+Knowledge, Ambiguous), plus 100 held-out paraphrase queries for prompt tuning.
• On Visual+Knowledge queries, VaseAgent (Full) achieves the lowest hallucination rate (.22) vs. Qwen3-VL-8B+Search (.44) and Direct (.29), and highest groundedness (.73) and neutrality (3.02) among search-based methods.
• External link validity improves from 28.89% (search-only) to 44.07% (without GRPO) to 60.00% (full system with GRPO).
• Neutrality score (0–5 scale) improves from 2.91 (search-only) to 3.04 (without GRPO) to 3.75 (full).
• Ablation: removing source-control reward drops link validity from 60.00 to 44.07; removing response-control reward drops it further to 33.33 and neutrality to 2.77; an accuracy-oriented GRPO variant raises accuracy to 59.00% but link validity falls to 28.89% — showing an accuracy/reliability trade-off.
• Direct VLM answering remains competitive on visual-only accuracy (69.0%) but is weaker on knowledge-intensive reliability metrics.

Why it matters / caveats: The results suggest inference-time reliability control (rather than larger models or retraining) is a practical path to trustworthy cultural-heritage VLM assistants; the authors note limitations including dependence on external source availability/stability, current scope limited to ancient Greek pottery, and reliance on human-judged metrics that need broader expert evaluation.