AI papers — 2026-05-22

Problem: Conventional transit route planning depends on structured map infrastructure and routing engines, which this work bypasses entirely by learning routes directly from navigation log data.

Method: Qwen3 base models (0.6B, 1.7B, 4B) are trained with a two-stage pipeline: Continual Pre-Training (CPT) on 12.9M route planning sessions plus 1.0M static descriptions using next-token prediction, followed by Supervised Fine-Tuning (SFT) on three benchmark tasks with 30K examples each. All 120,845 station IDs from four Chinese cities (Beijing, Shanghai, Shenzhen, Chengdu) are registered as dedicated vocabulary tokens, enabling direct station-level prediction without character composition. The dataset (TransitLM) contains 13.9M records across 13,666 lines sourced from Amap navigation logs.

Key results:

• Qwen3-4B achieves 97.0% connectivity, 98.5% station grounding, and 71.0% route exact match on the 10K test set
• Gemini-Pro (best general-purpose LLM baseline) scores only 75.5% connectivity and 40.2% exact match
• GPS-only ablation retains 70.4% exact match (vs. 71.0% with text), while general LLMs collapse to <1%, proving genuine spatial grounding
• Multi-task joint training (4B-Joint) achieves 73.7% exact match with no negative transfer

Why it matters / caveats: This is the first demonstration that end-to-end map-free transit planning is feasible at scale, with implicit spatial grounding learned purely from trajectories. The dataset is limited to four Chinese cities, and generalization to transit systems with different structures or lower data density is untested.

Problem: In RLVR training, per-token policy gradient updates are dominated by shared high-frequency patterns (formatting tokens, common entities) rather than the tokens that actually distinguish high-reward from low-reward responses.

Method: DelTA reframes RLVR policy-gradient updates as linear discriminators over token-gradient vectors, then iteratively refines token weights via K alternating steps of entropy-regularized soft assignment. Side-wise centroids (positive vs. negative reward groups) are updated as score-weighted averages, where tokens score higher when closer to their own centroid than the opposite side's centroid; this uses sigmoid-based scoring σ(Δ/γ) with side-specific temperatures. The resulting bounded reweighting coefficients λᵢ,ₜ ∈ [λ_min, λ_max] replace the standard advantage weights in the DAPO surrogate objective and require no external value estimates or dense annotations.

Key results:

• Qwen3-8B-Base: FIPO baseline 25.14 → DelTA 28.40 (+3.26 pts) across 7 math benchmarks (AIME24-26, HMMT25-26, Brumo25)
• Qwen3-14B-Base: 37.29 → 39.91 (+2.62 pts), best on all seven benchmarks at both scales
• Training with only top-ranked tokens by λᵢ,ₜ outperforms full-token training even with 50% fewer tokens
• Removing opposite-side comparison, entropy regularization, or refinement iterations each degrades performance in ablations

Why it matters / caveats: DelTA is a drop-in modification to the RLVR training objective that consistently improves strong baselines without architectural changes or additional supervision. Results are shown on math reasoning and code generation, but coverage of other RLVR domains is limited.

Problem: Existing agent benchmarks do not evaluate whether assistants can proactively discover and address hidden user intents in underspecified long-horizon tasks, measuring only reactive task completion after user clarification.

Method: π-Bench provides 100 multi-turn tasks across five domain-specific personas (researcher, marketer, law trainee, pharmacist, financier), each organized as 20 sessions with cross-session dependencies: six strong dependency groups (2–3 interconnected tasks) plus five independent tasks. Each task includes an initial underspecified request, a set of hidden intents ℐ={i₁,...,iₘ}, verification checklists 𝒞={c₁,...,cₙ}, and two grader types (rubric-based LLM and rule-based deterministic). Intent resolution is tracked through three terminal statuses: Completed (agent resolves without prompting), Inferred (agent elicits via targeted question), and Provided (user must volunteer information).

Key results:

• GPT-5.4: 67.0% proactivity, 65.6% completeness (highest proactivity)
• Claude Opus 4.6: 65.5% proactivity, 67.6% completeness (highest completeness)
• Kimi K2.5: 43.1% proactivity but 61.6% completeness, demonstrating the metrics are dissociated
• Removing prior session history drops proactivity by 9.5 points on average but only completeness by 2.5 points

Why it matters / caveats: The benchmark cleanly separates proactive intent discovery from reactive execution, revealing that even top frontier models succeed on fewer than two-thirds of proactive intent cases. The 100-task scale is modest, and results for specific models suggest this was conducted with very recent models whose exact version details and reproducibility may be limited by API access.

Problem: Long-context LLM inference suffers quadratic cost from full attention, but existing sparse attention solutions either require native sparse pretraining or rely on heuristic token eviction causing accuracy degradation.

Method: RTPurbo converts full-attention LLMs to sparse attention through three components requiring only ~600 training steps (~1M tokens total). First, offline head calibration identifies retrieval heads (a small subset requiring full context) via needle-in-haystack tests. Second, adaptive sparse attention applies 16-dimensional low-rank pre-RoPE projections (exploiting high-frequency RoPE components' minimal contribution to long-range retrieval) to route queries to relevant tokens, with dynamic top-p thresholding (not top-k) to handle query-dependent retrieval variance. Third, two-stage training uses KL divergence loss for projection matrices (~30M tokens) followed by self-distillation on top-10 logits (~1.2M tokens). A sort-free 256-bin histogram kernel and bandwidth-optimized sparse decoding implement inference efficiently.

Key results:

• LongBench average: 54.24% (RTPurbo) vs. 53.80% full attention, vs. 52.98% RazorAttn, 50.74% SnapKV
• AIME reasoning accuracy: 86.67% matching full attention (Quest: 46.67%, SnapKV: 43.33%)
• Prefill speedup: 9.36× at 1M context vs. FlashAttention-2; decode speedup: 2.01×
• Achieves 97.1% sparsity at 512K context while competitors show catastrophic degradation

Why it matters / caveats: This work establishes that full-attention pretraining remains competitive since sparsity can be imposed post-hoc in under 100 steps, challenging the assumption that sparse models must be trained from scratch. The evaluation is limited to specific benchmarks and does not yet show results on the widest variety of downstream tasks.

Problem: Multimodal LLMs often achieve low prediction error on personality assessment benchmarks by exploiting superficial visual correlations (e.g., smiling → agreeableness) rather than genuinely grounding ratings in behavioral evidence.

Method: The MM-OCEAN benchmark introduces Grounded Personality Reasoning (GPR), a three-task sequential protocol applied to 1,104 videos (~13.5K verified atomic behavioral observations): T1 requires ordinal Big Five ratings (1–5 scale), T2 requires open-ended reasoning citing observed cues, and T3 requires multiple-choice cue grounding across seven cognitive categories (Personality Attribution, Counterfactual Reasoning, Temporal-Causal chains, Micro-expression Detection, Spatial Localization, etc.). An LLM annotation pipeline uses four agents (Observer, Psychologist, Examiner, Aligner) with human verification, producing 5,320 cue-grounding MCQs. 27 MLLMs (13 proprietary, 14 open-source) were evaluated.

Key results:

• 51.3% Prejudice Rate: over half of correct T1 ratings are not supported by correct T3 cue grounding
• Mean Holistic-Grounding Rate (joint success T1+T2+T3): only 10.4% across all models
• Best model (Gemini 3 Flash): 33.5% holistic grounding despite 64.1% T1 accuracy
• Closed-source frontier models outperform open-source by 26.6 percentage points on T3 cue retrieval

Why it matters / caveats: The "Prejudice Gap" metric reveals a systematic flaw in how MLLMs are currently evaluated for social perception tasks — accuracy on ratings masks near-total failure on grounded evidence retrieval. Spatiotemporal grounding is identified as the main frontier; however, the video corpus and annotation process rely heavily on LLM-generated labels with limited human verification coverage.

Problem: Standard agent SFT masks tool responses and supervises only turn-level action selection, leaving long-context reasoning from multi-turn evidence unused and creating a "supervision blind spot" that prevents models from developing long-range dependency skills.

Method: Agent Context Compilation (ACC) converts multi-turn agent trajectories into single long-context QA pairs without additional annotation by assembling all tool responses and environment observations from a trajectory into one compiled context and training on direct question-answering over that context. For search agents, visited webpage text is extracted and unvisited search results are included as distractors; for SWE agents, files involved in correct patches are extracted with unopened files as distractors; for SQL agents, complete queried table contents are used. Evidence pieces are randomly shuffled to prevent position-based shortcuts. Qwen3-30B-A3B is trained on 10,802 compiled trajectories from search, SWE, and SQL agent logs.

Key results:

• MRCR benchmark: +18.1 points improvement (68.3 total), comparable to Qwen3-235B-A22B despite 8× fewer parameters
• GraphWalks benchmark: +7.6 precision improvement (77.5 total)
• GPQA-Diamond: +2.49, MMLU-Pro: +1.50, AIME'25: +3.33 with no negative transfer
• Training data distribution is clearly separable from benchmarks (classifier AUC 0.9986), ruling out data contamination

Why it matters / caveats: ACC provides a scalable, annotation-free way to repurpose existing agent logs for long-context capability development, and the gains transfer to general reasoning. The method is model-agnostic and complementary to other long-context training approaches, but the 10K-trajectory training set is relatively small, and mechanistic analysis is limited to two benchmarks.

Problem: Existing 3D generation methods focus on visual appearance and are limited to single object categories, leaving simulation-ready physical property generation (materials, kinematics, scale) unaddressed across diverse asset types.

Method: PhysX-Omni uses Qwen2.5-VL-7B-Instruct as its VLM backbone, fine-tuned on PhysXVerse (8.7K+ simulation-ready assets across 2.9K+ categories) for 5 epochs on 64 NVIDIA A100 GPUs (~14 days). A template-based run-length encoding (RLE) geometry representation encodes high-resolution 3D structures directly without compression or new special tokens, within a 16,384-token maximum sequence length. A global-to-local hierarchical part reasoning paradigm generates structure before fine-grained properties, and a TRELLIS decoder converts VLM output to meshes. The PhysX-Bench evaluation covers six dimensions: geometry, scale, material, affordance, kinematics, and description.

Key results:

• Chamfer Distance: 2.95 vs. 37.06 (PhysX-Anything baseline), 92% reduction
• F-score: 91.28 vs. 40.46 baseline, 125% increase
• Absolute Scale Error: 2.79 vs. 298.19, 99% reduction
• Kinematic Score: 0.9185 vs. 0.4191, 119% increase
• Spearman correlation ρ=1.0 on scale, affordance, and description evaluation dimensions

Why it matters / caveats: Unified physical 3D generation across rigid, deformable, and articulated objects is a prerequisite for scalable embodied AI and robotics simulation. The PhysXVerse dataset covers 2.9K+ categories but only 8.7K assets, which may limit generalization to rare object types; downstream robotic manipulation evaluations are demonstrated qualitatively rather than with full quantitative benchmarks.

Problem: Explicit text-based chain-of-thought in multimodal LLMs compresses continuous audio-visual signals into discrete tokens, weakening temporal grounding and biasing reasoning toward language priors rather than sensory evidence.

Method: LatentOmni interleaves textual token generation with continuous audio-visual latent reasoning phases, triggered by special tokens <Unified_Latent> and </Unified_Latent>. Latent embeddings are instantiated as transformer hidden states in ℝ^d before the language modeling head. An Omni-Sync Position Embedding (OSPE) extends time-aligned multimodal RoPE to assign shared physical timestamps to temporally corresponding audio and visual features. Training combines three losses: InfoNCE temporal synchronization loss (ℒ_sync), L2 latent alignment loss (ℒ_latent) against task-relevant feature anchors, and standard cross-entropy text prediction loss (ℒ_text). The LatentOmni-Instruct-35K dataset is constructed via a three-stage pipeline covering AVQA synthesis, segment-level captioning, and interleaved reasoning trajectory synthesis.

Key results:

• Daily-Omni: 67.4% (+4.5% over base model)
• OmniVideoBench: 35.4% (+6.1%)
• WorldSense: 48.9% (+3.5%)
• LVOmniBench: 35.1% (+3.1%)
• Outperforms Explicit Text CoT baseline by 1.8–3.0 percentage points across all four benchmarks; best open-source on all four

Why it matters / caveats: Continuous latent reasoning preserves fine-grained temporal and cross-modal signals that discretization destroys, offering a principled alternative to text-only chain-of-thought for audio-visual tasks. The 35K training set is modest in scale and the approach's scaling behavior with larger datasets or models is not yet characterized.

Problem: Autoregressive video diffusion models for interactive world generation lose long-term scene memory when using sliding-window KV caches, but maintaining full KV caches reduces throughput from 8.87 FPS to 3.61 FPS over one-minute rollouts.

Method: WorldKV is a training-free framework with two components applied to frozen backbones. World Retrieval stores evicted KV-cache chunks in GPU/CPU memory and selectively re-retrieves the top-k scene-relevant chunks using camera/action correspondence similarity (sim(a_cur, a_i)). World Compression reduces per-chunk storage by approximately 50% via key-similarity pruning: for each non-anchor frame, tokens are scored by average cosine similarity to anchor-frame keys, and only the bottom 25% of tokens (lowest similarity, most distinctive content) are retained, eliminating redundant information.

Key results:

• LingBot-World-Fast (14B): WorldKV LPIPS 0.455 vs. Full KV 0.441 and Sliding 0.581; throughput 4.78 FPS vs. Full KV 2.36 FPS (2× speedup)
• Matrix-Game-2.0 (1.3B): WorldKV LPIPS 0.462, PSNR 14.101, FID 93.561 — outperforming both Full KV and Sliding baselines; 16.25 FPS vs. Full KV 7.82 FPS
• Compression enables broader historical coverage within fixed attention budget, improving revisit consistency more than retaining fewer uncompressed chunks

Why it matters / caveats: WorldKV enables persistent, consistent world memory in real-time interactive generation without any retraining, achieving near-full-KV fidelity at double the throughput. The approach is tested on only two models, and the retrieval quality depends on camera/action signal availability, which may not generalize to all video generation settings.

Problem: LLM-based spreadsheet agents fail on complex multi-step workflows because existing approaches rely on prompting rather than model-specific training with realistic environment feedback.

Method: The framework combines three components: a Spreadsheet Data Agent that automatically collects 5,928 training tasks from ExcelForum discussion threads with rule-based filtering; a Spreadsheet Gym multi-turn interactive environment using real Microsoft Excel (version 2512) with workspace isolation for parallel RL training; and a spreadsheet-native tool harness with structured actions (fill_formula, clear_range, delete_rows/columns) enforcing an inspect-modify-verify workflow. GRPO (Group Relative Policy Optimization) on-policy RL training uses outcome-based rewards comparing final spreadsheets against oracle solutions. The base model is Qwen3-4B-Thinking-2507.

Key results:

• SpreadsheetBench Pass@1: base 12.0% → native harness 15.6% → full tools 19.3% → RL post-training 23.4%
• Domain-Spreadsheet (1,660 tasks, finance/supply/HR): 8.4% → 17.2% after RL
• Finance beginner tasks: 9.5% → 29.3%
• Mean response length decreased from ~16K to ~11K tokens; interaction turns reduced from ~20 to ~11 over 60 training steps

Why it matters / caveats: RL with real-environment outcome rewards substantially improves spreadsheet agents, surpassing OpenAI o3 on benchmarks and generalizing to domain-specific tasks despite training only on forum data. The training set is limited to one forum source and the environment requires Microsoft Excel, which constrains reproducibility and applicability to open-source spreadsheet tools.

Problem: Video diffusion models trained on short clips cannot generate long, temporally coherent sequences — bidirectional models degrade in quality and autoregressive models accumulate drift errors leading to repetitive motion.

Method: FlowLong operates at inference time without additional training, applying two components to existing video diffusion models. Tweedie Matching blends predicted clean samples across overlapping adjacent video segments through gradient-based corrections that reduce to per-frame interpolation on overlap regions, enforcing temporal consistency without modifying model weights. Stochastic Early-Phase Sampling injects stochastic noise during the initial ODE steps (breaking trajectory inertia) then switches to deterministic ODE sampling in later steps to preserve visual fidelity, using a binary schedule. The method is compatible with Wan 2.1-T2V (1.3B and 14B), LTX-2, and Wan 2.1-T2V-14B with AnySplat for text-to-3DGS.

Key results:

• 30-second videos on Wan 2.1: Overall VBench score 0.8233, Background Consistency 0.9305, Dynamic Degree 0.7800
• 60-second videos: Overall score 0.8251, outperforming autoregressive baselines on Dynamic Degree
• Text-to-3DGS: generates 1.64× more Gaussians per scene with improved confidence scores (41.52 vs. 26.27 baseline)

Why it matters / caveats: FlowLong is training-free and architecture-agnostic, enabling any short-clip video diffusion model to generate arbitrarily long sequences without fine-tuning. VBench scores are reported but comparison to the strongest baselines on all metrics is incomplete, and the method's scalability to very long durations (>60 seconds) is not fully characterized.

Problem: Spatial reasoning benchmarks for MLLMs assume perfect, high-resolution images, failing to evaluate robustness under the visual degradations (blur, low-light, haze, compression) that real-world embodied and autonomous systems encounter.

Method: SpaceDG reconstructs multi-view images from 584 indoor scenes into 3D Gaussian Splatting (3DGS) representations with auto-annotated spatial QA pairs, then applies a physically realistic degradation synthesis pipeline covering nine types across four categories: optical/dynamic (defocus blur, lens distortion, motion blur), meteorological (haze, water droplets), photometric (low-light, over-exposure), and digital (JPEG compression, low-resolution). SpaceDG contains ~1M QA pairs from 162K+ images; SpaceDG-Bench provides 1,102 manually verified questions (9,918 VQA instances across 11 reasoning categories). A degradation-aware SFT model (SpaceDG-SFT-Qwen3-VL-8B) is also trained and evaluated.

Key results:

• Gemini-3.1-Pro: 63.1% (clean) → 56.7% (degraded), -6.4% drop
• InternVL3.5-38B: 52.9% → 47.7%, -5.2% drop
• Human baseline: 80.4% → 59.5%, -20.9% drop
• SpaceDG-SFT-Qwen3-VL-8B: 49.1% → 73.2% (clean), 42.1% → 66.1% (degraded) — surpassing human degraded performance (59.5%) by 6.6 points
• Low-light and haze degrade performance most; camera-centric tasks are more robust than object-centric tasks

Why it matters / caveats: The benchmark exposes a critical but overlooked robustness gap for spatial reasoning in realistic conditions, and degradation-aware SFT simultaneously improves both clean and degraded performance. The dataset is currently limited to indoor scenes from a single reconstruction pipeline; outdoor, dynamic, and sensor-specific degradation patterns are not yet covered.

Problem: Current agent frameworks use monolithic LLMs with static routing, failing to exploit the complementary strengths of diverse expert models and hierarchical skills across heterogeneous tasks.

Method: Maestro trains a lightweight 4B orchestrator policy using GRPO (Group Relative Policy Optimization) with sparse outcome-based rewards (binary task success + format adherence penalty) to dynamically compose ensembles from a frozen pool of five expert models: GLM-4.6V-Flash 9B, Chart-R1 8B, Qwen3-VL-8B, Intern-S1-mini 9B, and MedGemma-1.5-4B. The compositional action space consists of triplets (model_m, skill_s, query_z) over a two-tier skill hierarchy: 5 coarse-grained Level-1 skills mapped to 8 fine-grained Level-2 skills. No step-level supervision is needed. New experts and skills can be registered without retraining the orchestrator.

Key results:

• In-domain (10 benchmarks) average: 70.1% — surpassing GPT-5 (69.3%) and Gemini-2.5-Pro (68.7%)
• ChartQA: 86.8%; Geometry3K: 77.4%
• VStar: 88.0%; HRBench-4K: 79.6% (out-of-domain)
• BFCL-V4: 78.09 vs. GPT-5.2 68.58; tau2-bench: 72.9 vs. Claude-Opus-4.5 70.2
• Average latency: 2.88 seconds; token consumption: 648.20 tokens
• Removing model pool causes -12.1% accuracy; removing format reward causes -13.1%

Why it matters / caveats: Maestro demonstrates that orchestrating small specialist models can outperform frontier monolithic models while being faster and cheaper, and the system generalizes to new experts without retraining. The approach requires maintaining multiple frozen models simultaneously, increasing memory overhead, and the benchmark suite skews toward multimodal visual tasks.

Problem: Autonomous driving validation requires diverse multi-sensor data, but in-the-wild monocular dashcam footage (abundant and rich in long-tail scenarios) cannot be used because it lacks the 360-degree camera and LiDAR structure required by autonomous driving systems.

Method: Sensor2Sensor uses 4D Gaussian Splatting (4DGS) with support for dynamic rigid and deformable objects to reconstruct scenes from proprietary AV logs and render synthetic dashcam views with realistic intrinsic/extrinsic parameters sampled from real distributions, creating paired training data. A multi-modal diffusion architecture then learns the inverse mapping: the image branch uses multi-view diffusion with 3D attention modules (1D cross-view, 2D spatial) and the LiDAR branch uses a dedicated VAE processing range-view spin images (range, intensity, elongation, validity channels). Cross-sensor consistency is enforced by concatenating features through shared self-attention. Video generation uses DAgger-based autoregressive rollout to reduce error accumulation.

Key results:

• Multi-view image FID: 6.47 vs. 8.30 (baseline); LPIPS: 0.316, PSNR: 19.06, SSIM: 0.539
• Video FVD: 278.12 vs. 293.73 (without view-concatenation); video PSNR: 22.42, SSIM: 0.623, LPIPS: 0.186
• LiDAR Chamfer Distance: 8.68 (13.37% improvement over X-Drive)
• Human evaluation: 83.46% ranked top on dashcam realism; 94.62% preference over baselines

Why it matters / caveats: This enables mining of previously incompatible external video sources for AV validation data, combining 4DGS reconstruction fidelity with generative model diversity. The approach requires proprietary AV log data for reconstruction training and the paired 4DGS pipeline is computationally expensive; downstream closed-loop ADS evaluation results are not yet reported.

Problem: Autoregressive video diffusion models (ARVDs) have prohibitive inference costs, and existing quantization methods designed for bidirectional diffusion transformers perform suboptimally because ARVDs exhibit frame-wise sensitivity that decays exponentially and heterogeneous outlier channel patterns that vary across layer types and block depths.

Method: Q-ARVD addresses both failure modes with two components. Final-Quality Guided Frame-Weighting quantifies each frame's sensitivity by measuring the impact of quantizing that frame on overall video quality, then assigns exponentially decaying importance weights to the reconstruction loss objective — treating earlier frames as more critical. Outlier-Aware Adaptive Dual-Scale Quantization uses Modified Z-score with a minimum magnitude constraint to automatically detect the ~2.1% of outlier channels, applies separate quantizers for outlier and normal channels, and reduces scaling factors for normal channels to minimize rounding error.

Key results:

• W8A8: FVD 61.67, LPIPS 0.335 (outperforming prior methods at same bitwidth)
• W4A8: FVD 106.04, LPIPS 0.452
• W4A6: FVD 140.38, LPIPS 0.486
• Deployment on NVIDIA A6000 (INT8): 1.30× latency speedup, 1.97× model size reduction

Why it matters / caveats: Q-ARVD is the first quantization framework specifically designed for the ARVD architecture class, making interactive video generation more practically deployable. The speedup (1.30×) is modest compared to what INT8 quantization typically achieves on other architectures, suggesting the memory bandwidth bottleneck persists; results are reported on two specific models (self-forcing and causal-forcing) and generalizability to other ARVDs is not fully demonstrated.

Problem: Linear recurrent attention models use a single scalar gate coupling memory erasure and writing, limiting the expressiveness of fixed-size memory state updates and causing interference among compressed associations.

Method: Gated DeltaNet-2 decouples the single scalar gate into two independent channel-wise gates: an erase gate (𝒃ₜ) controlling which key-side coordinates are read and removed from the decayed state, and a write gate (𝐰ₜ) controlling which value-side coordinates are committed to memory. The Gated Delta Rule-2 state update is: Sₜ = (I − kₜ(bₜ⊙kₜ)ᵀ)DₜSₜ₋₁ + kₜ(wₜ⊙vₜ)ᵀ. Efficient parallel training is maintained through a WY-form algorithm with channel-wise decay absorbed into asymmetric erase factors. The model generalizes KDA (Kernel Delta Attention) as a special case when gates are tied to scalars.

Key results:

• WikiText perplexity (1.3B, 100B FineWeb-Edu tokens): 15.90 (recurrent), 15.62 (hybrid) — outperforms Mamba-2, Gated DeltaNet, KDA, and Mamba-3 variants
• Multi-Key NIAH-1 @4K (RULER): 37.8% (recurrent), 48.0% (hybrid) — significant gains over baselines on interference-heavy retrieval
• Real-world retrieval F1 average (SWDE, SQuAD, TriviaQA, DROP, NQ at 2K): 29.88 (recurrent), 42.28 (hybrid)
• Throughput: near-flat scaling 38.0→36.1 Kt/s as sequence length increases

Why it matters / caveats: Decoupling erase and write directly targets the root cause of memory interference in linear attention, providing a principled and practical improvement over prior gated recurrences. Evaluation is at 1.3B parameters only; scaling behavior and comparison to full attention at larger scales or longer contexts remain to be characterized.

Problem: In disaggregated LLM serving (prefill/decode separation), KV cache transmission accounts for up to 60% of job completion time, but existing KV compression methods use static configurations that are suboptimal or accuracy-violating under dynamic production workloads and bandwidth conditions.

Method: KVServe introduces a three-stage adaptive framework. A modular strategy pool unifies KV compression as a composable Transform → Quantizer → Codec pipeline including novel Mixed-Precision Head-Wise Quantization (MixHQ), Hadamard/Affine transforms, and nvCOMP lossless codecs. A Bayesian Profiling Engine uses Gaussian Process-based Bayesian Optimization with heterogeneous-parameter encoding, bi-directional pruning exploiting the monotonic compression-accuracy tradeoff, and early stopping — reducing offline search from ~1000 hours to ~20 hours and producing a 3D Pareto frontier in accuracy × compression ratio × latency space. A Service-Aware Online Controller uses an analytical model to derive bandwidth thresholds where compression provides net benefit, plus a residual-corrected bandit with exponentially weighted moving average to correct offline-to-online drift, with <1ms per-request overhead.

Key results:

• PD-separated serving: up to 9.13× JCT (job completion time) speedup
• KV-disaggregated serving: up to 32.8× TTFT (time-to-first-token) reduction
• Compression ratios: 7.42–8.28× on Qwen2.5-7B-Instruct across datasets
• Communication time reduced from 82–90% to 6–9% of total latency
• Maintains >97% relative accuracy across diverse workloads; generalizes to unseen datasets (2WikiMQA, HotpotQA)
• Static baselines frequently violate accuracy constraints; KVServe maintains them

Why it matters / caveats: KVServe frames KV compression as a constrained, service-state-dependent optimization problem rather than a fixed algorithm, enabling robust gains in production disaggregated serving systems. Evaluation is limited to Qwen2.5-7B-Instruct; behavior with larger MoE or multimodal models and across diverse hardware interconnects is not yet characterized.