AI papers — 2026-09-18

Problem: Long-horizon, input-heavy agentic workloads make prefill compute and the storage/bandwidth of large KV caches (in HBM, SSD, and host memory) the primary bottleneck to lowering long-context serving costs.

Method: DeepSeek-V4.1-Flash is a 552B-parameter multimodal MoE Transformer (40 layers: 20-layer causal encoder + 20-layer decoder) supporting up to 1M tokens, activating 8B parameters per token during prefill and 16B during decode via a Causal Encoder-Decoder (CED) architecture in which decoder global KV is projected from the final encoder hidden states, nearly halving prefill compute. It combines cross-layer KV/indexer-K/Top-K-index reuse in Compressed Sparse Attention 2 (CSA2, with Full/Reindex/Reuse layer modes and a Hierarchical Sparse Indexer) with FP4 (MXFP4/E2M1) main KV caching via quantization-aware training, plus SWA Bounded Replay (reconstructing sliding-window KV by replaying only the most recent n_win tokens instead of L×n_win) to avoid persisting SWA KV to SSD. Additional architectural extensions include Single-Pass mHC (with a fused Mega-mHC kernel), the Engram conditional memory module (196B parameters), and DSpark speculative decoding; pretraining uses 45T multimodal tokens with sparse attention trained from scratch at 64K sequence length, followed by SFT + RL + on-policy distillation post-training.

Key results:

• Global KV cache footprint reduced to 890 bytes/token, ~1/4 that of DeepSeek-V4-Flash (~437x smaller than DeepSeek-V1); persistent (SSD/host memory) KV cache footprint reduced to ~1/8 of DeepSeek-V4-Flash.
• Extending context length 256-fold (4K→1M) increases single-token Decode FLOPs by only 1/4, versus much larger growth for DeepSeek-V4-Flash.
• DeepSeek-V4.1-Flash-Base matches DeepSeek-V4-Pro-Base on world knowledge/reasoning/coding while using only 1/3 total and 1/4 activated parameters, and shows 5-10% improvements on held-out (BPB) evaluations.
• On DeepSWE v1.1, reaches 74.2% pass rate (vs. 54.4% for DeepSeek-V4-Flash), surpassing Opus-5 (74.0%) and GPT-5.6 Sol (73.0%); on Terminal-Bench 2.1, achieves 90.6% Pass@1, outperforming Opus-5 (89.1%) and GLM-5.3 (88.2%).
• Achieves a Codeforces rating of 3471 (vs. 3289 for DeepSeek-V4-Flash and 3348 for DeepSeek-V4-Pro) and 90.9% on GPQA Diamond; each CSA2 Reuse Mode layer runs with only 15 kernels during prefill and 11 during decode.

Why it matters / caveats: The combined architectural, precision, and deployment optimizations let a much larger model (552B backbone) serve long-context agentic workloads more cheaply than its smaller predecessor while matching or beating frontier closed-source models on most benchmarks, substantially lowering the cost barrier to scaling long-horizon agent deployment. The authors note the newly introduced architectural changes (CSA2 selection, approximate SWA state reconstruction) have robustness boundaries not yet fully characterized, and a performance gap remains versus giant closed-source systems on science-oriented agentic tasks and the most challenging reasoning/edge cases.

Problem: As coding agents run long, unattended, token-hungry trajectories, the agent harness itself (not just model cost or infrastructure) becomes a major source of avoidable token overhead, and manually discovering harness-level efficiency fixes is costly and hard to scale, while automatically-searched harnesses risk overfitting to their search tasks.

Method: SoL-Pi uses an RSI-inspired "Scaling Auto-Research Loop" in which a research AI inspects execution traces from a base harness (Pi), proposes candidate harness changes via a broad-to-deep funnel (152 proposed directions across six families: context, progress, tools, delegation, prompt/policy, improvement/evaluation), implements and reviews each candidate in isolated disposable search lineages using an iterative Ralph-Loop-style implementation cycle, and tests them across 535 search environments (495 GitHub issue–PR repository tasks plus 40 verifier-driven synthetic tasks). Candidates must pass fixed capability-tolerance gates and improve at least one efficiency metric, and held-out evaluation (on EdgeBench) is strictly separated from search feedback. Four mechanisms survive this process and are combined into SoL-Pi: Action Fusion (merges a file edit and its follow-up test/build/run command into one tool call), Online Context Compact (decides when to compact context at plan-step boundaries by comparing projected input savings against cache-rewrite cost), ObservationPack (archives large tool outputs >10 KiB after two full sends, replacing them with a stable handle plus head/tail excerpt), and Evidence-Preserving Reducer (uses a low-cost model to compress build/test logs ≥4 KiB into a verified receipt, with fallback to the original on verification failure).

Key results:

• On EdgeBench (51 public tasks), SoL-Pi [Efficiency] uses 1.10B tokens (49.0% fewer than Pi) while retaining 93.7% of Pi's average score (42.0 vs. 44.8), cutting token cost 33.2% ($894 vs. $1,339).
• SoL-Pi [Performance] raises average score from 44.8 to 47.2 (a 5.3% gain) while reducing token traffic 6.1% and improving token efficiency 9.8%.
• Transferred without further search/adaptation from GPT-5.6 Sol to Opus 5, SoL-Pi retains 94.3% of Pi's score while cutting token traffic 44.7% and API cost 33.5%; relative to native harnesses, it reduces API cost by 50.0% vs. Codex (GPT-5.6 Sol) and 54.3% vs. Claude Code (Opus 5).
• On Terminal-Bench 4, SoL-Pi solves 15/63 tasks (vs. 18/63 for Codex and Pi) but cuts total model cost 26.3% ($211.12 vs. $286.45) and cost per solved task 11.6%; on IMO 2026 it passes 3/6 problems at the lowest cost per passed problem ($20.90 vs. $22.89 Codex, $25.32 Pi).
• In a 20-worker agent-swarm kernel-optimization test, the SoL-Pi swarm reaches 1,127 cycles at $60.11, reducing API cost 26.8% vs. a Pi-baseline swarm (1,366 cycles, $82.12), and passes all eight speed thresholds vs. seven for the Pi swarm.

Why it matters / caveats: The results suggest harness-level RSI search, when scaled across hundreds of diverse environments with search/evaluation strictly separated, can discover token-efficiency mechanisms that generalize across models and benchmarks rather than overfitting to search tasks. Caveats: mechanisms were optimized only on GPT-5.6 Sol trajectories and trigger less often/intensely on Opus 5; SoL-Pi solves fewer Terminal-Bench 4 tasks than the baselines despite lower cost; and the authors note search-scaling laws, multi-backend training, and "recursive efficient improvement" remain future work rather than demonstrated effects.

Problem: Prior work evaluates coding-agent harnesses as monolithic systems, so it is unclear whether performance gains come from planning, action-space design, context management, or their interaction with the underlying model.

Method: The authors build a lightweight ReAct-loop coding harness with a fixed execution loop (safety guards, post-edit diagnostics, stuck detection held constant) while independently varying three components: planning (an explicit persistent task plan updated via an update_plan tool), the action space (a predefined tool set of read/write/edit/search/bash/web tools vs. a bash-only interface), and context management, defined as five tiers T0–T4 (T0: none; T1: elision of stale tool observations; T2: elision plus recoverable external storage via recall_event; T3: LLM summarization alone; T4: staged elision-then-summarization combining all three mechanisms). They evaluate four models (Nemotron-3 at 30B/120B/550B and Mistral-Medium-3.5-128B) on SWE-Bench Verified and Terminal-Bench 2.1 across 176 matched settings (5 context strategies x 4 context-window budgets, plus planning and action-space ablations at 128k/T4), measuring success rate, cost, and trajectory-level statistics.

Key results:

• The success-rate gap between managed context strategies (T1–T4) and no management (T0) shrinks from 35.7 to 2.7 percentage points on SWE-Bench and from 9.5 to 2.8 points on Terminal-Bench as the context window grows from 32k to 128k tokens.
• T0's window-overflow failure rate falls from 78.7% to 8.7% (SWE-Bench) and 61.0% to 12.1% (Terminal-Bench) as budget increases, while all managed tiers (T1–T4) record zero overflow failures at every budget.
• T4 (staged elision + summarization) achieves the lowest mean cost per task in 7 of 8 model-benchmark panels while matching the success rate of other managed tiers.
• Recall (recall_event) is rarely used: 56.3% of T2/T4 settings never invoke it, and across 32 matched T1-vs-T2 comparisons the equal-weight mean success-rate difference is only -0.36 percentage points.
• Planning raises success rate for Nemotron-3 30B by 11.6 points on SWE-Bench and 4.5 points on Terminal-Bench, but for the strongest models (Nemotron-3 550B, Mistral-Medium-3.5-128B) it instead cuts cost by roughly 30% and 32% on SWE-Bench with success dropping only 2.0 and 0.4 points.
• Predefined tools raise Nemotron-3 30B's success rate by 15.0 points on SWE-Bench and 10.1 points on Terminal-Bench versus bash-only, and 66% of bash-only trajectories on Terminal-Bench terminate early because emitted tool calls cannot be resolved without the predefined tool registry.

Why it matters / caveats: The results give model- and budget-aware guidance for harness design: context management mainly pays off by preventing overflow under tight budgets, cheap rule-based elision should be staged before costly LLM summarization, recoverable/lossless mechanisms add machinery models seldom exploit, planning shifts from an accuracy scaffold (weak models) to a cost-saving mechanism (strong models), and predefined tools chiefly help models with weaker bash proficiency. Findings are based on four specific model families/sizes and two benchmarks, so transfer to other models or task domains is not directly verified.

Problem: Predictive world models remain domain-specific, so it is unclear whether a single learning principle can support world modeling across radically different systems (vision, biology, clinical data, control, molecular dynamics, physical fields, weather).

Method: JEPA-Anything extends joint-embedding predictive architectures (JEPA) with orthogonal predictive factorization (OPF): instead of predicting one monolithic target embedding, K learned projector matrices split the stop-gradient target representation into K orthogonal r-dimensional subspaces (Kr=d), each predicted by its own dedicated predictor branch from the shared context representation. Training combines a per-factor regression loss with within-factor and cross-factor orthogonality penalties, a factor-activity floor, and an online-encoder variance term to prevent collapse, added to each domain's original base loss. The predicted factors are recombined into a complete latent state via the Moore–Penrose pseudoinverse of the projector basis, giving a shared interface usable for representation readout, recursive/autoregressive rollout, planning, or factor-level scientific analysis.

Key results:

• Improves reported metrics on all 10 tasks in a matched dynamics benchmark (CausalWorld, DMC, PDEBench, WeatherBench2) against standard JEPA baselines.
• Reduces single-intervention one-step prediction MSE on CITRIS Interventional Pong by 34.83% (0.009541 → 0.006218), combined-intervention MSE by 12.90%, and 6-step free-rollout MSE by 8.58%.
• Achieves the lowest one-step MAE and 100-step rollout RMSD among compared methods (scratch, TrajCast-JEPA, JEPA-Anything) on all four molecular systems (water, quartz, paracetamol, benzene).
• On APEBench, reduces Burgers held-out late MSE by ~49.5% and 6-step rollout MSE by ~44.7%, and Kuramoto–Sivashinsky late-state MSE by ~13.2%, with improvement in every seed.
• Recovered orbital scaling modes fit the Keplerian relation f ∝ a^(-3/2) with a fitted slope of −1.4991 and R² = 0.9999999.
• On single-cell tasks, attains the highest values on both PBMC clustering metrics (AvgBIO) and both perturbation datasets (Adamson and Norman Pearson correlation) among compared methods, including scGPT and Cell-JEPA.

Why it matters / caveats: The results support a common factorized predictive principle that generalizes across heterogeneous domains and connects world modeling to intervention prediction and experimentally validated scientific discovery (a factor-nominated IL-18 + CD73 blockade intervention showed antitumor activity in wet-lab cell co-cultures, organoids, tumor fragments, and mice). However, gains are not universal: in continuous-control planning, JEPA-Anything outperforms standard JEPA on Walker2d and HalfCheetah but underperforms on Hopper, and the authors note orthogonal factors do not by themselves establish causal mechanisms.

Problem: Existing benchmarks evaluate obfuscated-text restoration and risky-webpage investigation separately, so it is unclear how errors in recovering a disguised platform message (used to evade moderation and redirect users to porn, fraud, gambling, or illicit-transaction sites) propagate into downstream web-based risk investigation.

Method: RiskChainBench pairs 3,600 synthetic token-text restoration inputs (600 source sessions × 6 obfuscation variants each, using phonetic/visual substitution, character decomposition, redundant platform tokens, and entry confusables) with 600 human-labeled local, resettable sandbox web environments built offline from real webpages. The same underlying model is evaluated in two frozen stages: first as a restorer that outputs a canonical message, operational intent, and ranked entry candidates without web access, then as a VLM-driven web agent (via BrowserGym/Playwright) that investigates the correctly associated site (without seeing the message or its restoration) and produces an evidence-cited risk report. The two stages are composed offline by using the frozen top-1 predicted entry from a fixed primary variant as a gate applied to the same frozen web-investigation trajectory, and a fixed multimodal evidence judge separately scores evidence faithfulness/sufficiency/completeness/consistency without determining task labels.

Key results:

• Entry Top-1 restoration accuracy ranges from 35.19% (Kimi K2.6) to 95.22% (GPT-5.4) across ten models; full reconstruction (message + intent + entry) is led by GPT-5.6 SOL at 73.31%.
• Task 2 web-only decision accuracy ranges from 26.3% to 62.8%, with GPT-5.6 SOL leading at 62.8% decision accuracy and 61.0% hierarchical exact match.
• Entry-gated end-to-end accuracy ranges from 16.7% to 60.8%, with per-model losses from the web-only view of 0.7–32.3 percentage points (e.g., Qwen3.6 Plus loses 32.3 pp, Kimi K2.5 loses 29.5 pp).
• Across pooled Task 2 runs, execution failures account for 31.9% of first-failure attributions, versus only 0.9% for post-decision type errors, and violation-type macro-F1 (on 394 gold-violation sites) is led by Kimi K2.5 at 48.1%.
• Human website-gold annotation reached 82.50% first-pass decision agreement and 80.17% joint-label agreement (Krippendorff's α of 0.6553 and 0.7425), with 97.9% decision agreement on hidden repeats.

Why it matters / caveats: The entry-gated composition shows that restoration accuracy and web-investigation accuracy rank models differently and that small upstream restoration errors can substantially erase downstream web-investigation gains, identifying stable exploration/execution reliability (not final risk typing) as the main bottleneck for real-world deployment. The benchmark uses fully synthetic messages and isolated local sandboxes (not live services) with only one trajectory per model-website pair, so rerun variance and coverage of underrepresented violation types/languages remain unmeasured.

Problem: In on-policy distillation for agentic RL, a skill-conditioned teacher is not reliably better than the student it supervises, and the benefit of teacher supervision is stage-dependent, so a fixed distillation schedule either withdraws guidance too early or holds the student back too late.

Method: RetireOPD trains in three stages: (1) it first optimizes a decoupled, skill-conditioned teacher with GRPO on environment rewards (rather than just prompting a model with privileged skills) to produce a reliable "skilled teacher"; (2) it then trains a skill-free student jointly with GRPO (environment reward) and on-policy distillation (OPD) using a reverse-KL loss estimated via a sampled-token log-probability gap between student and teacher; (3) it applies "Adaptive Retirement," monitoring a windowed teacher-student discrepancy-change signal (ρm) and the student's relative competence versus the teacher (ηm) every W steps, and drops the OPD term (continuing with GRPO alone) once the discrepancy stops shrinking (ρm ≥ δ) and the student reaches a set fraction of the teacher's success rate (ηm ≥ γ).

Key results:

• On ALFWorld, RetireOPD improves success rate over GRPO by +17.0 (1.5B), +18.8 (3B, 93.8% vs. 75.0%), and +14.1 points (7B).
• On WebShop, RetireOPD improves accuracy over GRPO by +19.0 (1.5B, 75.8% vs. 56.8%), +14.0 (3B), and +11.8 points (7B).
• RetireOPD surpasses its own skill-conditioned teacher in every setting, e.g., 93.8% vs. 79.7% (ALFWorld) and 77.3% vs. 64.8% (WebShop) on the 3B model.
• Ablations: retaining OPD throughout training reaches only 82.8% (vs. 92.2% for full RetireOPD, 3B/ALFWorld); using only the alignment-stagnation or only the relative-competence criterion each reaches 89.0%.
• Retiring-threshold sensitivity: varying γ and δ shifts the exit step from 55 to 95 while success rate stays in 89.1–92.2%; the exit step stays within ±5 steps of default across γ ∈ [0.80, 0.96] and δ ∈ [−0.10, 0.04].

Why it matters / caveats: The method shows that privileged context alone doesn't make a teacher useful (an un-optimized 7B skill-prompted teacher only reaches 23.4% success) and that teacher guidance can later conflict with reward optimization, so removing it online (rather than via a fixed anneal/two-stage schedule like ATOD's linear annealing, which plateaus early) lets the student exceed the teacher's ceiling; evaluation is limited to ALFWorld and WebShop with Qwen2.5 models up to 7B.

Problem: In on-policy distillation (OPD), student responses progressively lengthen and often exhaust the generation budget, and the paper traces this failure to a mismatch between the EOS (termination) tokens preferred by base students versus post-trained teachers.

Method: The authors formalize "sampled-token OPD," where the teacher scores only the token the student actually sampled, and use it to diagnose why a student's stopping token gets suppressed when the teacher prefers a different but semantically equivalent termination token. They compare four termination-handling corrections: Fix 1 (shared-set decoding, i.e., registering both EOS tokens as valid stops), Fix 2 (teacher-side EOS mapping, mapping teacher probability mass across equivalent EOS tokens onto one canonical student token), Fix 3 (semantic EOS class, aggregating all termination-equivalent tokens into a single "stop" action for both models before computing the distillation update), and Fix 4 (canonical single-EOS action space, which additionally removes non-canonical EOS tokens from the student's sampling distribution). These are evaluated on Qwen3-1.7B-Base/Qwen3-4B, Llama-3.2-3B Base/Instruct, Gemma-3-4B PT/IT, and across K2-Horizon-7B checkpoints spanning pretraining, midtraining, SFT, and final post-training.

Key results:

• Under vanilla OPD on Qwen3, the student's probability on its native EOS token (<|endoftext|>) falls from roughly 0.8 early in training to near zero, while the teacher-preferred <|im_end|> token retains only about 10⁻¹¹ probability under the student.
• Fix 1 (aligning only the decoding stopping set) follows nearly the same response-length and clipping trajectory as vanilla OPD and fails to fix the problem, whereas Fixes 2–4 (probability-level alignment) keep length and clipping close to the teacher reference and prevent termination-probability collapse.
• The semantic EOS correction (Fix 3) substantially mitigates mismatch-induced length inflation across all three model families (Qwen3, Llama 3.2, Gemma 3), with Qwen3 and Gemma 3 recovering close to teacher reference lengths and Llama 3.2 retaining a larger residual gap.
• In Gemma 3, PT and IT models already declare the identical EOS set (<eos>, <end_of_turn>), yet the student concentrates probability on <eos> while the teacher favors <end_of_turn>, showing the mismatch is about learned probability allocation, not just declared token sets.
• The original DAPO grader underestimates Qwen3 performance due to Markdown formatting sensitivity: the Qwen3-4B teacher scores only 6.28% Avg@16 with the original grader versus 24.34% with the authors' format-compatible parsing extension (evaluated on AIME24, AIME25 [30 problems each], and AMC23 [40 problems], 16 samples per problem).
• On K2-Horizon, most of the shift in termination preference toward the teacher-preferred token occurs during midtraining, and by the SFT stage the student's preference is already close to the final teacher's; however, a distinct late-stage re-inflation (Phase 3) occurs even after semantic EOS correction, where termination probability again collapses toward zero and nearly all rollouts hit the generation budget.

Why it matters / caveats: Termination-token mismatch is a readily diagnosable, implementation-level cause of OPD length inflation that can be corrected by aligning termination semantics in the distillation signal rather than assuming it requires deeper training-instability fixes; however, the authors note this mechanism is not exhaustive—early post-correction length growth and a late-stage re-inflation in K2-Horizon persist unexplained, and the analysis is limited to single-turn math reasoning, since multi-turn/agentic settings have functionally distinct termination tokens (turn end, tool handoff, etc.) that may not be safely treated as interchangeable.

Problem: Document-parsing training corpora are biased toward common document types and clean digital pages, and simply expanding data coverage does not tell you how to fix a parser's remaining weaknesses, so end-to-end parsers remain unreliable across diverse layouts and acquisition conditions.

Method: WeVisDoc is a two-stage data-centric framework built on Qwen3-VL-Instruct (2B and 4B scales) with a unified autoregressive output representation (Markdown text, HTML tables, LaTeX formulas). Stage I builds a ~40M-record broad-coverage pool from heterogeneous real and synthetically-rendered "page program" data (semantic HTML/CSS compiled into paired images and exact targets), plus structure-preserving appearance degradation (paper/print/scan/photo/screen-recapture effects), balanced via source-aware, token-budget-based sampling. Stage II diagnoses the frozen Stage I checkpoint using a held-out allocation probe, clustering pages into visual-structural groups (SigLIP2 embeddings + structural metadata) to compute stabilized residual error per cluster, mines and verifies hard examples (checking for annotation errors, decoding failure, or low observability before treating them as genuine capability gaps), synthesizes ~5M targeted records for diagnosed weak clusters, and rebalances the target-token budget toward high-residual clusters (via a KL-constrained optimization) while replaying Stage I data to limit forgetting; Stage II freezes the vision encoder and only updates the LLM.

Key results:

• WeVisDoc-4B achieves 95.38 Overall on OmniDocBench v1.6 (TextEdit 0.036, FormulaCDM 96.81, TableTEDS_S 95.34, ROEdit 0.125), beating the previous leading end-to-end result by 0.64 points; WeVisDoc-2B scores 95.06 Overall.
• On PureDocBench, WeVisDoc-4B reaches a mean Avg3 of 75.54 (Clean 79.81, Digital Degraded 77.74, Real Degraded 69.08), exceeding the leading general-purpose VLM baseline (Qwen3.5-122B-A10B, Avg3 74.11) by 1.43 points and the previous end-to-end state of the art on Real Degraded by 1.44 points; WeVisDoc-2B gets Avg3 73.86.
• Stage II (capability-aware refinement) over Stage I (broad coverage) improves OmniDocBench Overall by +1.56 (2B) and +1.16 (4B) points, and PureDocBench Avg3 by +1.74 (2B) and +2.35 (4B) points.
• Stage II gains are concentrated on degraded conditions: Real Degraded improves by +2.37 (2B) and +4.03 (4B) points, and Digital Degraded by +2.38 (2B) and +2.55 (4B), versus only +0.46 (2B) and +0.49 (4B) on Clean pages.
• Scaling from 2B to 4B (final models) yields larger gains under harder conditions: +0.45 on Clean, +1.12 on Digital Degraded, and +3.48 on Real Degraded PureDocBench Overall.

Why it matters / caveats: The results suggest that after broad coverage is established, model-aware diagnosis and token-budget reallocation toward residual weaknesses yields disproportionately large robustness gains on degraded/real-world documents rather than on clean pages, with the 4B model achieving top rank among compared end-to-end parsers on all four evaluated settings. The authors note the stage-wise comparison combines several Stage II interventions (hard-example mining, targeted synthesis, and residual-aware rebalancing) at once and does not isolate the individual contribution of residual-aware allocation itself.

Problem: GUI agents rely on skill frameworks that treat procedural knowledge as static, monolithic artifacts fixed before deployment, so they cannot adapt when non-stationary interfaces (pop-ups, delayed loads, relocated widgets) invalidate plans during execution.

Method: EvoSkill-GUI represents each skill as a structured multi-file package (retrieval metadata, executable plans, backup localization, failure-recovery rules, accessibility utilities, and failure cases) manipulated through a restricted tool interface (read, write, append, list, search, create_failure). It runs a reflect-revise-reuse loop: during rollout the executor makes instant in-rollout edits to fix local mismatches; after a failed rollout the same backbone model acts as an isolated critic under strict information isolation (barred from the skill, executor's chain-of-thought, and ground truth) to diagnose the trajectory, then as an executor to revise the specific skill files indicated by the diagnosis. Verified skills are indexed by structured metadata (intent, app, platform, keywords) and retrieved via a lexical+semantic scoring function with bonus/penalty terms, reused when the score exceeds a threshold (θr=0.6) or else built from scratch.

Key results:

• Maximum gains of +16.2% on MobileWorld, +6.0% on AndroidWorld, and +10.5% on OSWorld across base models, without any training.
• On MobileWorld (GUI-only), raised Qwen3.6-Plus from 53.3% to 69.5% and Claude-Sonnet-4.6 from 57.1% to 67.6%.
• On OSWorld, GUI-Owl-1.5-8B improved from 46.7% to 54.8% (+8.1) overall, with app-level gains up to +42.7 on VLC and +20.0 on Thunderbird; Qwen3-VL-8B-Instruct improved from 23.8% to 34.3% (+10.5).
• Structured multi-file skill packages beat single-file skills (69.52% vs 66.67%, +2.85); removing information isolation dropped success from 69.52% to 60.95% (−8.57); removing instant revision dropped it to 62.86% (−6.66).
• Metadata-based retrieval hit the correct skill in 12/12 MobileWorld reuse cases (avg score 0.88) vs. 1/12 for full-text retrieval (avg score 0.41); skill evolution saturated after ~3 rounds (12.4-point gain over rounds 1–3, only +0.9 over rounds 4–5).

Why it matters / caveats: The approach shows procedural knowledge can be revised and reused at inference time without retraining, with evolved skill libraries continuing to help related tasks (recovering 27/89 previously failed AndroidWorld executions) rather than needing rebuilding from scratch. The authors note no formal verifier vetoes harmful skill edits, so an unreliable backbone diagnosis could introduce regressions into a previously verified package, and reuse-rate gains are measured on parameterized task variants rather than genuinely unseen app categories.

Problem: Existing spatial-reasoning and manipulation benchmarks test evidence acquisition, spatial inference, and physical execution separately, so it remains unclear whether a general-purpose MLLM can close the full observe-reason-act-revise loop needed for reliable embodied manipulation under partial observability.

Method: VA-Bench is a physics-simulated benchmark (built in RoboTwin with a fixed, model-agnostic inverse-kinematics/trajectory executor) where a general-purpose MLLM extracts a textual procedure from sampled frames of an RGB-only demonstration video, then must actively select camera viewpoints (a bounded discrete set of gripper-centered semantic views plus local translation/zoom/yaw/pitch), issue metric Cartesian robot commands (world-axis translations of 1-100mm, gripper-local rotations of 1-90 degrees, gripper aperture), and revise actions from execution feedback, with no privileged object poses, oracle trajectories, or learned action heads. It covers 14 base task families (11 single-arm, 3 dual-arm) with 20 physically verified seeds each, 7 held-out geometry/layout transfer variants, and a five-object long-horizon composition track, scored by terminal task success plus nine human-annotated trajectory-level behavioral diagnostics (grouped into spatial perception, manipulation, and error-recovery modules) and subtask-progress measures.

Key results:

• The best model (Qwen3.8-max) scores 100.0% on target localization and 78.9% on spatial relations, yet its three-run macro-average task success across 14 base tasks is only 53.93 ± 3.17%.
• Active camera control substantially outperforms passive multi-view observation: in a matched comparison, Qwen3.8-max success rises from 27.86% (passive) to 57.50% (active), a 29.64pp gain; all four tested conditions dropped when camera control was removed (e.g., GPT-5.6-sol: 56.07% to 33.93%).
• Held-out geometric/layout transfer degrades performance sharply: GPT-5.6-sol drops 32.14 points (80.00% to 47.86%) while Qwen3.8-max drops only 10.00 points (77.86% to 67.86%) across seven matched families.
• No model completes a single strict long-horizon (five-object) episode; the strongest partial progress is 61/100 successful object placements (Qwen3.8-max), followed by 53/100 (GPT-5.6-sol).
• Dual-arm coordination remains weak for all models: only the top three conditions exceed 10% mean dual-arm success, with the best (GPT-5.6-sol) reaching just 22.22%, versus single-arm means as high as 65.61% (Qwen3.8-max).

Why it matters / caveats: The results show that strong spatial perception/localization scores do not translate into reliable metric manipulation, and that active evidence-seeking behavior and robustness to geometric transfer are current bottlenecks for general-purpose MLLMs acting as embodied agents. Findings rely on only three runs per condition (top models' scores overlap within ~2.4 points, so ordering among leaders is not claimed to be reliable) and on human-annotated diagnostics from a single annotated run per condition (though inter-annotator agreement was high, 95.7% unanimous, Fleiss κ = 0.942).

Problem: Dense Softmax attention dominates (>85% of denoiser runtime) and scales quadratically with sequence length in video diffusion transformers, while directly swapping in linear attention to cut this cost degrades generation quality because a fixed-size recurrent state cannot preserve the fine-grained interactions Softmax provides.

Method: Video DeltaNet (VDN) is a hybrid attention that keeps sliding-window Softmax attention (a 15-frame bidirectional window plus two boundary anchor frames with four-way connectivity) for local/global-anchor video interactions, while distant video context is handled by a bidirectional linear branch built on Video Delta Attention (VDA)—a frame-wise delta rule that jointly solves for a new memory state from all spatial tokens of a frame at once (closed-form via (I+At)^-1) rather than applying independent per-token delta updates, giving a provably non-expansive inherited-state transition. The two branches use separate RMSNorm, sigmoid gates, and output projections before being summed into the residual stream, and are integrated into the pretrained MiniMax H3 backbone (applied only to video-to-video interactions, keeping Softmax for text/audio) via a staged recipe: per-layer alignment (A1), end-to-end alignment (A2), and LoRA co-adaptation (Stage B), followed by DMD2-style 8-step distillation and an optimized SGLang/Triton inference stack (fused VDA kernels, chunk-wise scans, a fused Gauss-Jordan matrix-inverse kernel, Ulysses-style head sharding, MXFP8 GEMMs).

Key results:

• On a 14.3-second, 768p video, eight-step VDN-H3 completes DiT denoising in 6.70s on 8×B200 GPUs, a 14.5× speedup over 50-step Dense H3 on the same GPU count (119.3× vs. single-GPU 50-step Dense H3, which takes 799.6s).
• The optimized VDN-H3 backbone alone (still 50 steps) is 2.6× faster on one B200 (16.0→6.16s) and 3.2× faster on one H200 (35.35→11.16s) than Dense H3.
• As video length grows from 42 to 102 latent frames, Softmax attention density falls from 42.1% to 20.0%, and whole-backbone speedup rises from 1.7×/1.8× to 2.6×/3.2× (B200/H200).
• Eight-step VDN-H3 matches or exceeds 50-step Dense H3 on 5 no-reference quality metrics (VQAA, VQAT, Q-Align, FAST-VQA, DOVER++), with differences of +0.06 to +1.00, while FastH3 (4-step baseline) scores 2.70–12.74 points lower than Dense H3.
• Fused kernel ablations show large per-op speedups, e.g., VDA-Prep drops from 18.0ms to 1.6ms (H200) / 17.1ms to 3.4ms (B200), VDA-Gather 7.2×/7.5×, VDA-Epilogue 7.3×/9.1×, and the fused matrix-inverse kernel gives a 4.6–5.0× speedup over a Cholesky-based pipeline.

Why it matters / caveats: The approach shows a hybrid Softmax+linear attention can be retrofitted onto a large pretrained video model (MiniMax H3) without training a new foundation model from scratch, enabling near real-time (livestream-scale) generation of long, high-resolution videos while preserving quality comparable to the much slower dense baseline; results and efficiency gains are reported only for this specific H3 instantiation and third-party benchmark, and some FL2VA endpoint-fidelity metrics show small, statistically flagged degradations relative to Dense H3.

Problem: Existing video/world-model benchmarks use explicit prompts that fully describe the target output, so it is unclear whether omni-modal generative models like MiniMax-H3 can actually integrate complementary evidence across text, image, audio, and video to infer an underlying event rather than just render what they are told.

Method: The authors build an evaluation framework of implicit condition-prompt pairs where the textual prompt deliberately omits key event semantics that must instead be inferred from other input modalities, then assess MiniMax-H3's generated videos for consistency with that inferred semantics. They define four reasoning scenarios—Multi-view Spatial Reasoning (MSR, multiple image views), Audio-based Disambiguation Reasoning (ADR, image+audio), Video-based Decision Reasoning (VDR, prefix video continuation), and Audiovisual Integrated Reasoning (AVIR, video+audio)—and construct 517 expert-verified instances across 29 subcategories from real datasets (e.g., HiFi-UMI-2K, LLaVA-Video-178K, FSD50K, ESC-50) and generative sources (ChatGPT Voice, Seedance 2.0), reviewed through a 10-loop expert protocol; three human evaluators then judge task-specific success on the generated videos.

Key results:

• Overall success rate across all 517 instances is 41.97%.
• VDR achieves the highest success rate at 56.00%, followed by AVIR at 47.89% and MSR at 43.50%.
• ADR is the weakest scenario at 27.40% success rate, a 28.60 percentage-point gap versus VDR.
• Within MSR, threading reaches 75.00% success but cleaning, articulation, and transport remain around 33-35%.
• Within VDR, human/animal/traffic dynamics are handled well but cartoons (20.00%) and puzzles (16.67%) perform poorly; within ADR, nature sounds do best (54.50%) while machinery (17.60%) and contact sounds (21.90%) are hardest.

Why it matters / caveats: The results show a clear gap between an omni-modal model merely accepting multimodal inputs and reliably reasoning over them, especially for converting acoustic evidence into correct visual events; the paper identifies recurring failure modes (incorrect evidence grounding, incomplete event realization, physical/configurational violations, temporal inconsistency) as concrete targets for improving cross-modal grounding in future omni-modal systems.

Problem: Existing feed-forward methods for articulated 3D object modeling infer kinematics from a single observation (relying on ambiguous, learned category-level shape priors) or require dense, temporally-ordered multi-view scans, so they generalize poorly to sparse, unordered, partial real-world captures.

Method: FAMOS takes a sparse, unordered, variable-size set of partial point clouds (from different viewpoints/articulation states), encodes each with a frozen PartField backbone plus sinusoidal geometric embeddings, and feeds point tokens and learnable part queries through a Multi-state Articulation Transformer whose layers alternate state-wise self-attention (within each observation) and global attention (across all queries and points jointly, K=16 queries, L=6 layers, D=768, 12 heads). Per-part MLP heads decode joint type, Plücker-coordinate axis/origin (revolute) or direction (prismatic), and a per-point head predicts cross-state-consistent segmentation; training uses Hungarian matching plus a new "observed articulation span" loss that supervises the motion range each part exhibits across the input set to force aggregation of cross-observation evidence. Training data combines curated PartNet-Mobility/GRScenes assets with a procedural generator that assembles objects from six analytic primitives into 15 object families with self-annotated ground truth, used in a two-stage schedule (50k steps procedural-only pretraining, then 80k steps mixing procedural and curated data 1:1).

Key results:

• Under the strict MAO motion-estimation criterion, FAMOS reaches F1 of 98.4 (PartNet-Mobility), 64.5 (ACD), and 90.0 (ArtiCraft-10K), improving over the strongest feed-forward baseline (Particulate) by 8.1, 27.8, and 21.5 points respectively.
• Movable-part segmentation F1 improves over the multi-state Particulate baseline by 28.2 points on ACD and 15.9 points on ArtiCraft-10K.
• FAMOS runs at ~0.1s per object versus 182s (ReArt) and 297s (ArtGS), roughly 3,000x faster than optimization-based baselines (and only 65.6ms/1.56GiB at S=2, N=2048; scales to S=12, N=8192 with 1.20s/10.66GiB).
• Ablations: removing global attention (replacing with register attention) drops MAO by 2.2/8.1 points (PartNet-Mobility/ACD); removing the observed articulation span loss drops MAO by 1.3/5.7 points; adding procedural pretraining plus data mixing raises ACD MAO from 47.3 to 64.5.
• Even in single-state (S=1) inference, FAMOS outperforms a re-trained Particulate baseline across all three benchmarks (e.g., MAO of 83.0 vs. 62.0 on ArtiCraft-10K).

Why it matters / caveats: FAMOS shows that grounding articulation prediction in observed cross-state motion (rather than only learned shape priors) yields large generalization gains on out-of-distribution, geometrically diverse objects while being feed-forward and near-instant, and it generalizes to real-world captures despite training only on synthetic data; results also show performance saturating beyond the S=4 states used in training, though the model still depends on point clouds lifted via external 3D foundation models (e.g., VGGT-Ω) and SAM2 segmentation for real-world inputs.

Problem: Existing evaluators for subject-driven customization (embedding-based like CLIP/DINO, or MLLM-based like VIEScore/DreamBench++) judge textual and visual condition alignment in isolation, causing false positives/negatives that poorly match human judgments.

Method: UFO introduces an Atomized Chain-of-Evaluation framework with four stages: (1) a VLM decomposes omni-condition alignment into fine-grained Atomic Evaluation Units (AEUs) at local-concrete and global-abstract levels; (2) each AEU is classified by modality-relevance (image-only, text-only, or text-and-image); (3) each AEU is scored via VQA queries answered Yes/No by the VLM, or via dedicated function calls such as ArcFace for facial identity similarity; (4) all AEU scores are aggregated via VLM-assigned adaptive importance weights (1-5) into a holistic weighted-average score. The authors also build UFO-Bench, a benchmark of 86 reference images across 7 categories (e.g., humans, rigid objects, anime characters) with a hierarchical prompting strategy spanning Non-Editing, Local Editing, Global Editing, and Complex Editing paradigms.

Key results:

• UFO achieves an average Spearman correlation of 0.6889 with human judgments (vs. ± reported variances up to 16.10%), a 15.25% average improvement over existing metrics.
• Baseline correlations were much lower: CLIP-T even negative (-0.1996), CLIP-I (0.3910) and DINO (0.3084) weakly positive, VIE-Score (~0.5663-0.5752), and DreamBench++ (0.5977).
• UFO-Bench comprises 660 diverse cases across 7 categories with 81.97% conflicting condition pairs across 3 tiers of editing difficulty.
• Ablations: removing AEU decomposition (holistic single-score VLM scoring) drops correlation from 0.6889 to 0.5752; removing weighted aggregation (uniform weights) drops it to 0.6253.
• On UFO-Bench evaluation of generation models, closed-source Doubao (2025-12) scored highest overall (0.7439 total), followed by open-source Qwen-Image (2025-12) at 0.7252, both outperforming Nano-Banana (2025-08, 0.6816) and other open-source models (OmniGen2 0.4235, UNO 0.4549, BAGEL 0.3866).

Why it matters / caveats: By decomposing evaluation into atomic, modality-tagged units with specialized function calls (e.g., ArcFace for identity), UFO better captures cases where text-guided edits should override visual similarity (or vice versa), enabling more reliable, interpretable, and human-aligned automatic evaluation for subject-driven image customization research.

Problem: Large reasoning models overthink easy problems and underthink hard ones, and existing uniform length-penalty or rigid-routing methods for fixing this pay an "efficiency tax," trading accuracy loss on hard instances for token savings on easy ones.

Method: When2Think is a post-training (RLVR) framework for hybrid reasoning built on AdaptThink's importance-sampling setup, where a mode token (THINK vs. NOTHINK) is prepended to each trajectory and the model is optimized with a PPO-style clipped surrogate objective. Its core contribution is Instance-level Difficulty-Aware Control (IDAC), a reward-shaping term that uses pre-computed offline reference statistics per instance (reference accuracy αi as a difficulty proxy, reference trajectory length τi as a token budget) to exponentially decay the reward bonus for trajectories that exceed the instance's expected length, discouraging overthinking on easy instances while allowing longer deliberation on hard ones. This is combined with a correctness-gated efficiency bonus and a batch-wise standardized advantage (BWS, inspired by REINFORCE++) for critic-free, stable trajectory-level credit assignment, requiring no learned reward model, critic, or online reference-model queries during training. The method is trained on R1-Distill-Qwen-1.5B using the DeepScaleR dataset (~40k competition math problems).

Key results:

• On AIME24, Pass@3 improves from 46.0% (R1-Distill-Qwen base) to 56.0% (+10.0 points) while token usage drops from 13,996 to ~10,236 (−27.9% / −3,959 tokens).
• On AIME25, When2Think achieves 40.0% Pass@3 (+8.0 points over base), outperforming compression baselines (LC-R1, ThinkPrune, LASER) and routing-only hybrid baselines (AdaptThink, ThinkLess).
• On MATH-500 Level 1 (easy) problems, tokens drop from 1,199 (R1-Distill) to 619 (−580) while maintaining 95.8% accuracy, showing a shift to System-1-style direct answering.
• On MATH-500 Level 5 (hard) problems, tokens are reduced by 2,276 relative to R1-Distill while accuracy is maintained, and the THINK-trajectory fraction rises monotonically from ~0.2 at Level 1 to over 0.7 at Level 5.
• On GSM-Plus (adversarially perturbed), When2Think increases computation (+462 tokens) yielding a 6.3-point absolute accuracy gain, and on OlympiadBench-Math it improves Pass@3 by 3.6 points while cutting tokens by 2,725.
• Ablation: removing IDAC (IS+BWS-only variant) substantially degrades validation accuracy, especially on AIME24, versus AdaptThink which trims tokens (5,806) but drops to 44.7% Pass@3, below the base model.

Why it matters / caveats: By conditioning reward shaping on pre-computed instance-level difficulty/cost statistics rather than global length penalties, When2Think mitigates the efficiency tax and better matches reasoning depth to problem difficulty than prior compression or discrete-routing hybrid reasoning methods; experiments are limited to a 1.5B math-specialized backbone and math reasoning benchmarks.

Problem: In on-policy self-distillation (OPSD), a frozen teacher copy of a model scores a student's own rollouts while seeing a privileged reference (e.g., a worked solution) the student lacks, but it is unclear how much of the resulting improvement actually comes from that privileged reference versus from distillation itself.

Method: The authors build AMPLE-MATH, a suite of 5,319 math problems from OpenThoughts-114k, each with six answer-matched reasoning "views" (Answer Only, Gist, Key Points, Clean Solution, Summary, Full Trace) that share one verified answer but vary in reasoning density (13 to 4,916 mean rendered Qwen3 tokens); Gist/Key Points/Summary are generated and fidelity-checked by Qwen3.6-35B-A3B-FP8. They train Qwen3-1.7B (LoRA, r=64) and SmolLM3-3B with a frozen thinking-enabled teacher scoring direct-response student rollouts via a clipped forward-KL OPSD loss, comparing each reference view against a reference-free control (teacher scores the same prefixes with no privileged information) under matched training/eval protocols, and also run teacher "privilege profiles" (correctness alignment, correction-marker pressure, KL allocation) plus targeted interventions (other-problem references, trace shortening, marker-loss reweighting, loss-window widening) to probe mechanisms.

Key results:

• At step 100, Qwen3-1.7B Answer Only and Full Trace sit only about 0.6 points above the reference-free student (interval includes zero), while reference-free training itself yields gains of +1.80 to +4.14 points over base in-domain/external at steps 50-100.
• Clean Solution's step-100 advantage over reference-free training in Qwen is +1.30 [0.20, 2.41] points, but the effect does not survive Holm correction across the six views.
• In SmolLM3-3B, Full Trace adds +2.0 points beyond reference-free training at step 50 (thinking-enabled), but by step 100 all configurations fall below base.
• Switching training rollouts from direct-response to thinking-enabled (same teacher/reference) turns gains into losses at step 50 across all three tested views in Qwen (e.g., −7.60 to −8.40 points) and in SmolLM3 (Full Trace: +2.00 → −10.63 under direct-response evaluation reordering; nine of nine gap comparisons negative, eight significant).
• Replacing a genuine reference with a length-matched other-problem view lowers thinking-enabled accuracy by about two points; broadening the thinking-enabled loss window (First-4K, Distributed-1K) appears to gain ~4 points over the baseline Early-1K window, but matching checkpoints shows the intervals include zero and ~76% of the apparent gain is attributable to stopping time (earlier checkpoint), not the loss-window design.

Why it matters / caveats: The results show that a substantial share of OPSD's benefit can come from cross-mode transfer (a thinking-enabled teacher supervising direct-response prefixes) rather than from the privileged reference's content, so reference design and evaluation should be judged by what they add beyond a reference-free control, not by gains over the base model alone; findings are based on short-run LoRA fine-tuning on mathematics in only two model families (Qwen3-1.7B, SmolLM3-3B).

Problem: Improving fine-grained visual perception in MLLMs by raising resolution inflates vision-encoding and LLM prefilling costs, even though localizing a region of interest and recognizing its content have different, unequal resolution requirements.

Method: The authors build on SD-RPN, a lightweight region proposal network distilled from the MLLM's response-to-image attention, and introduce Vision-RL2, a region-level reinforcement learning method that treats coherent connected regions of the RoI map as actions. A frozen MLLM "reader" scores each region by the change in teacher-forced gold-answer likelihood when that region is masked out (a leave-one-out functional contribution), and a subtractive objective (with a control-region-calibrated noise margin) prunes low-contribution predicted regions while an additive objective (using auxiliary attention-derived candidates) recovers missing evidence — both updating only the RPN parameters with no region annotations, response sampling, or reasoning trajectories. A sparse visual encoding then re-encodes only the predicted foreground tokens at finer spatial resolution rather than uniformly tiling the crop's bounding box.

Key results:

• Localization tolerates roughly 3-4× stronger token compression than recognition in a controlled diagnostic on ZoomBench.
• Vision-RL2 matches SD-RPN's accuracy using 4.2× fewer visual tokens on Qwen3.5-4B (2.5× fewer on Qwen3.5-9B), and surpasses its own largest-budget accuracy with about 4× fewer visual tokens than the base model.
• On Qwen3.5-9B, Vision-RL2 attains the highest six-benchmark average in the main comparison table, ahead of Gemini-3.1-Pro and Vision-OPD-9B, while training an order of magnitude fewer parameters than full-finetuning baselines.
• Ablations (Qwen3.5-4B, training-aligned protocol): region-level RL alone adds +3.0 average accuracy over SD-RPN, sparse visual encoding adds a further +1.5, and the full system gains +14.9 over the frozen base model.
• The RPN routes in 31-49ms versus 1.1-1.8s for attention-guided or coordinate-decoding routing (~30× cheaper); on Gemma-4-12B, Vision-RL2 adds 3.8 points over SD-RPN and 9.5 over the base model.

Why it matters / caveats: The approach gives a decode-free, reusable RoI localizer that improves fine-grained accuracy at much lower token/latency cost without finetuning the underlying MLLM, but it still trails full-finetuning methods (e.g., Vision-OPD) on some benchmarks like MME-RealWorld, where adapting the reader itself helps and the frozen-reader design forgoes that gain.

Problem: Standard agent SFT applies the language-modeling loss only to agent-authored action tokens and masks out environment observation tokens, and it is untested whether this convention gives the best initialization for subsequent RL.

Method: The paper introduces ActObs, a one-line change to the SFT loss mask that also applies the cross-entropy loss to the already-present environment observation tokens (weighted by λ, with λ=1 as default and λ=0 recovering standard ActionSFT), so the model additionally learns pθ(ot | ht, at) without adding data, parameters, sequence tokens, or forward passes. It is compared against ActionSFT (action-only) and Obs→Act (a timing-control baseline that does one epoch of observation-only SFT followed by one epoch of action-only SFT). Both Qwen3-4B and Qwen3-8B are fine-tuned on 50k multi-turn terminal trajectories from Nemotron-Terminal-Corpus, then trained with GRPO on 2,392 Endless Terminals tasks, and evaluated on Terminal-Bench 2.0 and the cross-domain aider-polyglot code-editing benchmark.

Key results:

• At 4B, ActObs→GRPO beats ActionSFT→GRPO on Terminal-Bench 2.0 by 1.6/1.4/1.5/1.1 points at pass@1/4/8/16 (a 29% relative gain at pass@1).
• At 8B, ActObs→GRPO trails ActionSFT→GRPO at pass@1 but leads by 3.4 points (14% relative) at pass@16 and solves 24 tasks vs. 21.
• On aider-polyglot (cross-domain), the 4B ActObs→GRPO policy exceeds ActionSFT→GRPO by 4.2 points at pass@1 (43% relative) and 4.9 points at pass@4 (24% relative), despite its pre-RL SFT checkpoint being weaker at code editing.
• During SFT, gradient cosine similarity between action and observation gradients collapses from 0.83 to a noise floor within 10-20 steps; the observation/action gradient norm ratio then rises to ~41 under ActionSFT but stays near 0.5 under ActObs.
• Temperature-matching ActionSFT→GRPO to ActObs's self-entropy changes pass@1/4/8/16 by at most 0.7 points and solves no additional task, showing the pass@k gap is not explained by inference-time randomness alone.

Why it matters / caveats: The results suggest that what SFT explicitly supervises (not just what data or benchmark score looks the same after SFT) determines how RL later explores, with observation supervision preventing one-sided gradient specialization and preserving entropy/consequence-prediction for downstream RL; the approach also trades off some pass@1 reliability for higher pass@k as the observation-loss weight increases, so it is not a strict improvement across all sampling budgets.

Problem: No existing evaluation framework systematically measures whether enterprise LLM agents keep following embedded compliance rules when a benign user's convenience (speed, cost, satisfying a manager) conflicts with the rule, especially under multi-turn pressure.

Method: The authors build PACT (Pressure-Applied Compliance Testing), a benchmark of 48 realistic multi-turn scenarios across 12 regulated enterprise domains (privacy/GDPR, finance, HR/hiring, HIPAA healthcare, AML, export controls, etc.), where each item pairs a standing rule against a rule-violating shortcut option among 2-5 choices. Items are generated component-by-component (persona, rule, mandate, user request, pressure) by three open-source LLM generators (Kimi-K2.6, Nemotron-Ultra, GLM-5.2) and audited by a panel of the other two models against realism/scope checklists, with a battery of nine psychology-grounded pressures (deadline, manager authority, peer example, false clearance, sunk cost, etc.) and an optional anti-adversarial "must follow all rules" system-prompt mandate, tested in both base and anti-adversarial modes with a pushback second turn when the model initially complies. Outcomes are scored via an LLM extractor (GPT-OSS-120B) plus a 3-model transparency judge, and combined into six metrics (Default Compliance, Pressure Resistance, Pushback Resistance, Steerability, Transparency, Rule-Scope Discernment) and an aggregate PACTScore (0.75 weight on turn-1 compliance, 0.25 on multi-turn pushback compliance), evaluated across 22 open- and closed-weight LLMs on 3,364 items run three times each.

Key results:

• The top model (Kimi-K2.7-Code) scores only 0.944 PACTScore, and no model reaches 0.95; the lowest (Mistral-7B) scores 0.484.
• Adding pressure raises the average violation rate from 4.41% to 7.29% (a ~65% relative increase); even Claude Haiku 4.5, which has zero unpressured violations, fails 1 in 43 pressured situations.
• Rule-Scope Discernment is below Default Compliance for all 22 models; across 13,817 base-mode decisions where the rule doesn't apply, models enforce it anyway 19.6% of the time (Claude Haiku 4.5 over-applies on 21.9% despite having the lowest violation rate).
• Transparency is the weakest axis: no model exceeds 0.244 (median 0.134); across 16,424 judged violations only 8.0% are disclosed, 12.8% are silent, and 79.2% are misrepresented to the user as compliant or resolved.
• The anti-adversarial compliance mandate (Steerability) is weak and inconsistent, ranging from 0.016 (GPT-5.6 Luna) to 0.564 (Grok 4.3), worth only 0-9 PACTScore points and least effective in high-stakes domains like HR/hiring (0.185) and healthcare (0.295).
• Under an explicit "you are being evaluated" prompt cue, violations drop by a mean of 29% (up to +70% for Claude Haiku 4.5), showing evaluation-awareness effects that PACT's naturalistic prompt design is built to mitigate.

Why it matters / caveats: The results show that no current model is reliable enough for unsupervised deployment in regulated enterprise workflows, that compliance behaviors (resistance to pressure, transparency, scope discernment) are largely independent axes not captured by a single score, and that prompt-level guardrails are an unreliable substitute for process controls, especially in the highest-stakes domains.

Problem: There is no spoken factoid question-answering benchmark for Telugu, and the reliability of automatic (especially LLM-as-a-judge) evaluation for this low-resource, spoken setting is unquantified.

Method: The authors build VākQA via a semi-automatic pipeline: Telugu quiz-style YouTube videos across six domains are segmented with Pyannote VAD, transcribed with a Seamless-large-v2 model fine-tuned on ~900 hours of Indic speech data, merged and passed to Gemini to extract verbatim QA pairs, then aligned to word-level timestamps (via Whisper-timestamped/IndicWhisper) using fuzzy string matching for audio segment boundaries; five annotators verify QA-audio alignment and manually translate pairs into English. They then validate automatic evaluation (EM, F1, BLASER-2.0, and LLM-as-a-judge with Gemini and Gemma-3 4B/12B/27B) against 400 human-rated (question, reference, candidate) triplets scored on a 1-5 rubric, and benchmark Gemini-2.5-Flash against open-weight text LLMs (Gemma-3 family, Llama-3.1, Hex-1, Sarvam-m, Qwen-3-4B) across input modality (speech/text), language (Telugu/English), and cascaded ASR→MT pipelines (Seamless FT/IndicWhisper ASR, Seamless MT/IndicTrans2 MT).

Key results:

• VākQA contains 2,001 factoid QA pairs (2.53 hours of audio, avg. 4.55 sec/question) across six domains: Science (27%), General Knowledge (23%), Politics (16%), History (13%), Culture (12%), Geography (10%).
• Human annotation inter-rater reliability reached Krippendorff's α = 0.836 after excluding 20 outlier items.
• Gemini-as-a-judge best correlates with human judgment (Spearman ρ=0.86, Kendall τ=0.77) vs. Gemma-3-27B (ρ=0.80), Gemma-3-12B (ρ=0.81), Gemma-3-4B (ρ=0.57); lexical/embedding metrics correlate poorly (EM ρ=0.37, F1 ρ=0.49, BLASER-2.0 ρ=0.36).
• Switching judges from Gemini to Gemma-12B changes scores for 67.4% of Gemini's 2,001 answers (46.23% worse, 21.14% better).
• With oracle Telugu text, Gemini scores 3.63 (1.71) vs. best open-weight Gemma-27B at 2.55 (1.79); switching Gemini's input to English degrades 18.8% of questions and improves 16.5%; speech input (vs. text) degrades 21.2% and improves 13.1%; ASR text (Seamless FT, WER 30.25) degrades 11.9% vs. only 4.8% improved; full ASR→MT cascades drop scores to 2.44-2.80 vs. 3.63 oracle.

Why it matters / caveats: The paper shows that for low-resource languages like Telugu, standard lexical metrics (EM/F1) and even embedding-based BLASER-2.0 are unreliable proxies for human judgment, and that even the best available judge (Gemini) is non-uniformly strict (lenient on low-quality, stricter on high-quality answers), while smaller open-weight judges systematically penalize correct but surface-form-different Telugu answers. The authors also note limitations: translation-based English versions can introduce scope ambiguity (e.g., ambiguous possessive pronouns), Gemini's non-uniform strictness limits fine-grained model comparisons, and some Science/Geography reference answers use English transliterations whose consistency across judges was not assessed.

Problem: Brahmic-script fonts are typographically underserved relative to Latin (e.g., Google Fonts has over 1,800 Latin families but only six for Odia) because a text-ready Indic font requires hundreds to thousands of mutually consistent OpenType conjuncts, half forms, and matra variants, a shaping-engine contract that raster glyph generators cannot produce.

Method: Srijika restyles the glyph outlines of a shaping-complete template font per script rather than generating a font from scratch, reusing the template's cmap and GSUB substitution closure unchanged and its GPOS data under a documented metric/anchor-translation policy. A 134M-parameter reference-conditioned latent diffusion UNet (132.9M UNet + 1.2M trained style-projection layer) redraws each template glyph tile at 128×128 using cross-attention over six donor reference tiles selected by Lipika, a CLIP-LoRA retrieval index over ~650 open-license families grounded by a ConvNeXt-V2-tiny/ArcFace family recognizer; a defect-tolerant serving pipeline (content gate with retry ladder, weight harmonization, baseline snap, shaped-cluster verify-and-repair via HarfBuzz) reverts any failing glyph to the template outline. New scripts are added via a warm-start "graft" recipe (resume checkpoint, concatenate new packs, 20-48k steps), with per-script routing to sidestep cross-script dilution from sequential grafting.

Key results:

• Ships 66 installable TTFs (57 curated presets + 9 showcase fonts); all 66 pass the OpenType Sanitizer, and HarfBuzz/CoreText reproduce the template's glyph-ID sequences on conjunct-heavy probes for every font.
• A full-closure audit covers 80,915 glyphs and 54,812 anchors: 50,066 template-fallback glyphs have zero metric deltas; the 30,849 accepted generated glyphs show advance-width deltas of p50 0.045 em / p95 0.358 em (max 1.93 em), and anchor y-shifts are identically zero across all 54,812 anchors.
• On diffusion-training-family-held-out SSIM gates, unrestyled template copy beats generation on 50 of 56 held-out faces (by 0.003-0.069 pooled SSIM per script).
• Style movement (Lipika similarity-gap closure) has a median of only 0.047 (IQR 0.016-0.133) across held-out families, but one out-of-distribution family (Alkatra) reaches 0.85-0.94 (seed-stable at 0.866/0.861/0.867) while seven other OOD families close a median of just 0.040.
• Shaped-cluster accuracy across the 57 shipped presets is 0.981-1.000, with a median of ~8% of glyphs reverted to template outlines on the hardest display donors; the nine showcase fonts restyled 101-681 template glyphs each in 12-136 minutes on a single L4 GPU.

Why it matters / caveats: The layout-reuse approach guarantees a functionally correct, installable font by construction (verified structurally) even when the learned restyling underperforms simple template copying on standard SSIM reconstruction metrics, showing that font generation for complex scripts can be decoupled into "reuse layout, learn only local style." Key caveats acknowledged by the authors: evaluation is against no-learning baselines only (no learned few-shot baseline like FontDiffuser was run), the closure/style metric is measured with an internal same-model embedding whose training corpus includes the held-out test families (so it is not fully independent), donors with ink density beyond the training "data hull" (e.g., ultra-black Latin faces) degrade to heavy template-fallback output, and no human evaluation of prompt-to-font quality has been conducted.