AI papers — 2026-09-03

Problem: Autonomous ML-research agents (a fixed model backbone plus an execution harness) lack "operational knowledge" — the domain-specific know-how for choosing methods, configuring pipelines, and using package APIs correctly — which today exists only informally scattered across repositories and papers written for humans.

Method: The paper introduces DisCo, a skill-powered research agent (built on GPT-5.5 and GPT-5.6-sol with xhigh reasoning effort, using Codex as the harness) that distills declarative sources into verified Agent Skills (SKILL.md + references/ + scripts/, per Anthropic's Agent Skills spec) organized as skill graphs. Distillation runs in a four-stage pipeline (scope → ground → construct → verify) in two forms: task-agnostic (condensing whole repos/papers into reusable skills ahead of time) and task-oriented (generating skills on demand for a specific task). Applying this at scale over 1,000 widely-used ML repositories produces the AREX-Skill Library (5,353 skills, 20 areas, 178 capability families, with a generated router for retrieval).

Key results:

• MLE-bench (75 competitions): Codex+AREX-Skill scores 72.89% Any-Medal vs. 31.11% without skills (+134.3% relative), and beats the strongest public leaderboard baseline (64.44%) by +8.45 points; gains are largest on High-difficulty tasks (13.33%→62.22%, +366.8%).
• PaperBench (20 papers, 636 paper-derived skills from 153 source papers): average replication score rises from 29.45% to 39.59% (+34.4% relative), improving 18/20 tasks.
• FrontierCS (188 Agent Track tasks): score rises from 70.63 to 77.14 (+9.2%), using fewer tokens/steps/tool calls than Claude Code + Opus 4.8 (74.5) or Qwen3.7 Max (61.9), which use ~3x more tokens.
• PassNet (compiler-pass generation, 200 samples): AS Score rises from 1.343 to 1.5313 (+14.0%), surpassing TorchInductor (1.419); correctness improves from 81.35% to 90.76% and failed samples drop from 14 to 5.

Why it matters / caveats: The results suggest that distilling and reusing operational knowledge as a separate context layer can improve research-agent performance as much as or more than a stronger control loop, without modifying the model or harness. However, a small number of tasks regress with skills (e.g., two PaperBench tasks dropped by up to 5 points), attributed to retrieval-precision failures where retrieved skills distract from a better task-specific approach the agent would otherwise find unaided.

Problem: Building interactive, long-horizon video world models is hampered by heterogeneous training data (differing temporal scale, camera geometry, quality, captioning) and heterogeneous video-generation backbones, and existing open systems couple data and model choices in ways that make results hard to reproduce or compare across backbones.

Method: SolarWM combines a reconfigurable multi-source data engine that converts 1.43M canonical clips from 10 datasets (ABOT-World, DL3DV, MiraData, RealCam, SpatialVID, Sekai-Game/Walking, MIND, MultiCamVideo, OmniWorld) into a unified frame-aligned schema (video, metric camera poses/intrinsics, dense captions from Kimi-K2.6, quality/motion metrics, provenance) with frozen source-aware filtering into xhigh/high/rejected tiers, plus an LTX-2.3 Clean Plate pipeline to remove dynamic people/vehicles from select sources. On top of this it instantiates four models (SolarWM-wan2.2-5B, SolarWM-wan2.2-14B, SolarWM-ltx-2.5-22B, SolarWM-minimax-h3-33B, built on Wan2.2, LTX-2.5, and MiniMax-H3) sharing a fused-PRoPE camera-conditioning interface, trained with a unified three-stage recipe: bidirectional camera-conditioned adaptation, teacher-forced AnyFlow autoregressive (causal) initialization, and DMD-based few-step distillation for real-time causal rollout.

Key results:

• Processed corpus: 1.43M canonical clips (~25.85 TB, 29k shards) from 10 source datasets across real, synthetic, and game domains; 471,798 clips retained as "high" quality and 404,795 as "xhigh," with 549,101 clips kept in a fully-annotated "rejected" partition.
• LTX Clean Plate processing yields 543k additional cleaned clips (298k SpatialVID-Clean, 135k MiraData-Clean, 109k Sekai-Walking-Clean).
• Model family spans 5B–33B parameters (SolarWM-wan2.2-5B/14B, SolarWM-ltx-2.5-22B, SolarWM-minimax-h3-33B) trained under one shared data/camera/training interface.
• Models trained only on 5-second sequences produce coherent, camera-controllable rollouts at 16 fps with 4-step causal sampling extending to minute-scale and demonstrated hour-scale (60-minute) horizons, both in-domain and on out-of-domain images (e.g., from GPT Image 2 or Krea), without attention-sink mechanisms or long-sequence fine-tuning.
• No quantitative benchmark metrics (e.g., FVD/FID/VBench scores) are reported; evaluation in the paper is qualitative (rollout figures across backbones and domains).

Why it matters / caveats: The release (data, processing pipeline, training recipes, and weights for four models spanning three backbone families) targets reproducibility and cross-backbone comparability, which the authors argue is largely missing from prior open world-model releases. However, the paper's own evaluation is qualitative only, with no numerical benchmark comparisons to substantiate claims of "state-of-the-art performance."

Problem: Evaluating LLM agents on agentic benchmarks is prohibitively expensive (hundreds to thousands of dollars per full pass), and prior efficiency work only reduces the number of tasks run, leaving the cost of executing each retained task untouched.

Method: EarlyEval trains a pair of LightGBM classifiers (a success predictor and a failure predictor) on historical, outcome-labeled agent trajectories decomposed into step-wise prefixes, using behavioral features (activity counts, timing, working patterns, error/test status), textual features (TF-IDF + SVD embeddings of prompt, actions, feedback), and optional reference-solution features (gold patch overlap, when available). Predictions are calibrated via Platt scaling, and a run is halted the moment either classifier's calibrated confidence crosses a fixed threshold (dual-threshold decision rule), otherwise the agent continues. Evaluation uses a leave-one-agent-out protocol across SWE-bench Verified, TerminalBench, and Toolathlon, with over 21,000 collected trajectories from 16, 37, and 22 distinct agents respectively.

Key results:

• On SWE-bench Verified at threshold 0.95: eliminates 26.0% of execution steps, 32.7% of input tokens, and 28.7% of output tokens, at 95.0% accuracy, while shifting resolve rate (∆|Pass@1|) by only 1.1 percentage points.
• Across all three benchmarks: eliminates 13%–26% of agent steps and up to 44.1% input tokens / 29.4% output tokens at 89%–97% prediction accuracy, with resolve-rate deviation of roughly 1–2 points.
• Leaderboard ranking is preserved: Spearman's ρ = 0.991 on SWE-bench Verified (16 agents, only 3 agents shift by one rank), and ρ between 0.959 and 0.994 across all evaluation targets (TerminalBench, Toolathlon).
• Feature ablation (RQ3) shows redundancy: removing the entire Reference-Solution feature family only drops coverage from 34.8% to 32.1% and step savings from 26.0% to 24.7%, explaining robustness on reference-free benchmarks; Behavioral features are the single most important family (removal drops coverage to 23.4%, step savings to 16.4%).
• Architecture ablation (RQ4): LightGBM dominates alternatives — a LoRA-fine-tuned Qwen-0.5B LLM judge reaches only 90.7% accuracy and 17.9% step reduction (vs. LightGBM's 95.0%/26.0%) while requiring a costly model forward pass per step, and linear/MLP baselines perform substantially worse.

Why it matters / caveats: EarlyEval offers a complementary, orthogonal axis to benchmark distillation for cutting agent-evaluation cost, using a lightweight tree ensemble with sub-millisecond CPU inference so it adds negligible overhead per step. Limitations include reduced success-predictor reliability on benchmarks without reference solutions (e.g., TerminalBench, Toolathlon) and degraded performance when an evaluated agent's scaffold (rather than just its base model) is entirely unseen during training.

Problem: In LLM-augmented retrieval, generative models are typically used only for query-side augmentation while final matching still relies on a separate downstream retriever, leaving item-side representations unoptimized and the two sides misaligned.

Method: CoGR trains two separate LLM keyword generators (query-side Gq and item-side Gi, both Qwen3-4B-Instruct or Qwen3-1.7B) that produce compact keyword sets matched directly through a standard inverted index (ranked via BM25). Training has two phases: (1) SFT initializes both generators, with query-side targets built from the top-N most frequent keywords of each query's relevant items; (2) co-evolving RL alternately optimizes each side with GRPO against the opposite side's frozen index — the query side is rewarded directly with retrieval F1, while the item side gets a counterfactual marginal reward measuring the change in query-side F1 from swapping in its keyword set (five alternating rounds, 10 GRPO epochs for query side and 5 for item side per round, max keyword budget 30).

Key results:

• Best overall F1 on both benchmarks: 0.396 on the internal APP Marketplace dataset (39,600 apps) and 0.682 on WANDS (42,994 products), beating the strongest baseline (ANCE-Qwen4B) by 10.9% and 36.1% respectively.
• Ablations show all three design choices matter: removing the marginal item-side reward (Transposed F1), using a shared generator instead of separate ones, or skipping SFT initialization each reduce F1 (e.g., full CoGR 0.3963 vs. No-SFT 0.3751, Shared generator 0.3798, Transposed F1 0.3743 on Internal).
• Evaluation F1 on the Internal dataset rises from ~0.16 before co-evolving RL to ~0.40 after five rounds, with the largest single gain in round one.
• Adding external search-engine results to the query prompt raises F1 from 0.3963 to 0.4379 on Internal; removing item descriptions (title only) drops it to 0.3759.
• Keyword vocabularies become more specific and mutually aligned over training: the share of unigrams drops from 37% to 13% while 3+-word phrases rise from 12% to 31%, and query/item vocabulary sizes converge after 3-4 rounds.

Why it matters / caveats: By having LLMs generate matchable keywords on both the query and item sides and co-training them against each other via RL, CoGR stays compatible with existing inverted-index infrastructure while outperforming sparse, dense, and generative-ID baselines. The ranking stage still relies on plain BM25 over generated keywords, which the authors note as a target for further improvement, and gains partly depend on richer item/query context (descriptions, search results) being available.

Problem: Agent benchmarks typically fix the harness (execution loop, tools, context management, verification) and only measure task-solving performance, leaving unexplored whether LLMs can themselves build and maintain the harness they run inside.

Method: The authors introduce HarnessDev, a two-stage benchmark: in Creation, a creator LLM starts from a deliberately weak, non-acting "seed" harness plus 1–3 development cases and must build a complete, runnable execution system (loop, tools, context, state, lifecycle, verification); in Evolution, the creator iteratively revises its own frozen Creation harness using downstream execution feedback from a fixed SWE-bench Pro/Terminal-Bench feedback set, under a budget of ten full-evaluation candidate pairs. Six creator LLMs (Opus 4.8, GPT-5.5, Gemini 3.1 Pro, DeepSeek V4 Pro, Qwen 3.7 Max, Seed 2.0 Pro) are evaluated using Claude Code (or Codex for GPT-5.5) as the development environment, across four domains (code, data/ML, writing, search) and five downstream benchmarks (SWE-bench Pro, Terminal-Bench 2.1, MLE-bench, EQ-Bench3, BrowseComp), scored under both Self-Eval (creator also executes) and Unified-Eval (fixed Gemini 3.1 Pro executor for all harnesses).

Key results:

• Creation covers 2,207 unique downstream instances; under Self-Eval, Opus 4.8 has the best overall average score (67.8) versus a human-engineered reference average of 86.2, and the seed harness scores 0.0 everywhere.
• Model-built harnesses match or exceed the human reference on writing (Opus 4.8: 84.6 vs. reference 83.7) and ML experimentation (Opus 4.8/Gemini medal rates of 32.9/32.4 vs. reference 24.0), but trail badly on search/BrowseComp (best 52.6 vs. reference 92.2*) and code.
• Execution cost varies enormously and doesn't track quality: MLE-bench token use varies ~19-fold across creators with no reliable score benefit; Gemini's Code harness uses only 1,006 added lines yet gets the best Terminal-Bench score (68.8).
• In Evolution, all five self-runtime lineages improve on visible feedback (e.g., Opus 4.8: 71.1→74.1 pair score) and on the held-out 630-task SWE-Pro split (mean +3.11 points, Opus best at +4.44), but gains are unstable (of 64 official version switches, only 2 show clear improvement beyond a ±4.75-point noise band) and only 2/9 declared final versions are actually held-out-optimal.
• Under a fixed Gemini runtime instead of self-runtime, only the Opus lineage still improves on held-out tasks while three other lineages regress (e.g., GPT-5.5 held-out score falls by 10.32 points), showing gains depend strongly on the executor model.

Why it matters / caveats: The work reframes agent evaluation around building/maintaining reusable infrastructure rather than one-off task outputs, revealing that current LLMs can produce workable harnesses in some domains but leave major gaps (dead code, unused state/memory mechanisms, executor-specific hard-coding) and that self-improvement of a harness is not robust or monotonic. Limitations include uneven/non-optimal human baselines, only one Evolution trajectory per creator-runtime cell (no uncertainty estimates), held-out evaluation limited to SWE-Pro, and a fixed development environment across both stages.

Problem: Existing LLM self-evolution work assumes humans have already operationalized a broad capability request into a fixed task, dataset, and metric, leaving untested whether an agent given only a vague natural-language goal (with no visible task specification or reward) can decide what to learn, not just how.

Method: The authors introduce Aspire, a benchmark and unified interactive environment where an agent is given only a natural-language capability goal (e.g., "improve mathematical reasoning") while a hidden, expert-authored evaluation set of 520 items across six goals (science, humanities, medicine, math, logic/reliability/instruction-following, writing) stays sealed. A single agent-facing tool lets it search/import/synthesize data, launch SFT or GRPO+LoRA training, run its own validation, and branch/stop candidates, while the controller handles scheduling, verification, and safe rollback (retaining a checkpoint only if it beats the base score). Experiments use Claude Opus 4.8, GPT-5.6, and Qwen3.5-4B/9B as decision models and runtime models across three studies: RQ1 (vague goal vs. explicit task on PostTrainBench), RQ2 (self-directed weight evolution from an instruction-tuned checkpoint, final-only and adaptive-feedback protocols), and RQ3 (harness-only evolution with weights frozen, evaluated against a Qwen-Agent reference).

Key results:

• RQ1: Under vague-goal prompting, Claude Opus 4.8 scores 27.07 (vs. 32.90 official Claude Opus 4.8 Max on PostTrainBench) and GPT-5.6 scores 29.58 (vs. 36.23 official); vague-goal runs show +2,109s decision-model thinking, +0.61 GPU-idle hours, and −1.27 active GPU hours per matched pair, with LoRA use jumping from 24.1% to 89.8%.
• RQ2 final-only protocol: across 24 runs (Qwen3.5-4B/9B × 6 goals × 2 runs), only 1/12 model-goal pair means (Qwen3.5-9B on scientific reasoning, 45.33→48.00, +2.67) beat the base score; individual-run rate is 3/24; rollback retains the base model in 21/24 runs.
• RQ2 adaptive-feedback protocol: of 30 configuration-goal cells, 28 produce an evaluated checkpoint and 21 are eligible, but only 1 (GPT-5.6 "Terra" on Qwen3.5-4B mathematics, 17.86→20.10) yields a retained above-base improvement; of 62 checkpoint-to-checkpoint transitions, 28 improve, 13 tie, 21 decline.
• RQ3: all three valid harness successors (created by GPT-5.6 Luna/Terra/Sol) score below the fixed Qwen-Agent reference (28.64 task-macro/27.65 example-micro); the closest, Sol's, reaches 27.22/25.97, about 1.42 points lower.
• Failure mode: 5 final-only checkpoints trained on numeric-label MMLU SFT collapsed to near-zero scores (0, 0, 0, 6.14, 0) because all 21,000 training targets and resulting outputs were single digits, mismatching the actual evaluation format.

Why it matters / caveats: The results suggest current agents can reliably execute the mechanics of a training or harness-editing loop (data → update → verify) but struggle to translate a vague goal into learning signals that generalize to hidden, goal-aligned evaluation — closing the "training loop" is shown to be distinct from closing the "capability loop." Findings are bounded by six goals, largely single-run trajectories per configuration-goal cell, and evaluation-item coverage/quality from the authors' own expert-authored set.

Problem: Long-context transformer decoding must read the entire KV cache at every step even though attention concentrates on a small fraction of tokens, and existing sparse-attention methods still incur O(N) per-step cost to approximate which tokens matter.

Method: The paper introduces Declarative Attention (DA), a zero-shot prompting protocol that has the model itself declare, via chain-of-thought tags (<global>, <focus magic_chunks="K">, <local>), which context regions it needs at each reasoning step; a DA state machine parses these tags during decoding and dynamically builds a block-aligned attention mask (compatible with FlashAttention) that skips unneeded KV cache blocks. Long context is delivered as "magic chunks" (~2048-token segments) framed as a simulated tool-use transcript so segment boundaries align with special tokens models already track. The method is evaluated zero-shot with no fine-tuning on Gemma-4-{31B,12B,E4B} and Qwen-3.6-27B/Qwen-3.5-{9B,4B}, integrated into vLLM, across 15 long-context tasks drawn from RULER, LongBench v1/v2, LooGLE, and ZeroSCROLLS.

Key results:

• DA cuts average attended tokens per response by 52.0% on Gemma-4-31B (13.43M→6.45M) and 31.1% on Qwen-3.6-27B (22.54M→15.52M), with average accuracy drops of only 1.27pp (87.01%→85.74%) and 2.75pp (85.31%→82.56%) across the 15 tasks.
• Ablating the mask alone (DA-no-mask keeps the DA prompt but full causal attention) shows the mask itself is responsible for cutting attended tokens by up to 71.1% (Gemma) and 46.5% (Qwen) relative to the maskless variant, confirming the masking mechanism—not the prompt format—drives savings.
• Accuracy gap to vanilla narrows with scale: relative accuracy rises from 29% (Gemma-4-E4B) to 99% (Gemma-4-31B) and from 64% (Qwen-3.5-4B) to 97% (Qwen-3.6-27B); focus-tag parse success similarly rises from 58% to 99%.
• Absolute token savings grow with context length, up to ~21M tokens saved per response in the longest bin; a roofline wall-clock analysis on a B200 GPU projects decode time reduced to 0.71× (Gemma-4-31B) and 0.77× (Qwen-3.6-27B) of vanilla.

Why it matters / caveats: DA offers a training-free, interpretable alternative to learned/heuristic KV-cache sparsification, since the attention plan is legible text rather than an inferred pattern, and its benefits scale favorably with model capability and context length. Results are a zero-shot lower bound (no post-training on the protocol), the method requires disabling model "thinking" mode, gains vary by task (larger losses on multi-span reasoning), and the smallest models (e.g., Gemma-4-E4B) largely fail to follow the protocol reliably.

Problem: Existing knowledge-based VQA (KB-VQA) retrieval pipelines rely on CLIP-style dual encoders that capture surface-level visual similarity rather than entity-level semantic alignment, causing failures when the same entity appears with large visual variation or different entities look visually alike.

Method: The paper proposes KBMR, described as the first MLLM-based embedding retriever for KB-VQA, which prompts an MLLM (trained variants use Qwen2-VL-7B or LLaVA-OneVision-7B) with "<Image> Summary above image in one word:" and takes the last-token hidden state as the retrieval embedding. An MLLM-based Semantic Discriminator (built on Qwen2.5-VL-7B, compared against InternVL3-8B/14B) scores query-candidate pairs via calibrated Yes/No logit scaling to produce continuous entity-consistency weights, which drive stratified hard-negative sampling (default k=8 negatives, EVA-CLIP-8B used for the initial top-50 candidate pool) and a continuous semantic distillation objective that aligns the retriever's softmax similarity distribution with the discriminator-induced soft prior via symmetric KL divergence. Training uses LoRA (rank 16) with DeepSpeed ZeRO stage-2 on 8×A100 GPUs, 5,000 steps over 600k samples drawn from E-VQA, InfoSeek, and OK-VQA training sets.

Key results:

• Retrieval: on InfoSeek, KBMR surpasses the strongest CLIP retriever (EVA-CLIP-8B) by +14.7 points R@1 and the best MLLM retriever baseline by +19.0; on E-VQA it beats the best MLLM retriever by +12.8 R@1 and EVA-CLIP-8B by +11.4.
• End-to-end VQA: plugging KBMR into existing pipelines (EchoSight, ReflectiVA, MMKB-RAG, OMGM) as a drop-in retriever yields up to +9.4% VQA accuracy gain, with new SOTA of 54.7 on E-VQA and 50.8 on InfoSeek accuracy tables (reported as up to 14.7% retrieval R@1 and 9.4% VQA gains in the abstract).
• On OK-VQA, KBMR improves OMGM's pseudo Recall@5 to 80.4 (vs. 73.4 for OMGM, 70.9 for PreFLMR) and reaches a new SOTA VQA score of 79.3 (+12.7 points over OMGM's 66.6).
• Ablations show SD-based supervision achieves AUC 0.91 vs. 0.79 for CLIP similarity on a 20,000-pair entity-identity discrimination test, and that soft SD weights outperform hard one-hot labels and corrupted (inverted) weights, confirming the soft supervision signal is the source of the gains.

Why it matters / caveats: The results indicate that moving the KB-VQA retriever itself from CLIP-style visual matching to MLLM-based entity-aware semantic embeddings substantially raises retrieval recall and downstream answer accuracy across multiple existing pipelines, suggesting retrieval-augmented VQA systems have been bottlenecked by their first-stage retriever rather than only by reranking or reasoning. The approach adds training/inference cost (MLLM as encoder plus a separate MLLM discriminator for data construction) compared to lightweight CLIP dual encoders.

Problem: Existing pixel-based text encoders (which render text as RGB images and encode it with a single vision encoder) are held back by fixed-resolution pretraining, visual shortcut learning, weak grounding to real-world visual concepts, and poor multilingual visual text understanding, and it is unclear which design choices actually fix these issues.

Method: Through controlled ablations on a 13M-example dataset, the authors identify four design principles: (1) variable natural-image resolutions and rendered font sizes act as "spatial proxies" letting small-canvas pretraining generalize to high-resolution documents; (2) natural image-text pairs (from LAION) are required alongside rendered text-text pairs to prevent representation collapse; (3) layout-aware rendering (393 fonts, 5,000+ DTD textures, blur/rotation/cropping augmentations) prevents pixel-level shortcut learning; (4) a two-stage multilingual curriculum (unsupervised multilingual Wikipedia pretraining, then curated semantic pair mid-training) is needed for cross-lingual alignment. These principles are scaled into PixelLinguist II, a NaViT-based (initialized from Qwen2.5-VL's ViT) native-resolution vision encoder trained with unified InfoNCE contrastive loss over 280M examples (62M multilingual Wikipedia docs + 26M curated text pairs + 26M LAION image-text pairs, 2 epochs each stage, batch size 32,768 across 64 GPUs).

Key results:

• Removing resolution variance in ablations drops ViDoRe nDCG@5 from 37.83 to 33.45; also fixing font size drops it further to 30.97; removing natural images entirely (rendered-text-only, fixed font/plain canvas) collapses performance to 1.65.
• On English Visual STS, PixelLinguist II (mid-training only) beats the strongest baselines despite ~1/7 the parameters and ~1/87 the training examples seen; with AllNLI fine-tuning it reaches 79.80 average Spearman correlation (vs. 71.99 for EVA02-CLIP-bigE-14-plus).
• On ViDoRe (VDR), PixelLinguist II reaches 50.94 average nDCG@5 vs. 46.94 for the strongest SigLIP baseline, with a 16.6-point gain on ShiftProject.
• Outperforms the strongest SigLIP variant by ~15% (cross-lingual) and ~16% (multilingual) in Spearman correlation on Visual STS; the two-stage curriculum adds ~3.3–3.5 points over curated-pairs-only training.
• Achieves a 2.75% average relative improvement over Qwen2.5-ViT as an MLLM vision encoder; remains on par with CLIP at 60% visual token compression and still beats uncompressed CLIP on ViDoRe at 80% compression.

Why it matters / caveats: The findings show pixel-text encoders don't just need more data/scale but specific inductive biases (spatial proxies, real-image grounding, layout diversity, staged multilingual training) to avoid shortcut learning and collapse, and the resulting model's robustness to heavy token compression suggests promise for optical context compression in MLLMs. The authors note PixelLinguist II is trained at smaller scale than billion-pair CLIP/SigLIP baselines, underperforms on diagram-heavy scientific subsets like ArxivQA, and its rendered-text training data may not capture real-world noise patterns such as scans, blur, occlusion, or handwriting.

Problem: Existing agent benchmarks evaluate LLMs as fixed policies, leaving it unclear whether an agent can actively test its own behavior, judge the resulting experience, and use that experience to reliably improve future decisions.

Method: The paper introduces S3Gym (Self-Testing, Self-Judging, Self-Improvement Gym), a benchmark built on seven text-based games with executable verifiers (Chess, Minesweeper, Nullify, Tetris, Snake, Plants-vs-Zombies, Trust Evolution), each with a permissive exploration configuration and a stricter held-out evaluation configuration. Each self-improvement cycle runs Explore → Judge → Consolidate → Update → Evaluate, where the agent jointly produces an action and a self-judged score without access to verifier-computed rewards during exploration. Three experience-incorporation pathways are compared under identical seeds/budgets: History ICL (raw score-annotated trajectories appended to context), Summary Memory (model-generated compressed rules/mistakes/directions), and parameter Training (supervised fine-tuning on self-judged trajectories, tested on Qwen3-8B over 20 checkpoints). Seven proprietary LLMs (GPT-4o, GPT-4.1, o3-mini, Gemini-2.5-Flash, Gemini-2.5-Pro, GPT-5.5, Gemini-3.5-Flash) are evaluated with metrics Avg., Max., and AUC+ (positive area above the initial-score baseline).

Key results:

• Summary Memory is not uniformly better than History ICL: Gemini-2.5-Flash's Minesweeper AUC+ rises from 0.000 (ICL) to 7.794 (Summary) and PvZ AUC+ rises from 24.402 to 238.501, while GPT-5.5's PvZ AUC+ falls from 548.499 to 33.219 and Gemini-3.5-Flash's Chess AUC+ falls from 12.280 to 0.000.
• Self-Judging reliability (98 runs, 116,117 transitions) varies widely by game: event agreement with verifier rewards ranges from 0.496 (Chess, near-random) to 0.881 (PvZ), and PvZ has the largest calibration error (NMAE 0.882) despite high agreement.
• Judgment quality barely predicts subsequent improvement: correlation between event agreement and next-checkpoint score gain is ρ(A,g) = -0.010 across the benchmark, and NABA correlates only weakly with agreement (Pearson r = -0.23, Spearman ρ = -0.11).
• Parameter training (Qwen3-8B) is highly task-dependent: Trust Evolution score rises from 0 to a max of 30 (post-training avg 8.684, AUC+ 163.5), while Minesweeper/Nullify/Tetris show no improvement, and Plants-vs-Zombies degrades sharply (initial score 23 vs. 6 at every updated checkpoint), indicating severe negative transfer.
• Summary Memory beats History ICL on average NABA in Nullify, Tetris, and Trust Evolution (mean ∆NABA up to +8.43), but underperforms in Minesweeper, PvZ, and Snake (mean ∆NABA as low as -2.11), showing summaries help when experience compresses into stable rules but hurt when success needs precise state-contingent detail.

Why it matters / caveats: The results show that recognizing successful actions (accurate Self-Judging) is not sufficient for Self-Improvement—agents must also convert feedback into transferable, executable policies, and the best pathway (raw history, summary, or training) depends strongly on task structure; parameter training in particular can produce unstable gains and severe negative transfer, motivating future work on judgment calibration, memory selection, and trajectory filtering rather than simply more interaction data.

Problem: Existing 3D tokenizers that compress object geometry into a fixed set of global tokens suffer sharp reconstruction quality degradation when the token budget is reduced to only a few tokens, since they optimize the complete latent set without explicitly determining which information should be preserved first.

Method: ZipTok3D is a 3D tokenizer that uses nested dropout during training to randomly truncate the encoded latent sequence (sampled from exponentially spaced budgets K = {1,2,4,8,16,32,64,128} out of a max length M=128, width d=512) and requires each retained prefix to reconstruct the full object, forcing leading tokens to carry object-wide geometry. The decoder replaces single-pass triplane decoding with a parameter-shared six-layer Transformer block applied recurrently for L refinement steps (5 in main results) to progressively unfold the compact prefix into a 128×128 triplane, trained with a reconstruction BCE loss plus an intra-loop self-distillation loss on a randomly sampled intermediate step (following COD-VAE's auxiliary losses). Trained on ShapeNet-v2 and TRELLIS-500K with AdamW (lr 1e-4, batch size 672, FP16) on four A800 GPUs; a causal stage-2 VAE maps the K×512 prefix to a K×32 latent for class-conditioned generation via an EDM diffusion model.

Key results:

• On ShapeNet, ZipTok3D with 1 token matches COD-VAE-32's rounded CD (0.012) and F1 (97.8) and is within 0.3 IoU points (96.8 vs 97.1), using a 32× shorter sequence; COD-VAE-2 degrades sharply to IoU 77.7.
• On TRELLIS, ZipTok3D with 4 tokens (IoU 75.31, CD 0.0166, F1 95.92) matches/slightly exceeds COD-VAE-32 (IoU 75.25, CD 0.0172, F1 95.67) with an 8× shorter sequence (bootstrap CI confirms CD and F1 gains exclude zero).
• For class-conditioned generation on 5 ShapeNet categories, ZipTok3D-2 (16× shorter stage-2 sequence than COD-VAE-32) achieves comparable distribution metrics (MMD 5.142 vs 5.020, COV 84.91 vs 84.96, 1-NNA 54.21 vs 53.11) while providing higher sampling throughput (50.62 vs 46.02 samples/s) and similar full throughput/memory.
• Ablation at K=2 on ShapeNet shows nested-prefix training alone raises IoU from 77.7 (COD-VAE) to 92.3 (Prefix only, same 12-layer decoder); adding iterative refinement (6×5 vs 6×1) raises IoU from 91.9 to 96.6, and intermediate supervision adds a further gain to 96.9, at a decoder cost of only 23.5M parameters vs COD-VAE's 39.3M.

Why it matters / caveats: The method decouples "what to encode first" from "how much computation to decode it," enabling far more compact yet faithful 3D representations for downstream generative pipelines, with lower decoder parameter count than the fixed-depth COD-VAE baseline. Iterative refinement increases decoding latency (2.65ms vs 1.47ms per shape at K=2 in the ablation), and the authors note future work is needed on generative models that better exploit short sequences and on adaptively allocating prefix length/refinement depth per input rather than using a fixed schedule.

Problem: RLVR's coarse outcome-level rewards give no credit for how far a reasoning trajectory got before failing, so a near-correct solution is penalized the same as a completely wrong one, and existing fixes (process reward models, on-policy distillation) require extra reward-model training or teacher-student similarity assumptions.

Method: Cliff extends GRPO by using an off-the-shelf teacher LLM (e.g., a SOTA model, Qwen3-32B, or Gemma3-27B) to locate the "Pitfall Step"—the first sentence where a student rollout's reasoning goes wrong—splitting each incorrect rollout into a valid prefix and an erroneous suffix. It converts this into token-level advantages: the correct-prefix tokens get advantage λ·Acor − b (with λ=0 used by default to avoid length hacking) and suffix tokens get the standard GRPO incorrect-rollout advantage Ainc − b, with an offset b recentering advantages to zero mean; reference solutions are filtered by an automatic verifier so Cliff only applies when the teacher itself solves the problem correctly (otherwise falling back to vanilla GRPO). Trained on DAPO-math-17k and Deepcoder with student models Qwen3-4B-Base and Phi-4-mini-Instruct.

Key results:

• Across 12 settings (math + coding, 2 students × teachers), Cliff outperforms on-policy distillation by 15% and vanilla GRPO by 7% on average.
• On Qwen3-4B with a SOTA teacher, math average accuracy rises from 61.68 (GRPO) to 65.66 (Cliff), including AIME 32.01→36.98 and DAPO-math 42.90→49.30.
• Teacher judge accuracy on a 100-example human-annotated set exceeds 90% (with correct reference solutions), and the Pitfall Step position matches human annotation within 1 sentence for 68-82% of cases depending on teacher.
• An ablation ("GRPO with teacher," same advantage over full rollout) gives only marginal gains over vanilla GRPO, confirming the gain comes from Cliff's prefix/suffix credit assignment rather than merely adding a teacher signal.
• λ=1.0 causes length hacking (response length grows to ~1959 tokens vs. 1506 at λ=0) and hurts accuracy (avg. 63.98 vs. 65.66), motivating the λ=0 default.

Why it matters / caveats: Cliff is a simple, model- and task-agnostic reward-shaping method that needs only an LLM judge (no trained reward model, no teacher-student architecture match) and works even with modest-capability teachers, though it depends on the teacher's ability to both solve problems (for reference-solution filtering) and accurately localize errors, and the authors note future work is needed to extend it to agentic settings and rule-based pitfall detectors.

Problem: Sampled-token on-policy distillation (OPD) is computationally cheap but suffers "diversity distillation failure," where student pass@1 improves while pass@k plateaus because the student fails to inherit the teacher's output diversity.

Method: The authors derive First-Order Local Entropy Influence IH(y) = Ay·Dy, a signed first-order proxy that decomposes each sampled-token update's entropy effect into the teacher-student log-probability gap (advantage Ay) and the student's local probability structure (Dy), requiring only the teacher's sampled-token log-probability (no full-vocabulary logits). Based on the finding that entropy contraction concentrates at low-divergence positions, they propose Influence-Directed Adaptive On-Policy Distillation (IDA-OPD), which keeps entropy-expanding updates (IH(y) ≥ 0) unchanged and attenuates entropy-contracting updates (IH(y) < 0) using a divergence-adaptive shrinkage weight wy = |qy − py| / (qy + py) applied to the advantage. Experiments distill Qwen3-8B-Non-Thinking-RL-Math → Qwen3-8B-Non-Thinking and Qwen3-4B-Non-Thinking-RL-Math → Qwen3-4B-Non-Thinking on DeepMath103K (math) and Qwen3-4B-Non-Thinking-RLCode → Qwen3-4B-Non-Thinking on a code subset, evaluated with an unbiased pass@k estimator (n=128 samples) against baselines OPD, REOPOLD, entropy bonus, advantage shaping, AOPD, and EOPD.

Key results:

• On 4B math distillation, IDA-OPD improves pass@16 over vanilla OPD by +8.3 on HMMT Feb, +4.6 on AIME 2024, and +4.3 on AIME 2025; on 8B, gains are +7.2, +4.2, and +6.0 respectively.
• IDA-OPD matches or surpasses the teacher's pass@16 on several benchmarks, e.g., 83.3% vs. teacher's 80.0% on 4B AIME 2024, and 76.7% vs. teacher's 73.3% on 8B AIME 2025.
• On code tasks (Qwen3-4B), IDA-OPD improves over OPD by +2.1/+0.9 pass@1/pass@16 on MBPP+ and +1.3/+1.0 on LiveCodeBench, while vanilla OPD's pass@16 stagnates (72.9 vs. student's 73.5 on MBPP+).
• Ablations show removing the IH(y) gate drops AIME24 pass@1 from 56.7 to 53.8; hard-masking entropy-contracting updates instead of shrinking drops pass@1 to 52.1; using sign(Ay) as gate instead of IH(y) worsens pass@16 to 76.8.
• Token-level entropy analysis on AIME 2024 shows OPD piles ~74% of mass in the lowest-entropy bin (0.0–0.1) vs. teacher's ~57%, while IDA-OPD stays close to the teacher at ~55%.

Why it matters / caveats: IDA-OPD achieves diversity preservation comparable to expensive teacher-informed methods (which require top-K teacher distributions, O(L_sel·K) cost) while retaining sampled-token OPD's cheap O(L) per-position cost, using only the teacher's sampled-token log-probability and reusing student logits already computed in the forward pass. Results are demonstrated on Qwen3 models (4B/8B) for math and code domains only.

Problem: Vision-language models produce fluent but visually incomplete captions (missing attributes, counts, textures, materials, spatial relations), and existing fixes for this "fine-grained captioning" gap rely on costly multi-stage generate/verify/rewrite pipelines that add substantial inference latency.

Method: The authors propose SimLoss, a reference-free embedding-space objective that trains a VLM to align its projected, mean-pooled hidden-state representation with a frozen image embedding (Qwen3-VL-Embed) via an InfoNCE contrastive loss over in-batch positives/negatives, requiring no fine-grained caption targets. Two instantiations are trained with LoRA on Qwen2.5-VL-7B using only MS COCO images (captions discarded): SimLoss FFT backpropagates directly through the locally available embedding model and projector, while SimLoss GRPO uses a black-box embedding model (Gemini 2 Embed) as a reward signal optimized with GRPO. Both are evaluated single-pass at inference (encoder/projector removed) against baselines including the 5-stage CapMAS verification pipeline, FeedQuill (PPO with a composite F1/CLIP/CIDEr reward), and PAPO (perception-aware GRPO with random/YOLO-based masking) on the IIW-400 benchmark.

Key results:

• On IIW-400, SimLoss FFT reaches F1 0.7023 versus CapMAS's 0.7025 (a 0.0002 gap) while having the highest precision of all methods, 0.8485 vs. CapMAS's 0.8467 and the plain Qwen2.5-VL-7B baseline's 0.7884.
• SimLoss FFT cuts mean caption length from 347.55 words (plain baseline) to 114.86 words while recall stays essentially flat (~0.60) across methods.
• SimLoss FFT runs at 5.77 sec/image vs. CapMAS's 115.31 sec/image on an A100, a 20.0× speedup, while SimLoss GRPO achieves the highest recall (0.6015) and CLAIR score (0.858) but lower precision (0.8227) than FFT.

Why it matters / caveats: SimLoss shows that embedding-space supervision before decoding can recover most of the quality gains of expensive multi-stage verification pipelines in a single forward pass, making fine-grained captioning far more practical at scale. A caveat: CapMAS still edges out SimLoss FFT in F1, both SimLoss variants can amplify fine-category errors (e.g., misidentifying an abstract driftwood sculpture), and the approach's quality depends on the biases/granularity of the frozen embedding teacher used for supervision.

Problem: Existing end-to-end LLM-based speaker-attributed ASR models (e.g. VibeVoice-ASR) only support offline recognition, reading the whole recording before emitting output, which fails the low-latency needs of real-time voice assistants that must know "who said what" as speech arrives.

Method: VibeVoice-ASR-Streaming extends VibeVoice-ASR by interleaving fixed-size audio chunks (15 or 22 latent frames, i.e. 2.0s or 2.9s, at 7.5 latent frames/sec) plus a fixed 4-frame (0.5s) lookahead with previously generated speaker-attributed text in a single autoregressive sequence, using VibeVoice's frozen dual Acoustic/Semantic tokenizer encoders feeding a trainable Qwen2.5 LLM backbone (1.5B or 7B). Training proceeds in three stages: Stage 1 offline speaker-attributed pretraining, Stage 2 streaming pre-training on ~420,000 hours of English/Chinese speech with the interleaved chunk format, and Stage 3 streaming fine-tuning on ~13,000 hours of curated data (including 50,884 synthetic multi-speaker recordings totaling 4,519.6 hours with room-impulse-response augmentation). Word-level alignment for chunk-target construction uses Qwen3-ForcedAligner-0.6B.

Key results:

• On recognition-only streaming ASR (5-set mean, WER/CER), the 7B model scores 24.66, beating Gemini 3.5 Transcribe Live (25.23), GPT Realtime Whisper (39.31), GPT Live Transcribe (40.55), and ElevenLabs Scribe v2 Realtime (41.39).
• For speaker-attributed ASR across 13 settings (4 meeting benchmarks + 9 MLC-Challenge languages), it achieves best or tied-best cpWER/cpCER on 12/13, improving over Azure ConversationTranscriber by 2.39–12.45 points on meeting benchmarks and lowering the MLC-Challenge average cpWER from 27.06 to 22.75.
• Expected speaker-attribution latency is 2.00s (22-frame config) vs. measured wall-clock 8.21s for Azure CT and 9.12s (initial)/51.06s (final, revised) for Google STT.
• Streaming conversion costs 0.75–3.53 WER/CER points and 5.13–6.67 cpWER/cpCER points versus the non-streaming VibeVoice-ASR checkpoint it's initialized from.
• Real-time factor stays ≤0.104 (146–208ms decode per 2000ms chunk on one A100 80GB PCIe with vLLM, bfloat16); on single-speaker benchmarks (AISHELL-1, LibriSpeech, GigaSpeech) it is not best-in-class but its 4-set mean (5.76, 22-frame) is level with the strongest specialized system.

Why it matters / caveats: This is presented as one of the first end-to-end LLM-based streaming systems to unify ASR and speaker diarization into a single generative pass rather than a cascade, cutting speaker-attribution latency from tens of seconds to ~2 seconds while matching or beating deployed cloud APIs. Limitations: released checkpoints support only up to 10 languages and recordings up to 8 minutes (compute-bound), degrade under long overlapping speech since output must be serialized, and first-packet latency (3.5s/2.5s) is higher than steady-state latency.

Problem: Existing information retrieval benchmarks mostly cover closed-ended queries or open-ended queries whose supporting evidence spans only a single subject domain and modality, leaving it unclear whether retrievers can comprehensively cover the diverse implicit perspectives, domains, and modalities that real open-ended questions require.

Method: The authors build MULTI3IR, a benchmark of 104.9K open-ended queries from 77 Stack Exchange sites, constructed via an automated pipeline (GPT-5-mini perspective extraction, uniqueness filtering with all-MiniLM-L6-v2, faithfulness verification with Bespoke-MiniCheck-7B, document retrieval from Google Image Search/C4 via Qwen3-Embedding-4B, and exclusive-support verification with Qwen3-VL-30B-A3B-Instruct), plus human-verified test data. They also propose SPIN (Steering Perspectives by Injecting Noise), which injects a small set of learnable noise vectors into an intermediate layer of a frozen multimodal retriever (e.g., MM-Embed-8B, GME-Qwen2-VL-7B-Instruct, Qwen3-VL-Embedding-8B) to produce multiple perspective-aware query embeddings, trained with a noisy-OR-based contrastive loss using only perspective descriptions (no document-level relevance labels).

Key results:

• MULTI3IR contains 104.9K queries, 521.7K perspectives, and 1.01M multimodal documents, averaging 3.34 domains and 1.91 modalities per query (vs. e.g. 1.30 domains/1.00 modality for BeRDS).
• Zero-shot naive retrievers reach only up to 28.83 HC@10 / 39.52 SC@10, while oracle perspective descriptions boost HC@10 by 15.58–35.31 points and SC@10 by 31.28–36.72 points, revealing strong single-perspective bias.
• SPIN outperforms LLM-Expansion by 8.18–13.13 points on HC@10 and 17.99–21.92 points on SC@10, and outperforms document-supervised ARE by 3.62–4.47 points on HC@10 and 14.47–14.62 points on SC@10, despite using only perspective-level (not document-level) supervision.
• SPIN generalizes zero-shot to unseen benchmarks PIR (+9.37 to +11.73 HC@10) and BeRDS (up to +20.10 HC@10 for GME-Qwen2).
• With mid-layer injection, SPIN achieves 63.4 HC@100 using only 20.5K parameters, beating LoRA (r=16, 104.9M params, 55.9 HC@100) and adapters (41.9M params, 55.9 HC@100) by 7.5 points while using over three orders of magnitude fewer parameters.

Why it matters / caveats: The benchmark exposes that current strong multimodal retrievers collapse onto a single dominant perspective for open-ended queries, and SPIN offers a lightweight, label-efficient fix that needs no document-level annotations or query-time LLM calls. The authors note SPIN uses a fixed number of perspective vectors regardless of true perspective count, does not jointly leverage supporting documents during training, and the benchmark's automatic construction pipeline may carry residual LLM-induced biases.

Problem: Historical newspaper scans are a huge, largely untapped data source for LLM training and digital-humanities research, but their dense, irregular layouts make extracting accurate, structured, machine-usable text at scale both difficult and expensive.

Method: The authors built a modular 15-step pipeline, developed jointly with Boston Public Library, that runs on workstation-grade GPU hardware: a YOLO26x (55.7M params) object detector segments each scan into "crops," each crop is OCR'd with both Tesseract 5 (tessdata_best) and the 3B-parameter VLM dots.mocr (served via vLLM), and further steps add language detection (Lingua), crop-type classification (a YOLO26m-cls image classifier plus a Model2Vec/potion-base-32M text classifier, trained on data auto-labeled by Qwen3-VL-30B-A3B-Thinking), HDBSCAN-based reading-order detection, NER (Flair ner-fast), zero-shot subject classification (ModernBERT-large-zeroshot-v2.0), and DINOv2-small/potion-multilingual-128M embeddings.

Key results:

• Processed 1,473,635 public-domain newspaper scans (1795–1930) into 83,147,041 crops; dots.mocr yielded 16.3B o200k_base tokens vs. 14.7B for Tesseract.
• Segmentation model: precision 0.927, recall 0.910, F1 0.918, mAP@50 0.955 on a 153-scan held-out test set.
• Crop-type classifiers reach 91.3% (image) and 92% (text) top-1 accuracy, agreeing on 90.55% of crops; image classifier much stronger on visual categories (e.g., "Photograph or illustration" F1 0.84 vs. 0.48 for text).
• Reading-order detection achieves 72.1% macro / 80.8% micro position accuracy and Kendall's τ of 0.922/0.959 against 723 manually annotated scans.
• NER extracted 155.6M location, 142.2M person, and 49.1M organization mentions; estimated pipeline cost on rented 8xL40S hardware is ~$25,000 (~1650 GPU-hours), versus an estimated $250K–$800K to generate the same 16B tokens with frontier proprietary LLMs.

Why it matters / caveats: The work shows a reproducible, cost-efficient (small-model) route to extracting billions of high-quality tokens plus rich metadata from historical newspaper archives for both AI pretraining and humanities research, and the authors release the pipeline, models, and dataset. Several components (NER, subject detection, thesauri term matching) are explicitly labeled experimental, reading-order detection assumes strictly column-based layouts, and the models were trained/tested on one library's collection so generalization to other newspaper collections is untested.

Problem: It is unclear which components (data, SFT, RL, model scale, or test-time compute) are actually responsible for LLMs reaching gold-medal performance in top-tier coding competitions like IOI and ICPC, since prior gold-medal systems are closed or entangle these factors.

Method: The authors curate 22,000 competitive-programming problems and use DeepSeek-V4-Flash to generate 1.2M synthetic reasoning traces (for the small model) and 477,642 traces (for the large model), then apply supervised fine-tuning (SFT) and GRPO-based reinforcement learning (RL) with executable/compilation rewards to Nemotron-3-Nano-30B-A3B (producing Nemotron-3-Nano-CC, SFT+RL) and Nemotron-3-Ultra-550B-A55B (producing Nemotron-3-Ultra-CC, SFT only). They also introduce GenCorrect, a test-time compute strategy that generates up to 200 candidate solutions per round, selects a diverse subset via token-shingle diversity clustering, submits 10 for evaluator feedback, and iteratively refines over up to five rounds within a 50-submission budget.

Key results:

• On IOI 2025, Nemotron-3-Nano-CC improves Score@1 from 130 to 291 points after SFT+RL, then to 468 with GenCorrect (5 rounds), exceeding the gold threshold of 438.3; Nemotron-3-Ultra-CC reaches 502.
• SFT alone drives most gains: Nano-CC's IOI Score@1 rises from 21.7% to 47.3% over 3 SFT epochs; RL adds only 46.7%→48.5%; Nano-CC also improves ICPC 2025 Pass@1 from 16.9% to 51.0% and LiveCodeBench Pro Pass@1 from 17.6% to 71.6%.
• Live prospective evaluation on IOI 2026 (under official time, submission, and no-internet constraints): the competition-tuned Ultra-CC system scored 535.4/600, beating the gold threshold (361.12) and the top human contestant's score (498.27) — described as the first AI system to outscore the highest-scoring human on an IOI problem set.
• NVFP4 quantization with MTP=5 nearly doubles inference throughput (345.9→736.8 tokens/s/GPU) while costing only ~0.6-6.6 points of Score@1, enabling the large GenCorrect candidate batches within the competition time window.

Why it matters / caveats: The work isolates and quantifies the contribution of each pipeline stage (SFT > RL > test-time refinement) toward gold-medal coding performance and demonstrates, under matched human contest conditions, an AI system outscoring the best human IOI contestant. The authors caution this is a system-level (not equal-resource) comparison given substantial training/inference compute, RL was not applied to the larger Ultra model due to cost, and findings may not generalize beyond competitive programming.

Problem: Dynamic sparse-attention prefilling methods like FlexPrefill route attention heads using an indirect, costly proxy (Jensen-Shannon Divergence) and select tokens via cumulative coverage thresholds that ignore the post-softmax "mass cliff," causing them to either terminate selection inside attention sinks or accumulate large amounts of background-noise tokens at long context lengths.

Method: CRISP replaces JSD-based routing with C_struct, a structural-mass proxy that measures proxy-attention mass at Vertical-Slash-compatible anchor positions (architectural sinks and local recency window) using an already-computed attention map slice, avoiding the extra pooling matmul and KL-divergence computation. For Vertical-Slash heads it replaces cumulative γ-thresholding with a sink-aware, noise-floor threshold (parameter α, default 1.0) that selects blocks whose mass exceeds the expected background mass, while Pooled-Estimation heads keep γ-cumsum selection since they lack a sharp mass hierarchy. It is evaluated training-free on Meta-Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct against FlashAttention, MInference, and FlexPrefill (γ=0.95/0.97) on InfiniteBench, RULER, and LongBench.

Key results:

• C_struct reproduces FlexPrefill's JSD routing decisions on 94.0% (Llama) and 88.1% (Qwen) of measured heads, with top-1 attention mass falling at sink/recency positions for 91.3% and 98.3% of heads respectively.
• On InfiniteBench (131K), CRISP α=1.0 matches or exceeds exact dense attention (Llama: 48.7 vs 48.6; Qwen: 28.7 vs 24.0), while MInference scores only 36.5/25.1.
• Retrieval-task accuracy recovery over FlexPrefill γ=0.95: +17.8pp (InfiniteBench KV Retrieval, Llama), +28.0pp (InfiniteBench Passkey, Qwen), +12.5–13.5pp (LongBench Passage Retrieval En).
• Attention-only latency speedup over FlashAttention reaches up to 5.40× (α=1.5) and 5.30× (α=1.25) at 512k tokens, versus 3.90–4.41× for FlexPrefill γ=0.95/0.97; CRISP α=1.0 is 25% faster than FlexPrefill γ=0.97 at 512k.
• Pushing FlexPrefill to higher coverage (γ=0.97) yields minimal/inconsistent accuracy gains but incurs large latency penalties, empirically confirming the mass-cliff limitation.

Why it matters / caveats: CRISP gives a theoretically grounded, calibration-free (α=1.0) way to navigate the sparse-attention "mass cliff," delivering both accuracy and O(n)-noise-elimination speed gains without training. Limitations acknowledged by the authors: C_struct relies on sink-having architectures (breaks for sink-suppressing designs like gated attention), the binary VS/PE routing doesn't handle mixed-structure heads well, PE-path improvements are left for future work, and evaluation is limited to 7–8B-scale models and prefilling only (not decoding).

Problem: Paper-to-code agents pass implementation knowledge between stages as free-form plans or summaries that downstream coding steps can ignore, reinterpret, or compress, causing algorithmic simplification and cross-file inconsistency in generated repositories.

Method: PaperCompiler reframes repository generation as specification compilation in three phases: (1) Paper Grounding builds a compact implementation blueprint B=(M,Z) plus a reference registry Q that preserves long/format-sensitive evidence (prompt templates, schemas, algorithms) verbatim, tagging each item as paper-supported, inferred, externally delegated, or unresolved; (2) Specification Compilation reconciles this evidence into non-degradation requirements K (each with source, required behavior, boundaries, forbidden substitutions), builds a repository ownership graph G assigning files/interfaces/artifact producers-consumers, and slices this into file-level contracts Si (interfaces, implementation recipe, artifacts, cross-file handoffs, constraints); (3) Constraint-Guided Repository Generation generates files in topological dependency order using Si, previously committed code, and downstream specs as compatibility constraints. Papers are parsed to Markdown via MinerU; generation and evaluation backbone is o3-mini/o3-mini-high, matching the PaperCoder baseline setup.

Key results:

• On Paper2CodeBench (90 papers, ICLR/ICML/NeurIPS 2024), PaperCompiler improves over PaperCoder: reference-free 4.562→4.777 (+4.7%), P2C-Ex 4.535→4.728 (+4.3%), reference-based fidelity 3.647→4.152 (+13.8%).
• Per-paper win ratios vs. PaperCoder favor PaperCompiler in all three subsets/protocols, with the largest margin under reference-based evaluation (e.g., 92.6% vs 7.4% on ICML 2024).
• On a 10-paper subset also including AutoP2C and AutoReproduce, PaperCompiler scores highest on all protocols (ref-free 4.813, P2C-Ex 4.850, ref-based 4.263) using 1.71M tokens/repo vs. PaperCoder's 0.98M and AutoP2C's comparable 1.68M (yet much lower AutoP2C scores), at an estimated $1.88–$7.51 per repository.
• Reference-based evaluator critique analysis (~2.6K items each) shows high-severity critiques drop from 13.2% to 6.1%, high-or-medium severity from 54.2% to 37.9%, missing core components from 12.3% to 6.8%, and evaluation mismatch from 13.4% to 8.4%; API/schema mismatches rise modestly from 2.3% to 4.0%.
• Ablation on 9 sampled papers shows reference-based fidelity drops most when removing Reconciliation (4.38→3.86) and Contracting (4.38→3.92), with a smaller drop for Context Slicing (4.38→4.11).

Why it matters / caveats: Explicit, source-linked, ownership-tagged specifications reduce the algorithmic degradation and cross-file inconsistency common in paper-to-code pipelines, particularly improving fidelity against author reference implementations rather than just surface completeness. The approach costs roughly 1.7x more tokens than the PaperCoder baseline, and API/schema mismatches actually increase, indicating a remaining interface-alignment weakness the authors flag as future work.

Problem: No existing code-retrieval benchmark plants execution-verified, single-edit buggy variants of each query's canonical implementation directly in the search pool, so it is unknown whether embedding retrievers can rank a functionally correct implementation above near-identical but incorrect ("near-clone") code.

Method: The authors build ExecRetrieval, 939 Python tasks across ten algorithmic domains, each with one execution-verified canonical implementation, up to four mechanically mutated buggy distractors (single targeted edits: off_by_one, wrong_operator, swap_arguments, remove_edge_case_check, wrong_comparison, off_by_one_boundary), and a 7–10-assert test suite, for a 4,694-snippet corpus. Data is generated via a two-phase registry-driven pipeline (Claude Sonnet 4.6 proposes tasks; GPT-5.4 with high reasoning effort writes canonical/distractor code and tests) with a five-stage validation gate and isolated-subprocess execution oracle. They evaluate 23 dense embedding configurations (Gemini Embedding, Mistral Embed, OpenAI, Qwen3, BGE, BGE-M3, E5, GTE, Sentence-Transformers) plus BM25 under each provider's documented best-fair-shot invocation, reporting exec@k, execution_precision@k, and canonical-ID nDCG with paired McNemar tests and query-level bootstrap intervals.

Key results:

• Best system (Gemini Embedding 2) reaches exec@10=1.00 across all 939 queries but only exec@1=0.331 (95% CI [0.299, 0.362]); several hosted models also saturate exec@10 near 1.00.
• When rank-1 is wrong, the top result is a paired buggy variant 91.5–99.4% of the time across the four leading systems (e.g., 624/628 misses for Gemini Embedding 2 = 99.4%).
• The canonical scores below at least one of its four paired distractors in 66.8% of queries for Gemini Embedding 2 and 78.4% for Qwen3-Embedding-8B.
• Pool-density ablation: adding just one near-clone distractor drops the strongest system's exec@1 from 0.993 (zero clones) to 0.678, a 27–49% relative drop across all 23 configurations.
• Deception rate (distractor cosine ≥ canonical cosine) averages 44.3% across 86,135 (query, distractor, model) triples, ranging 39.3% (remove_edge_case_check) to 48.0% (wrong_comparison) by mutation type; BM25 lags dense models at exec@1=0.058 and exec@10=0.422.

Why it matters / caveats: The results show current embedding models are strong at coarse topical recall but structurally unable to discriminate correct from near-clone-but-broken code at rank 1, meaning downstream rerankers/execution/agentic verification must absorb correctness-discrimination that embeddings alone cannot provide. Limitations: benchmark is Python-only, uses LLM-generated (not human-written) mechanical mutations, is a closed-world corpus of exactly 4,694 snippets, and every reported gap is conditional on a near-clone actually being present in the pool (how often that occurs in real deployed corpora is unmeasured).

Problem: When an LLM orchestrator must collaborate with a non-language subagent (e.g., a chess engine, robotics policy) whose expertise lives in continuous latent representations, forcing that expertise through a text ("verbalized") interface may discard information the LLM needs to reason effectively.

Method: The paper introduces "latent state internalization," where a frozen subagent's penultimate-layer activations are projected via a 3-layer MLP ("LatentBridge") into k=32 continuous tokens injected directly into the LLM's token stream, interleaved with language and action tokens, and dynamically re-encoded as the environment state changes. The resulting model, LLAMIA (built on Qwen3, 4B/8B/14B), uses the Lc0-BT4 chess engine (240M-parameter, 15-layer transformer) as the subagent and is trained in two stages: Stage 1 supervised alignment of LatentBridge on state–policy pairs (LLM frozen), then Stage 2 joint fine-tuning of LLM and LatentBridge via DAPO reinforcement learning. The authors also build LLAMIA-Bench, six chess tasks (behavior cloning, puzzle difficulty/interest, rationale generation, game commentary) with a new Agadmator-2K commentary dataset, and compare against a matched verbalized ablation (LLAMIA-Verb), untrained tool-use (Qwen3+Lc0), GPT-5.1+Lc0, and task-specific finetuned experts.

Key results:

• LLAMIA-14B achieves the top score on all six LLAMIA-Bench tasks, beating GPT-5.1+Lc0 and matching/exceeding task-specific specialists; LLAMIA-8B already leads on all six tasks and LLAMIA-4B on four of six.
• On Puzzle Interest (a signal with no text proxy), every verbalized system scores ≤12 while LLAMIA-14B reaches 52.
• The "verbalization debt" (LLAMIA vs. LLAMIA-Verb gap, same backbone/data/recipe) widens throughout DAPO training, reaching 2–3x by convergence, and persists from 4B to 14B scale (e.g., Interest: LLAMIA-Verb-14B scores 8 vs. LLAMIA-4B's 38).
• Interface ablation table (14B): behavior-match accuracy rises from 39% (untrained tool use) to 45% (LLAMIA-Verb) to 53% (LLAMIA); Rationale BLEU-2 rises from 18.8 to 33.2 to 45.8; Commentary G-eval from 0.15 to 0.40 to 0.75.
• LLAMIA invokes the subagent less often than LLAMIA-Verb (1.9 vs. 2.9 calls/query at 14B), making it inference-cost neutral despite the added latent tokens; training cost stays within ~6% of the verbalized pipeline.
• Human study (n=12, ≥1700 Elo): LLAMIA-14B passes as human in 61% of trials (LLAMIA-Verb detected 72% of the time), and its commentary is preferred in 72.2% of 180 pairwise judgments over LLAMIA-Verb.
• Generalization to Go with KataGo-b18: LLAMIA-Go-14B achieves 48/50 top-1 human move-match at 5k/5d ranks, matching a rank-calibrated KataGo-HumanSL expert and outperforming the verbalized control by ~10 points.

Why it matters / caveats: The results suggest text is a structural bottleneck for LLM–subagent collaboration on tasks needing multi-step state tracking or signals (like policy/value distributions) that have no faithful textual surrogate, and that this gap does not close simply by scaling the LLM or RL budget. A key limitation is that latent internalization requires white-box access to the subagent and LLM weights, so it cannot be applied directly to closed-source models (the paper proposes using a trained LLAMIA as an indirect bridge instead); most evaluation is chess-specific with only a single additional domain (Go) explored.

Problem: Existing multimodal retrieval benchmarks either test only clean image-caption pairs or perturb image/text independently without keeping the target entity, gallery, and labels fixed, so none can isolate how mobile-capture artifacts (blurry snapped photos, short/mistyped questions) affect paired "snap-and-ask" retrieval.

Method: The paper introduces SnapBench, a paired benchmark built via a four-stage pipeline: (1) entity-centric question generation using Gemini-3-Flash and GPT-5.4-mini semantic checks over a private web-crawled image pool; (2) gallery collection with entity-aware captions from a fine-tuned Qwen-VL captioner; (3) deterministic snap-and-ask artifact simulation — 15 image operators (Add, Remove, Degrade, Transform) at 3 severities (45 conditions) and 8 text operators at character/word/sentence granularity — applied only to the query while the gallery stays fixed; (4) human annotation (10 trained annotators, 95% audit pass rate) producing positive/hard-negative/easy-negative labels via a 4-dimension rubric. The authors evaluate 16 vision-language retrieval models (dual-encoder: CLIP, SigLIP, SigLIP2, BLIP-ITM; VLM-embedding: Jina-V4, Qwen3-VL-Embedding-2B/8B, E5-V, VLM2Vec-V2/Full, UME-R1-7B, GME-2B/7B, Ops-MM-2B/7B, RzenEmbed-7B) and propose MOOR (Modality-anchored, Outlier-aware, Optimal Reweighting), a training-free adaptive fusion method that reweights four score paths per query using Pearson-correlation gating and score variance from the frozen encoder's own outputs.

Key results:

• SnapBench contains 1,145 queries, 9,085 gallery items, 53 corruption conditions, 5,462 positives (avg 4.77/query) and 4,005 hard negatives (avg 3.50/query).
• Clean R@1 ranges from 39.8 (SigLIP2-SO400M) to 79.1 (Ops-MM-7B), with no model exceeding 80; VLM-embedding models outperform dual-encoder baselines by ~13 points on average.
• Image corruption drops mean R@1 from 63.2 to 57.9 (severity-dependent: 0.8-pt drop at s1, rising to 11.2-pt drop at s3), while text corruption barely moves joint R@1 (63.5–63.9 vs. clean).
• Joint image+text corruption is super-additive: observed drops (13.6 R@1 avg) exceed the naive sum of single-modality drops (6.0), a +7.6-point interaction gap across six paired conditions and 16 models.
• Joint retrieval (IT→IT) underperforms image-only retrieval by a mean of −11.21 R@1 across all 16 models ("coarse-text drag"); replacing a coarse question with an entity-label answer on GME-7B cuts the drag from −7.68 to −0.17 (97.8% reduction).
• MOOR improves R@1 over fixed fusion by roughly +8.6 to +19.8 points depending on model class and corruption setting (Figure 4), and the per-query optimal fusion weight has std 0.086–0.138 across all 16 models, indicating miscalibration is architecture-independent.

Why it matters / caveats: SnapBench shows that clean-input accuracy does not predict robustness to realistic mobile capture artifacts, and that naively fusing image and text signals can actively hurt retrieval when text is coarse — motivating reliability-aware, per-query modality calibration rather than fixed fusion weights. The authors note limitations: corruptions are synthetic/programmatic rather than drawn from real noisy uploads, the benchmark prioritizes dense annotation over raw scale (1,145 queries), and MOOR offers no benefit when both modalities are uniformly informative or uninformative.

Problem: Animal motion generation lags far behind human motion research because high-quality 3D animal motion data is scarce — motion capture is impractical for most species, and existing datasets are either tiny, lack 3D motion, or rely on artist-crafted animations limited to a few predefined actions.

Method: Kirin has three stages: (1) reconstruct temporally coherent 3D SMAL-based motion from the AiM in-the-wild video dataset by combining AniMer per-frame initialization with sequence-level pose refinement (projection + SO(3) smoothness losses) and SpatialTrackerV2 for global translation, then caption clips with Gemini 2.5 Flash to build AiM3D (29,979 motion sequences, 179,874 captions across 23 quadruped categories); (2) train an image+text-conditioned motion diffusion model built on MDM with a frozen DistilBERT text encoder and frozen DINOv3 image encoder fused via additive conditioning into a transformer decoder, trained with classifier-free guidance and per-modality dropout (0.2 for text, 0.2 for image); (3) animate generated motion onto a Rodin-generated T-posed mesh via SMAL template fitting, k-NN skinning-weight transfer, and linear blend skinning.

Key results:

• On the AiM3D test set, Kirin (text+image) achieves FID 6.248 vs. 30.043/30.516 for AniMo baselines (trained on AniMo4D or AiM3D data) and Top-3 R-Precision 0.130 vs. ~0.088–0.089 for AniMo.
• On the out-of-distribution AnimalML3D benchmark (1,260 hand-crafted motions), Kirin gets FID 138.145 vs. ~150.7–155.7 for AniMo baselines, and Top-3 R-Precision 0.111 vs. ~0.093.
• For 4D motion reconstruction, Kirin reaches PCK@0.1 of 0.751 (vs. 0.537 for AniMer, 0.664 for 4D-Fauna) at under 1s/frame, comparable in speed to feed-forward methods and far faster than optimization-based baselines (~30s/frame for SMALify/4D-Fauna).
• Human validation of 100 reconstructed test samples found 86% physically plausible/smooth motions and an average caption correctness/diversity score of 4.62/5, with 90% of samples having fully correct captions.
• Adding image conditioning to text-only Kirin improves FID from 11.889 to 6.248 and Top-3 R-Precision from 0.097 to 0.130, at a small cost to Diversity/Multimodality.

Why it matters / caveats: Kirin and the AiM3D dataset provide the first large-scale, text-video-motion-aligned resource for quadruped animals and the first image+text-conditioned animal motion generator, with a downstream pipeline that automatically rigs and animates 3D meshes more reliably than a video-generation-based baseline (Puppeteer). Limitations include reconstruction failures on top/front/back-view or fast-motion videos due to occluded legs, and the FID gap to ground truth remains large on the out-of-distribution AnimalML3D benchmark.

Problem: Multimodal encoders such as ColPali-style visual document retrievers are typically built by repurposing generative VLMs (a pretrained vision tower bolted to a causal decoder), inheriting parameter and compute overhead that is unnecessary for non-generative encoding tasks.

Method: NeoMME is a family of 260M and 800M-parameter bidirectional single-tower Transformer encoders, trained from scratch, that feed raw 32×32 RGB image patches (via a 2-layer MLP, no pretrained vision tower) and multilingual text tokens (custom 131,072-vocab BPE tokenizer) through the same shared layers with 2D RoPE and a 16,384-token context. Pretraining uses a masked discrete-diffusion text-denoising objective (following MDLM/MD4/LLaDA) on ~524B tokens (55% text-only, 45% multimodal, e.g. FineWeb-Edu, FineWeb2-HQ, FineVision, PDFA), with higher corruption rates (30-100%) on multimodal segments to force reliance on image evidence. For downstream retrieval, the backbone is fine-tuned as NeoMME-Retriever with jointly trained dense (Matryoshka) and late-interaction (ColBERT-style MaxSim) heads via contrastive learning, using custom fused "Late-Interaction Kernels" (LIK) to reduce MaxSim memory/compute.

Key results:

• ViDoRe v3: NeoMME-Retriever 260M scores 0.523 nDCG@10 (best model <300M params, +26.1 points over the best other sub-300M model), and 800M scores 0.556 (within 0.9 points of ~850M-param Vultron Flash and 3.2 points above 3.75B-param ColQwen2.5).
• Throughput: NeoMME-260M encodes 2048×2048 pages at 51.3 pages/sec on an NVIDIA L40S, ~1.97× ColModernVBERT's throughput at matched resolution.
• Compression: hierarchical token pooling (factor 8) + asymmetric quantization (int8 query, binary document) shrinks late-interaction embeddings from ~1.5MB to 6kB per document (255× compression) while retaining 95.19% of baseline nDCG@10; int8/int8 quantization alone gives 3.9× storage reduction at ≤0.02 nDCG@10 cost.
• BEIR-15 text retrieval: late-interaction head reaches 0.513 nDCG@10 (800M) and 0.488 (260M), outperforming its own dense head by 14.4 and 18.3 points respectively, though below dedicated text retrievers like LateOn (0.572).
• Tokenizer emits 44.4% fewer tokens than ModernBERT and 6.3% fewer than mmBERT-base across 14 FLORES-200 languages, though coverage weakens across the full 204-language set.

Why it matters / caveats: Shows a vision-tower-free, from-scratch multimodal bidirectional encoder can match or beat much larger VLM-derived retrievers at a fraction of the parameters, and the released compression recipe makes high-resolution visual document retrieval storage-tractable at scale. The authors note NeoMME was pretrained on far less data than ModernBERT (~524B vs ~2T tokens), has no visual reconstruction/prediction objective (weakening pure image tasks), used much less retrieval supervision than SOTA text retrievers like mLateOn, and lacks broad multilingual coverage in visual retrieval data.

Problem: There is no method for converting unstructured origami instructional videos into explicit, executable parametric folding programs, since existing computational origami methods assume a structured input like a crease pattern is already given.

Method: FoldingAgent uses a pretrained VLM (Gemini 3.1 Pro Preview) as a zero-shot agent that, given keyframes from an origami video, predicts a sequence of Pureland folding actions (add_vertex, fold, unfold, rotate, flip) via a tool library, executed by a custom origami simulator operating on an extended FOLD-based state representation (vertices, edges, faces, per-face orientation, and layer ordering). A deterministic Controller dispatches tool calls (origami actions, viewing tools like view_frame/observe_movement/render_current, checkpoint save/restore, and a separate visual-critic VLM via ask_critic) and supports rollback/re-planning with a bounded exploration budget (C=5 attempts per keyframe, capped at 300 total tool calls) to mitigate compounding errors across long folding sequences.

Key results:

• On the newly curated PurelandFold benchmark (27 diverse Pureland folding sequences, self-captured and annotated with ground-truth geometry/actions from 40 Pureland models on the OrigamiWay "Easy Origami" set), full FoldingAgent completes 100% of sequences vs. 85% for VLM-S and VLM-S-C ablations and 96% for a naive VLM-CP (direct crease-pattern prediction) baseline.
• FoldingAgent achieves the best scores on all metrics: TSS 0.90±0.1, GS 0.58±0.1, CS 0.77±0.3, FFS 0.39±0.3, and lowest Crease-Pattern Dissimilarity (CPD) 189±145, versus e.g. VLM-S-C's TSS 0.80±0.1, CS 0.49±0.2, CPD 216±159.
• Naive VLM-CP (no proposed representation/tools) achieves only 8% Compilation Validity (CV), while switching to the paper's state/action representation (VLM-S) raises CV to 96%.
• A perceptual user study (600 judgments over 10 sequences) found FoldingAgent's reconstructions preferred over VLM-S in 84% of comparisons and over VLM-S-C in 80%.
• The visual critic has an 88% success rate at correctly judging match/mismatch; average per-sequence cost is 140 agent queries, ~4M input / 68K output tokens, $8.9.

Why it matters / caveats: The work is a first step toward converting the large body of human-shared origami video knowledge into structured, executable representations usable for robotics, generative design, and instructional tools. Limitations include dependence on the underlying VLM's reasoning ability, difficulty with heavily occluded folds and accumulating paper layers, inability to handle compound simultaneous actions (e.g., rotate+flip), and a scope restricted to Pureland (flat, single-crease) origami rather than more advanced folding techniques.

Problem: Agent benchmarks rarely control whether a task's difficulty comes from missing private/domain knowledge versus insufficient model capability, making it hard to construct tasks that genuinely require specialized knowledge while remaining exactly verifiable.

Method: The authors propose a knowledge-gated task-construction protocol that pairs each task instruction (kept byte-identical across conditions) with a separate kilobyte-scale "artefact" containing private conventions, reference tables, or utility operators, distinguishing "convention gates" (non-derivable planted choices) from "operator gates" (hard-to-hand-write algorithms). Tasks are checked with a static leak audit, verified via deterministic Python-based solvers (or criterion-level rubrics for free-form outputs), and screened using a configuration-relative calibration rule requiring ≥60% pass rate with the artefact under a frontier configuration, 0% without it, and ≤40% under a second, weaker configuration. Evaluation uses Claude Opus 4.8 via Claude Code (BenchFlow harness) as the frontier configuration and Qwen3.6-Plus via OpenCode as the mid-tier configuration, across a 15-task calibration batch.

Key results:

• Pooled pass rates across 15 tasks: 68.0% for Opus with artefact vs. 0% without it (5 trials/cell), and 22.7% for Qwen with artefact.
• A perturbed-artefact control on the bom-costing task (plausible-but-wrong conventions) yielded 0% pass rate, matching the no-artefact condition, versus 100% with the correct artefact.
• The configuration-relative screen retained 7 of 15 tasks (e.g., legal-clause, paper-index, bom-costing, airline-booking, clinical-variant, mortgage-escrow, dependency-audit); 5 are publicly released.
• A recitation probe on 2 retained tasks found Qwen recited gated conventions correctly and scored 100% in a single-turn setting, yet 0% under the full agentic harness — localizing failure to execution (tool use/file discovery/output compliance), not missing knowledge.
• On paper-index, compressing the artefact 4× (4379 B → 1078 B) caused no drop in pass rate (both 100%), while 0 B gave 0%.
• Context benchmarks (τ²-bench, SciCode) show only 18–36 point gains from adding helpful (not strictly gated) knowledge, versus the paper's 0%→68% jump for construction-time-gated knowledge.

Why it matters / caveats: The protocol offers a way to build verifiable agent tasks whose difficulty is attributably knowledge-gated rather than capability-gated, useful for evaluation and potential future RL training design. Caveats: only 15 calibration tasks with 5 trials each (high sensitivity to single outcomes), leak audits are automated for only 9/15 tasks and cannot rule out semantic leakage, the two calibration configurations differ in both model and harness (confounding comparisons), and the authors explicitly do not demonstrate that retained tasks improve post-training — no training experiments were run.

Problem: Post-training weight pruning methods like SparseGPT can amplify existing demographic biases in LLMs, but no prior pruning method explicitly mitigates this bias amplification during compression.

Method: Debias-SparseGPT extends SparseGPT's second-order (OBS-style) pruning objective by adding a term over paired pro-stereotypical and anti-stereotypical calibration inputs (X0, X1) from the StereoSet development set (4212 examples), producing a bias-aware Hessian H = X0X0^T + X1X1^T + 2ΔXΔX^T (ΔX = X0−X1) that jointly informs pruning-mask selection and second-order weight reconstruction, while preserving SparseGPT's O(d_hidden^3) complexity. It is implemented on top of LLM-Compressor and evaluated on nine LLMs (LLaMA-3.1-8B-IT, Vicuna-7B-v1.5-IT, Qwen-2.5-7B-IT, Mistral-7B-v0.3-IT, Aya-Expanse-8B-IT, Phi-4-4B-Mini-IT, Gemma-9B-IT, Qwen-3-8B, DeepSeek-8B) across unstructured (25%, 50%) and semi-structured (1:4, 2:4) sparsity, against magnitude pruning, Wanda, and SparseGPT baselines.

Key results:

• On LLaMA-3.1-8B at 1:4 sparsity, UnQover accuracy improves from 35.60% (SparseGPT) to 60.46% with Debias-SparseGPT, vs. 41.98% for the next-best baseline (Wanda), while perplexity stays comparable (8.19 vs 8.17 for SparseGPT).
• Distance-to-Optimum (DTO, computed on MMLU+UnQover) drops from 0.539 (SparseGPT) to 0.399 for LLaMA, from 0.311 to 0.291 for Qwen, and from 0.695 to 0.674 for Vicuna.
• On Qwen-2.5-7B, MMLU accuracy is preserved (67.73% vs 67.35% for SparseGPT) while UnQover improves (74.41% vs 70.60%).
• Under aggressive 2:4 sparsity, augmenting StereoSet calibration with 256 UltraChat examples raises Debias-SparseGPT's UnQover from 24.9% to 47.3% and MMLU from 48.16% to 54.17% (vs. SparseGPT+UltraChat's UnQover 42.46%, MMLU 53.84%).
• On Qwen-2.5-7B with 2:4 structured sparsity, Debias-SparseGPT matches SparseGPT's inference throughput (73.05 tok/s, both up from 27.54 tok/s dense) and CO2 cost (0.0376 kg/Mtok) while achieving lower DTO (0.494 vs 0.522).

Why it matters / caveats: Debias-SparseGPT shows bias amplification from pruning can be mitigated at pruning time with no extra training cost or inference slowdown versus SparseGPT. Limitations noted by the authors: evaluation is monolingual (English only), representational bias is the primary safety dimension studied (toxicity/harm results are only in an appendix), and under the most aggressive 2:4 sparsity perplexity still degrades substantially, requiring richer calibration data to recover performance.

Problem: It is unclear whether text-only LLMs that can generate code producing recognizable images actually hold an internal representation of 2D spatial layout, or merely translate explicit spatial descriptions into code, since prior evaluations conflate spatial composition with the ability to express it in an output medium.

Method: The authors introduce AM-Bench, built around "autoregressive mosaics" — short LLM-written Python programs (using six primitives: fill, set pixel, rect, circle, line, poly) rendered to a 24×24 raster via a custom, contamination-resistant API. It separates a translation task (fully-specified geometry in words, scored via an exact-area PIoU metric) from a layout task (underspecified prompts across 150 prompts/15 subcategories in 3 tiers, scored by two VLM judges, Qwen2.5-VL-7B and InternVL3-8B). Eight open-weight text-and-code models (8B–34B: Qwen2.5-Coder 32B/14B, Gemma 2 27B/9B, GLM-4 32B/9B, CodeLlama 34B, Llama 3.1 8B) are evaluated, plus a canvas-vs-SVG output-medium ablation and a linear residual-stream probe (with a causal truncate-and-resume intervention) to test for pre-generation spatial plans.

Key results:

• All 8 models pass the translation task with median PIoU = 1.0 (τ=0.6 threshold), while layout scores vary substantially (GLM-4 32B best at 3.57/5 pooled under Judge 1 vs. CodeLlama 34B worst at 1.75/5), showing layout gaps aren't explained by coding ability.
• Replacing canvas code with raw SVG improves layout scores across all 8 models, pooled +0.37 (95% CI [0.26, 0.47]), with CodeLlama 34B improving most (+0.76).
• Both judges agree exactly on model ranking (Spearman ρ=1.0) despite moderate per-sample agreement (Pearson r=0.43–0.54); human validation matches judges on 82.4% and 76.7% of agreed pairs.
• Pre-generation residual-stream probes decode layout above a text (TF-IDF) baseline for all 8 models (mean ΔR²=+0.164), but decomposition shows only the shared/prompt-implied component is decodable (ΔR²=+0.26–0.28) while the model-specific realization is not (R²≤0.005).
• Causal intervention: displacing a drawn shape into an unused region raises the model's likelihood of continuing to draw from 43.8%→84.6% (GLM-4 32B) and 2.5%→41.2% (Gemma-2 27B), indicating models track the evolving textual geometric state rather than executing a pre-fixed plan.

Why it matters / caveats: Results show 2D spatial performance in text-only LLMs depends on both the model and the output medium and is not just a code-generation issue, with an incremental/autoregressive (not pre-planned) drawing process. Limitations: only 8B–34B open-weight models tested (no larger/closed models), VLM-as-judge introduces variance, and the causal test intervenes on code text rather than internal activations.

Problem: Mean-variance portfolio construction requires a cross-asset return covariance matrix, which is unreliable to estimate in short, high-dimensional return panels, so the paper asks how much portfolio risk can be certified from firm information alone before any covariance matrix is estimated.

Method: Each firm is represented as a distribution ("cloud") of its news article embeddings rather than a single pooled vector, and pairwise quadratic Wasserstein-2 (W2) distances between firms' embedding clouds are computed via balanced optimal transport. Under a maintained "antilipschitz carrier" map plus firm-specific slack linking observed information distance to latent systematic exposure separation, the paper derives a sharp multi-firm transport-dispersion bound and a computationally simpler weighted-pairwise relaxation, both yielding a one-sided upper bound on portfolio variance (1 − C(q)) without using return covariances; minimizing this certified bound over a compact long-only simplex gives an "information-certified" allocation. The empirical instantiation ("news-only allocation") uses frozen Qwen3-Embedding-8B (4,096-dim) embeddings of Nasdaq per-symbol news articles (2018–2022) for 52 aligned S&P/Nasdaq-style firms with daily Yahoo Finance returns, and is compared against equal-risk (inverse-volatility) weights and the ex post sample global-minimum-variance (GMV) portfolio across four prespecified Dirichlet-capped reference portfolio populations; robustness checks vary embedding width (4096/1024/256/64), model (Qwen3-8B/4B, BGE-large-en-v1.5), and the authors' own EttaX encoder vintages.

Key results:

• News-only allocation's in-sample standardized variance percentile ranges from 0.69% to 1.33% across four reference populations, vs. 21.1%–28.6% for equal risk weights.
• Standardized variance: news-only 0.357 (135.6 relative to GMV=100) vs. equal risk 0.390 (147.8) vs. sample GMV 0.264 (100.0) — news-only is 8.3% lower variance than equal-risk but 35.6% above the covariance-informed GMV.
• Allocation holds 38 of 52 firms with 12.1% max weight and effective N of 23.65; 87.1% of certificate credit comes from cross-sector firm pairs vs. 12.9% within-sector.
• Representation sensitivity: Qwen3-8B at 1024 dims is statistically indistinguishable from the full 4096-dim result (paired interval includes zero), but Qwen3-4B ranks significantly worse (Holm-adjusted), and an extreme 64-dim compression paradoxically achieves the lowest variance rank in the ladder (0.02 percentile).

Why it matters / caveats: The method offers a covariance-free, text-derived way to certify (bound) rather than estimate portfolio risk, potentially useful when return histories are too short for reliable covariance estimation. However, the authors stress the evidence is descriptive and in-sample only (same 2018–2022 window used for both news geometry and return evaluation, no out-of-sample test), depends on unidentified maintained assumptions (carrier/slack/return-bridge restrictions) and on a coverage-screened 52-firm survivor panel, and the certified allocation still underperforms the covariance-informed GMV by 35.6%.

Problem: Spatial asset-pricing models take the firm-interaction matrix W as an exogenous input and leave both its origin and the economic meaning of the estimated spatial coefficient ρ unexplained.

Method: The authors represent each firm as a probability distribution over a language-model embedding space (article-level empirical laws) rather than a single point, and introduce "target-anchored Wasserstein barycentric reconstruction": for each target firm, quadratic optimal transport aligns its articles with every candidate peer's articles, then a convex simplex step fits nonnegative unit-sum weights that best reconstruct the target cloud from its aligned peers, producing a directed, row-stochastic barycentric interaction field W♭. This field is plugged into a quadratic exposure-adjustment optimization (peer-adjusted exposure vs. stand-alone exposure) that yields a spatial autoregression with feedback coefficient ρ = λ/(1+λ), estimated via QMLE with stationary-bootstrap inference. Firm text embeddings are produced with Qwen3-Embedding-8B (4,096-dim, plus sensitivity checks at other widths/models including Qwen3-4B, BGE-large-v1.5, and matched EttaX encoders); news comes from Nasdaq's syndicated per-symbol archive and returns from Yahoo Finance (yfinance).

Key results:

• On 52 firms with W♭ frozen from 2018–2022 news, pooled 2023–2026 QMLE gives adjustment index λ̂ = 3.46 (95% CI [2.89, 4.17]), implied ρ̂ = 0.776, with higher conditional quasi-log-likelihood (110,994.7) than RBF-Wasserstein diffusion (108,580.9, λ̂=2.65) and equal-active-support weighting (108,987.4, λ̂=2.91).
• In a joint model with a persistent news co-mention field, barycentric channel λ̂B = 2.33 [2.00, 2.70] and news channel λ̂N = 0.86 [0.61, 1.18]; boundary-calibrated QLR tests reject exclusion of either field (QLR_B = 1772.1, QLR_N = 339.5, both p = 0.0005).
• The two fields select substantially different peers (barycentric field averages 38.8 active peers/row vs. 27.1 for news co-mention; only 0.579 overlap vs. a matched-random null of 0.531, p=0.001) but produce correlated induced return signals (correlation 0.898 vs. null floor 0.700, p=0.001).
• Annual re-estimates under the same frozen field show persistence: λ̂ ranges 2.89–3.94 across 2023–2026, all intervals excluding the zero-feedback benchmark.

Why it matters / caveats: The paper reframes spatial factor models by deriving the interaction matrix from firm-level text distributions instead of assuming it, giving ρ an economic (peer-misalignment penalty) interpretation. The authors stress the estimates are conditional working-model quantities, not causal peer effects or structural parameters: predetermination of the field removes same-sample reflection but not omitted-variable confounding from industries, technologies, or attention, and results depend on a fixed embedding model, ground metric, and article-balancing/truncation choices.