AI papers — 2026-08-26

Problem: RL post-training for video MLLMs is sample-inefficient because on-policy rollouts sampled by methods like GRPO rarely recover the precise intervals, boxes, masks, or trajectories in the annotations, leaving many groups without a reliable positive anchor, while chain-of-thought (CoT) generation adds cost without clear gains.

Method: The paper introduces OraRL, which appends each annotation, serialized by a task adapter into the model's response format, as an additional "oracle rollout" to the on-policy GRPO group (annotation-as-rollout). Because naively normalizing the mixed group lets the high-reward oracle raise the group baseline and flip otherwise-positive on-policy advantages to negative ("advantage inversion"), OraRL uses a decoupled advantage estimator: an on-policy-only baseline determines a base advantage, an oracle-gap directional gain amplifies above-mean on-policy rollouts, and a separate detached, bounded oracle advantage encodes the residual oracle-policy gap. For efficiency, sign-balanced pruning retains only the oracle plus the strongest positive and negative on-policy rollouts (with post-selection moment correction to re-center/rescale), so only K of n+1 rollouts are back-propagated through. The resulting model, Video-ORA (0.8B–9B, built on Qwen3.5/Qwen3-VL backbones), is trained answer-only (no CoT) across seven task families using this single recipe.

Key results:

• Efficiency: OraRL requires 2.2× the step time of SFT, versus 4.9× for GRPO with CoT; sign-balanced pruning gives a 1.48× speedup with only 0.4-point average loss (step time 92.5→62.4 s, peak memory 62.4→50.9 GB at κ=0.5).
• Video-ORA-9B decodes in 130 ms per response instead of 4,780 ms for CoT decoding after time-to-first-token; median end-to-end latency on 10-min videos drops from 29.0 s (CoT backbone) to 24.3 s (answer-only).
• Temporal grounding: best mIoU on all three TimeLens benchmarks (Charades 61.8, ActivityNet 63.6, QVHighlights 72.5), beating TimeLens2-8B by 2.3–5.0 mIoU and exceeding Gemini-2.5-Pro.
• Tracking (GOT-10k): AO of 78.2 vs. 73.0 for OneThinker-8B (+5.2).
• Segmentation: MeViS J&F 61.3 (vs. 52.7 best baseline), ReasonVOS 63.7–63.8, RefCOCO cIoU 79.4.
• Spatial-temporal grounding (STVG): leads all four metrics; +15.5/+11.9 points sIoU@0.5/mean sIoU over Qwen3.5-9B backbone.
• Video QA: ranks first on 5/7 benchmarks, macro average 66.8 (backbone 61.9); e.g., +15.2 on VideoHolmes, +5.6 on VideoMME.
• Spatial intelligence: VSI-Bench average 73.1 vs. 55.0 (GPT-5) and 55.1 (Gemini-3-Pro); three-benchmark (VSI/MMSI/MindCube) macro average rises from 51.0 (prior best open model) to 56.1.
• Scaling: OraRL improves its Qwen3.5 backbone at every scale from 0.8B to 9B (macro average 51.8→66.2) and outperforms GRPO at every data budget up to 100k prompts (e.g., +5.2/+3.6 points on video-perception/spatial-intelligence aggregates vs. +2.8/+3.1 for GRPO from 6.4k to 100k prompts).
• Advantage inversion analysis: naive oracle mixing inverts 22.4% of GRPO-positive rollouts (42.5% of groups affected, 8.3% lose all positive rollouts); reward shaping (Tempsamp-R1 style) reduces this to 11.9%; OraRL reduces it to 1.9% before pruning and 0.3% after.
• Training paradigm comparison (3-task protocol, Qwen3.5-4B): OraRL reaches 62.7 average vs. 60.3–60.7 for GRPO/Dr.GRPO/GDPO/CPPO and 54.7 for LUFFY-style off-policy injection, and 58.7 for continued SFT.
• Component ablations: removing directional gain (-1.5 avg, -2.7 tracking), detached oracle advantage (-1.4 avg, -2.5 temporal), or reward-gap weight (-1.1 avg) each hurts; removing sign balance costs -0.6 avg vs. magnitude-only pruning.
• CoT ablation: under GRPO, CoT lowers backbone average (58.7→58.5) while increasing step time 44.4% (93.9→135.6 s); OraRL without CoT improves average by 2.7 points over answer-only GRPO while cutting step time 33.5%.

Why it matters / caveats: Annotation-as-rollout offers a task-agnostic way to inject reliable positive supervision into on-policy RL for video MLLMs without CoT, improving both sample efficiency and inference latency at scale. The authors note limitations: the method assumes each annotation can be serialized as a valid oracle rollout scored by a scalar reward, its behavior under ambiguous/partial/noisy supervision or learned oracles is untested, and on some spatial-reasoning tasks requiring complex reasoning (e.g., route planning) Video-ORA still trails proprietary models like Gemini-3-Pro and GPT-5.

Problem: Build a family of universal multimodal embedding models that can encode arbitrarily interleaved text, image, and video inputs into a shared representation space with strong performance across public benchmarks and real-world WeChat recommendation/search applications.

Method: WeMM-Embedding comprises 2B, 4B, and 9B models built on Qwen3.5 multimodal LLM backbones, using last-token pooling via a dedicated <embedding> token and Matryoshka Representation Learning (MRL) for flexible output dimensions. Training uses a two-stage strategy: Stage 1 performs large-scale multimodal alignment on several hundred million pairs using InfoNCE contrastive learning plus a score-gap-weighted CoSENT-style ranking objective for graded-relevance data, with task-consistent batching and duplicate-aware masking; Stage 2 fine-tunes on a curated dataset (built via Semantic-ID-guided resampling with RQ-KMeans quantization, MLLM-based quality filtering, and hard-negative mining) using additional reranker supervision (CoSENT-style with reranker scores) and bidirectional embedding distillation (KL divergence between teacher/student source-to-target and target-to-source similarity distributions) from a larger frozen 9B teacher (2B/4B models) or via model merging of multiple specialized variants (9B model, since no larger teacher exists).

Key results:

• MMEB-v2 (78 datasets): WeMM-Embedding-2B scores 77.9 overall, beating Qwen3-VL-Embedding-2B by 4.7 points and DME-2B by 3.1 points, and slightly surpassing Qwen3-VL-Embedding-8B (77.8); 4B reaches 79.2; 9B reaches 80.6, ranking first on the official MMEB-v2 leaderboard, ahead of all listed open-source and proprietary models (e.g., DME-Large 80.2, Octen-VL-Large 80.1).
• MMEB-v3 (190 tasks, V3-All): 2B model scores 56.0 (Text 45.3, Agent 45.1), 4B scores 58.2, 9B scores 59.5 — all outperforming baselines including Qwen3-VL-Embedding-8B (53.5).
• Cross-modal retrieval suite (12 public benchmarks from the Gemini Embedding 2 report): 2B scores 79.8 AVG, 4B 80.8, 9B 81.7 — comparing favorably with proprietary models Gemini Embedding 2 (79.5), Amazon Nova MME (70.2), and Voyage Multimodal 3.5 (71.8).
• In-house 26-task WeChat benchmark: 2B model scores 72.0 AVG vs. 60.9 for Qwen3-VL-Embedding-2B, with gains across all five categories (Classification, Search, Cross-domain matching, Article relevance, Video relevance).
• Deployed at scale in production with consistent gains across 14 online A/B tests in WeChat Channels, Official Accounts, Moments, and e-commerce.
• MRL dimension analysis: at 256 dimensions the 2B model retains 98.7% of 2048-dim performance on image/video tasks; retrieval tasks are most sensitive to dimension reduction, classification least sensitive.
• Stage-1 ablation (2B model, MMEB-v2 AVG 71.9 full): removing task-consistent batching causes the largest drop (-3.4 points to 68.5), removing task-specific instructions costs -0.8, removing duplicate-aware masking costs -0.5.
• Stage-2 cumulative ablation (2B model): starting from Stage-1 checkpoint (75.7) to full Stage-2 (77.9) — curated data +0.9, reranker supervision +0.1, embedding-teacher distillation +0.9, expanded visual input budget +0.3.

Why it matters / caveats: The compact 2B model already beats prior 8B open-source baselines, showing strong parameter efficiency, and the approach has demonstrated real production impact (14 A/B tests, deployment across WeChat Channels/Official Accounts/Moments/e-commerce) rather than only benchmark gains. Caveat: the models do not support audio input (11 MMEB-v3 audio tasks scored as zero), and several top baseline numbers (e.g., DME-Large, QQMM-embed-v4, Octen-VL-Large) come from closed-source leaderboard submissions without public weights, limiting independent verification of some comparisons.

Problem: Manually tuning the external harness (prompts, tools, control logic) that wraps an LLM agent to make it reliable on long-horizon tasks is time-consuming, expensive to evaluate, and hard to scale.

Method: AutoSaddler formulates harness optimization as an offline, mini-batch learning problem over a harness parameter space θ = (θ_prompt, θ_tool, θ_middleware), iterating a Diagnosis-Patch Session, Reflection Session, and Evolution Session (all implemented as agents built on the Claude Agent SDK). The Diagnosis-Patch Agent performs in-depth, evidence-grounded diagnosis of failed traces plus the harness codebase and emits structured patches from a fixed taxonomy (Prompt, Tool, Middleware, each split into Capability Patches and Steering Patches), following Phased Patch Scheduling (Capability-patch phase before Steering-patch phase, analogous to LR scheduling). Accepted patches (those improving mini-batch score) are checked for generalization on a held-out dev set, and a Reflection Agent extracts lessons (fixed/regressed/still-failing/still-passing) stored as node/edge attributes in EvoDAG, a directed acyclic graph of harness evolution history that an Evolution Agent consults to synthesize/recombine the next candidate harness.

Key results:

• GAIA2: default agent 53.0% → AutoSaddler 62.0% Pass@1 (+9.0 pp); beats strongest automated baseline (GEPA, 54.6%) by +7.4 pp; Meta-Harness scored 53.2%.
• SWE-Bench Pro: SWE-agent baseline 37.3% → AutoSaddler 46.9% (+9.6 pp, paper abstract) / reported as +8.4 pp in body text; beats GEPA (42.5%) by +6.2 pp and Meta-Harness (35.3%).
• Terminal-Bench 2.0: Terminus 2 baseline 40.0% → AutoSaddler 50.0% (+10.0 pp); beats strongest automated baseline Meta-Harness (43.3%) by +6.7 pp, and also beats the manually expert-tuned Terminus KIRA harness (47.5%) by +2.5 pp.
• Compute/learning efficiency on GAIA2: AutoSaddler reaches 72.3% dev accuracy with ~1,000 task executions, vs. GEPA (64.6%) and Meta-Harness (61.5%) saturating after ~2,800 executions; AutoSaddler reaches its best dev score after only 147 leveraged traces, ~10x fewer than Meta-Harness (1,400 traces).
• Cross-model transfer: harnesses optimized with Claude Opus 4.6 still improve the base harness by +5.6 pp when the task agent is switched to Claude Haiku 4.5.
• Ablation (in-depth diagnosis): removing it drops GAIA2 Pass@1 from 62.0% to 57.8%; full diagnosis-patch sessions invoke on average 6.2 more tool calls and 5.8 more file accesses per step, and yield more accepted patches (13 vs. 5 by end of Epoch 1).
• Ablation (structured intervention): removing the patch taxonomy/phased scheduling drops Pass@1 from 62.0% to 56.9% (removing only phased scheduling: 60.7%→54.8%); without it, patches collapse onto Steering edits (91.5%) vs. a balanced mix in the full method; Capability patches (New Tool 83%, Loop Change 71%, Infra Change 67% acceptance) are largely unexplored (4% share) without structured intervention vs. >25% with it.
• Ablation (generalization-aware selection): removing reflection + dev-set filtering drops Pass@1 from 62.0% to 50.6% (largest single-component drop; removing dev-set filtering alone: 60.7%→50.0%, further removing reflection/EvoDAG: 44.9%); AutoSaddler shows a decreasing regression-rate trend (-0.24 pp/iter) vs. an increasing trend (+0.16 pp/iter) for the ablation.

Why it matters / caveats: The results suggest automatic harness optimization can outperform both manual expert tuning and prior automated baselines (GEPA, Meta-Harness) while using far fewer rollouts, and the ablations indicate three specific ingredients — deep trace/code diagnosis, constrained/structured patching, and validation-based generalization filtering — are each independently necessary rather than incidental design choices. Caveats stated in the paper: AutoSaddler incurs moderately higher optimizer-side monetary cost per patch than baselines; evaluation is limited to three benchmarks (GAIA2, SWE-Bench Pro, Terminal-Bench 2.0) with a single main optimization run per method (three repeated test-set runs); code/website are only promised ("will be available"), not yet released at time of writing.

Problem: Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint (outcome-level) rewards do not specify how an intermediate denoising prediction should change, creating a structural mismatch between where reward is observed and where the diffusion policy acts.

Method: The paper introduces DiffusionOPSD, an on-policy self-distillation framework with three stages run each outer iteration: a frozen behavior policy generates trajectories and supplies low-noise query states and clean-output anchors (via the affine clean-output map y = z − σv); reward ascent/descent steps construct bounded positive and negative clean-output targets around each anchor (trust-region-projected, stop-gradient detached); and the trainable policy fits these targets as detached supervision through positive/negative fitting branches (with group-normalized endpoint-reward weighting and adaptive normalizers) under a finite update budget (Mfit, canonically 1), after which the behavior policy is refreshed via exponential moving average (EMA) before the next iteration. This design (contrasted with FlowGRPO's trajectory-credit likelihood ratios, ReFL's differentiable reward backprop, and DiffusionNFT's endpoint-conditioned regression) lets target construction and finite realization be measured separately, and is evaluated on SD3.5-M and step-distilled Z-Image-Turbo trained on Pick-a-Pic prompts and evaluated on held-out DrawBench prompts.

Key results:

• Best final held-out score in 19 of 20 reward-matched settings across two backbones (SD3.5-M, Z-Image-Turbo) and ten evaluators (PickScore, CLIPScore, HPSv2.1, Aesthetic, ImageReward, HPSv3, DeQA, AltCLIP, VLM-Pointwise, VLM-Pairwise), outperforming the strongest competing method by up to 44.0%.
• On SD3.5-M: leads 9/10 comparisons, with gains of 43.0% on HPSv3 and 44.0% on VLM-Pairwise; only loses on Aesthetic (12.08 vs. ReFL's 12.09).
• On Z-Image-Turbo (native 9-step): leads all 10 comparisons, with gains of 9.7% (Aesthetic, 9.79→10.74), 30.7% (ImageReward, 1.37→1.79), 4.9% (HPSv3, 13.77→14.44), 3.9% (DeQA, 4.60→4.78), and 14.6% (VLM-Pairwise, 0.481→0.551) over the strongest baseline; DiffusionNFT falls below the unadapted backbone on 8/10 objectives here.
• Reduces training GPU-hours relative to DiffusionNFT by 40% (SD3.5-M) and 63% (Z-Image-Turbo: 149.8 GPU-hours/100 updates); also cheaper than ReFL on SD3.5-M (28.2 vs. 47.7 GPU-hours/100 updates), though ReFL is cheaper on Z-Image-Turbo (102.1 vs. 149.8) yet still lower-quality.
• Across 71 single-reward runs, DiffusionOPSD reaches a median terminal position of 98% of its observed reward range, vs. 97% (ReFL), 90% (DiffusionNFT), 83% (FlowGRPO).
• Jointly trained (3-reward) policy: PickScore 25.51, CLIPScore 0.333, HPSv2.1 0.389 vs. DiffusionNFT's 23.62, 0.294, 0.340 (relative gains 8.0%, 13.3%, 14.4%); retains 113.0%/92.9%/99.6% of the gains of separate 100-update specialists.
• Human evaluation (100 held-out prompts): annotators prefer DiffusionOPSD over base model, FlowGRPO, DiffusionNFT, and ReFL on 64%, 71%, 90%, and 61% of prompts respectively.
• Target-construction analysis (512 prompts): DiffusionOPSD's positive target raises fixed-suffix reward by 0.03511 (gradient alignment 0.7094), vs. DiffusionNFT's rollout endpoint changing reward by −0.03551 (alignment ≈ −0.0002); after 50 optimizer updates, canonical target reaches CLIPScore 0.3122 vs. 0.2363/0.2303/0.1256 for no-op/random-direction/rollout-residual targets.
• Key analytical finding: a larger target-construction gain does not guarantee a larger realized gain after one fitting update — on HPSv2.1, target ordering reverses on 62.3% of 512 prompts (95% CI 58.2–66.6%); on CLIPScore the reversal rate is 29.5% (CI 25.6–33.4%).
• Ablations (SD3.5-M, CLIPScore, 50 updates): replacing the reward-gradient target with random/no-op/rollout-residual targets drops CLIPScore from 0.3117 to 0.2311/0.2280/0.1456; swapping the query source to a forward-noised control only drops it to 0.3103 (loss of 0.0014), showing reward-gradient direction dominates over query-state provenance.

Why it matters / caveats: The method offers a more efficient and analyzable alternative to existing reward-to-target interfaces (FlowGRPO, ReFL, DiffusionNFT) for diffusion post-training by making target construction and model fitting separately measurable, and it works even under native few-step (step-distilled) sampling where DiffusionNFT often degrades performance. Caveats acknowledged by the authors: it does not always win (e.g., SD3.5-M Aesthetic where ReFL is marginally better), the same-query reward-target vs. finite-fitting ordering can reverse (larger construction gain does not ensure larger realized gain), CFG training was found to create inference-scale dependence without improving best scores, and some ablation findings (e.g., a DiffusionNFT+reward-gradient auxiliary-loss variant scoring higher in a small screen) were not confirmed on the full DrawBench protocol.

Problem: Open-source cybersecurity LLM efforts lack a reproducible, unified, and scalable training pipeline that turns real-world vulnerability evidence into verifiable agentic supervision spanning vulnerability detection, patching, and question answering.

Method: CyberFactory converts public vulnerability artifacts (ARVO, OSS-Fuzz, and from-the-wild CVEs) into executable, difficulty-calibrated task instances by reconstructing pre-patch/post-patch Docker builds and validating candidate PoCs with a differential oracle (crashes pre-patch, not post-patch). A reusable, task-independent vulnerability-analysis "skill" guides a teacher model (GLM 5.2) through source inspection, domain-guided exploration, and evidence-based validation during trajectory synthesis, with long-horizon context compaction (triggered at 90% of the 256K context) preserving verified evidence while discarding redundant logs. The resulting agentic trajectories (PoC generation, patch generation, CyberQA) are used for full-parameter supervised fine-tuning (3 epochs, Adam, cosine LR peak 2e-5, 131,072-token packed sequences, BF16 on 256 GPUs) of Qwen 3.5-397B-A17B to produce OpenAegis, which internalizes the skill-guided procedure without needing the skill at inference time.

Key results:

• On CyberGym (one-hour per-task budget), OpenAegis achieves 58.1% Pass@1, vs. 29.6% for its Qwen 3.5-397B-A17B base (+28.5 points), 43.3% for GLM 5.2-744B-A40B (+14.8 points), and 51.7% for Kimi K2.7-1T-A32B (+6.4 points) — despite OpenAegis having fewer parameters than the other two baselines.
• Context management ablation: compaction at 90% threshold gives 58.1% overall Pass@1 (48.7% on long-horizon tasks) and 7.0% context-exhaustion rate, vs. 52.1%/40.2%/18.7% for full history and 45.6%/36.8%/24.5% for simple truncation; compaction at 80% gives 54.5%/45.5%/10.4% and at 95% gives 56.8%/47.1%/8.2%.
• Applying the vulnerability-analysis skill to GLM 5.2 at inference time raises Pass@1 from 43.3% (one 60-min attempt) to 46.5% (five 15-min attempts), while increasing exploration coverage from 3.78% to 99.85% of trajectories and validation coverage from 0.13% to 98.41%.
• Skill-elicitation vs. internalization behavior: GLM 5.2 uses domain-guided exploration in 35.6% of trajectories vs. 99.7% for GLM 5.2+Skill; Qwen3.5 base uses it in 61.7% vs. 85.2% for OpenAegis after SFT — showing OpenAegis reproduces the shift without the skill at inference.
• Exploration/validation calls per trajectory: GLM 5.2 0.05/0.001 → GLM 5.2+Skill 2.06/2.17; Qwen3.5 0.01/0.00 → OpenAegis 1.32/1.05.
• Tool-use shifts after SFT (Qwen3.5 → OpenAegis): shell calls 70.1%→89.9%, read calls 28.4%→7.3%, single-operation calls 31.4%→13.5%, calls with 6–10 operations 4.3%→30.8%, calls with >10 operations 1.5%→9.0%, operations per shell call 2.7→5.5.
• Instrumentation/submission discipline (Qwen3.5 → OpenAegis): ASAN compilation events 155→1,795, ASAN-output checks 1,281→2,099, trajectories with exactly one submission 37.9%→48.2%, trajectories with ≥5 submissions 10.4%→2.0%.

Why it matters / caveats: Demonstrates that a reusable domain-analysis "skill" used only during data synthesis can be distilled into model parameters via SFT, yielding a smaller open-weight model (397B-A17B) that outperforms substantially larger general-purpose backbones (744B-A40B, 1T-A32B) on real-world vulnerability reproduction under matched scaffolding and budget. The authors note the evaluation is bounded by available CVE artifacts, benchmark coverage, and a fixed one-hour budget; some targets still require manual input construction rather than a fuzzing-first strategy; and patch generation and CyberQA lack equally rigorous evaluation in this paper.

Problem: In long-horizon LLM agent tasks, growing interaction histories cause agents to lose track of unresolved goals and invoke experiential skills that no longer match the current task state, blocking recursive self-improvement because final task outcomes alone give no signal about which memory mechanism needs repair.

Method: The paper introduces Recuris, a recursive Experiential–Working Memory (EM–WM) architecture built on a frozen LLM and a Skill Memory M_k = (E_k, W_k, ρ_k, C_k) — experiential skills (E), a working-memory state specification (W), a skill-invocation policy (ρ), and a checker set (C). Within a task, Working Memory tracks goal status (pending/done/blocked) grounded only in verified environment feedback (a "checker" confirms completion via tool receipts rather than the model's own claims), and this state drives state-grounded skill invocation (call-time or boundary-triggered) rather than retrieval from the full history. Across tasks, a fixed Meta-Agent (an LLM agent, e.g. built on Claude Code or reimplemented on DeepSeek Harness) reads structured execution traces from failed runs, attributes each failure to one of the four memory components, proposes a component-scoped patch, and a fixed validation gate admits the patch only if it repairs the source failure without regressing a held-out development set — forming a bounded recursive evolution loop that never updates the base model weights.

Key results:

• Across 4 benchmarks (τ²-Retail, τ²-Airline, SkillFlow, Terminal-Bench 2.1) and 10 models, Recuris improves task success in 35 of 37 completed model–benchmark pairs.
• On τ²-Retail: +23.3 points for the deployment model Doubao-2.0-Pro (58.1→81.4%), +17.8 for GPT-5.6 Sol (58.3→76.1%), +15.6 for Claude Opus 5 (72.4→87.9%, SOTA on that benchmark in this evaluation).
• On SkillFlow: +16.8 for Doubao-2.0-Pro (34.6→51.4%), +16.6 for Qwen3.6-27B (42.2→58.7%), +13.5 for Qwen3.6-35B (35.3→48.8%).
• Gains grow rather than shrink with task length: +17.0 to +44.7 points across horizon quartiles on τ²-Retail, up to +32.2 on the longest tasks; six long-horizon failure modes reduced by 20–86%.
• Ablations: adding experiential memory alone is worth only +2.0/+1.5 points (τ²-Retail/Airline, CIs include zero); adding verified working memory alone is worth +23.9; full EM+WM coupling (Recuris) reaches +25.4/+8.5. A model-controlled variant with identical skills injected every turn scores 18.0 points below Recuris and costs 147k vs 101k tokens per success.
• Component-fault localization accuracy: 13.0% from outcome alone, 37.0% from raw trajectory, 64.8% from the structured trace (macro accuracy against a 33.3% floor).
• Evolved memory transfers held-out: 9 packages across 3 evolution runs each clear the initial memory M0 by +9.01 to +17.44 points on 86 held-out tasks (all CIs exclude zero); two independent Meta-Agent implementations (Claude Code vs. DeepSeek Harness) converge to statistically indistinguishable gains (+11.92 vs. +10.47/+9.30) and the same repaired components.
• Gated updates break only 9.5% of previously-solved anchor tasks vs. 25.9% for re-running an unchanged package (p=0.013), showing the gate discards noise rather than real capability.
• Test-time adaptation mode (Terminal-Bench 2.1, no shared task structure) solves 60.9% of 87 tasks vs. 34.5% single-attempt baseline; at matched budget, adaptation adds +2.3 points over retrying with a frozen memory (not statistically significant, p=0.774), while the attempt budget itself accounts for +26.4 points.

Why it matters / caveats: The results suggest the externalized memory-control layer, rather than model weights, can serve as an attributable, reversible, and cross-model-portable surface for recursive self-improvement, and that frontier models (GPT-5.6 Sol, Claude Opus 5) are far from saturated on long-horizon tool-use tasks. Caveats stated in the paper: gains depend on shared task structure across the task family (cross-task evolution admitted no patches in 13 runs on the structurally heterogeneous Terminal-Bench 2.1); several benchmark cells show confidence intervals including zero (e.g., most τ²-Airline results, some SkillFlow/Terminal-Bench cells); and the critical memory component differs by domain (double dissociation between write-review and status-board mechanisms across τ²-Airline and τ²-Retail), meaning no single fixed design choice generalizes across domains.

Problem: Standard critic-based training recipes for LLM reinforcement learning are unstable, which is why group-based methods like GRPO that avoid a critic (at the cost of multiple rollouts per prompt and only outcome-level, per-response advantages) are typically preferred instead.

Method: The authors introduce Best Practice Critic Optimization (BPCO), a single-rollout actor-critic recipe built by ablating instability sources one at a time in a controlled sanity test. BPCO combines: DPPO (Divergence Proximal Policy Optimization, which clips based on the sampled token's absolute probability change rather than a probability ratio) instead of standard PPO; a critic value head bounded to the reward range via a scaled arctangent transform; decoupled GAE with an unbiased Monte Carlo value target (λ_V = 1) for critic training while using λ_π < 1 for the policy advantage; removal of batch-wise advantage normalization (using raw/unnormalized GAE advantages); and length-adaptive GAE (λ_π(L) = 1 − 1/(αL)) so the terminal-reward weighting is invariant to response length. BPCO can optionally condition the training-only critic on privileged, reward-defining information (e.g., reference answer, official solution, or grading rubric) that is hidden from the policy, analogous to centralized-training-with-decentralized-execution in multi-agent RL.

Key results:

• Sanity test (DeepSeek-R1-Distill-Qwen-1.5B, 1,460-problem solvable dataset): plain PPO with λ=1 collapses training reward; DPPO with λ=1 is stable but DPPO with λ=0.99 (used as a stress test) becomes unstable again, motivating the remaining fixes (bounded values, unbiased MC targets, no advantage normalization, length-adaptive GAE each individually improve stability/AIME 2025 avg@32 in this stress setting, per Figures 2–6).
• Larger dataset (DeepScaleR, ~40.3K problems, DeepSeek-R1-Distill-Qwen-1.5B, up to 24,000-token responses): BPCO variants (BPCO+Ans, BPCO+Ans+Sol) consistently outperform both a group-based (Dr. GRPO, 16 responses/prompt) baseline and a strong critic-based baseline (decoupled GAE + unbiased MC target + length-adaptive GAE, but unbounded value head and batch-wise normalization) in training/validation reward and AIME 2025 avg@32, with higher explained variance throughout training.
• Ablations on the larger dataset: removing the value bound slows training-reward improvement and reduces AIME 2025 avg@32; reintroducing batch-wise advantage normalization increases advantage magnitude growth (though gains from removing it are modest here); privileged information (reference answer, and to a smaller degree the official solution, available for only 7.3K/40.3K problems) speeds training and raises explained variance and AIME 2025 performance.
• Larger models (Qwen3-30B-A3B-Base and Qwen3-30B-A3B on DAPO-Math-17K, 12,000-token max generation): the critic baseline fails to improve AIME 2025 accuracy beyond the first 100 steps on Qwen3-30B-A3B (unstable), while BPCO achieves substantially higher AIME 2025 accuracy on both models; BPCO performs better than the group baseline on Qwen3-30B-A3B and comparably on Qwen3-30B-A3B-Base.
• Rubric-based rewards (Qwen3-4B-Base/Instruct-2507 judge on OpenRubrics): both BPCO and BPCO+Rubrics learn faster than group and critic baselines (group baseline eventually catches up); privileged rubric information does not improve final performance despite higher explained variance, attributed to the task's relative simplicity.

Why it matters / caveats: The results indicate a carefully designed critic (aligned output range, unbiased targets, unnormalized advantages, length-adaptive weighting, and optional privileged inputs) can match or exceed group-relative advantage estimation while requiring only one rollout per prompt, offering a practical single-sample alternative to GRPO-style methods. Caveats stated by the authors: evidence is limited to mathematical and rubric-based reward settings, BPCO assumes a known reward range, privileged variants require access to evaluator/reward-defining information, and critic training adds computation and memory overhead not captured by trajectory-matched comparisons; privileged-information gains are task-dependent and can be offset by overfitting in small-data regimes.

Problem: On-policy Distillation (OPD) provides dense token-level supervision from a teacher model but ignores trajectory correctness (capping student performance at the teacher's level), while RLVR provides task-level correctness but only sparse, outcome-level feedback, and prior attempts to combine the two rely on extra hyperparameters or heuristic switching rules.

Method: The authors reformulate sampled-token OPD's implicit per-token reward (the log-ratio log(πT/πθ) or log(πθ/πT)) as an RLVR-style reward and show its sign is misaligned with trajectory correctness in some cases; they then apply a ReLU gating mechanism—On-policy Distillation with Verifiable Reward (OPDVR)—that zeroes out ("gates") exactly the tokens whose implicit OPD update conflicts with the verifier signal, forcing correct trajectories to get non-negative rewards and incorrect ones non-positive rewards, with no added hyperparameters. Because this reformulation turns sampled-token OPD into a proper RLVR method, it can be combined with any policy-gradient algorithm; the authors instantiate this with GRPO to create Group Relative Policy Distillation (GRPD), which replaces the binary verifier reward with a GRPO-style group-relative advantage inside the same ReLU-gated reward. Training uses the Verl framework with Qwen3-4B/Qwen3-1.7B students distilled from GRPO-trained Qwen3-4B/Qwen3-4B-base teachers on DeepMath and DAPO-Math-17k data.

Key results:

• Same-architecture setting (Qwen3-4B student ← Qwen3-4B-RL teacher, avg@16): OPDVR reaches 36.9 AIME24, 28.1 AIME25, 64.8 AMC, 84.7 MATH500, 33.2 Minerva, 47.0 OlympiadBench (avg 49.1), vs. sampled-token OPD's 34.2/26.0/63.1/85.5/31.6/46.5 (avg 47.8) and Top-64 OPD's avg 47.4; OPDVR gains +2.7 on AIME24 and +2.1 on AIME25 over sampled-token OPD and surpasses the teacher (36.0) on AIME24.
• Cross-architecture setting (Qwen3-1.7B-Base student ← Qwen3-4B-Base-RL teacher): OPDVR reaches avg 22.8 vs. sampled-token OPD's 20.9 and Top-64 OPD's 21.7, with gains of +5.5 on AMC and +1.7 on MATH500 over sampled-token OPD.
• GRPD (OPDVR + GRPO-style group-relative advantage, same-architecture setting on DAPO-Math-17k): avg 49.4, beating GRPO's avg 44.8 (+6.5 AIME24, +10.9 AIME25) and OPD's avg 48.4 (+2.8 AIME24; better on 5/6 benchmarks).
• Ablation: an "inverse-gated" variant (gate applied in the opposite direction) scores avg 44.6, below both OPD (47.8) and OPDVR (49.1) on all six benchmarks, confirming the gating direction matters.
• Training dynamics: the zero-gated token ratio stays stable around ~0.48–0.50 (4B student) and ~0.40–0.44 (1.7B student) throughout training, while entropy/response-length trends vary by teacher-student pair rather than by objective.
• A theoretical single-token analysis (Appendix A) proves OPDVR's gradient is never anti-aligned with the verifier gradient (⟨ΔOPDVR, ΔRLVR⟩ ≥ 0), unlike OPD, and shows a case where OPDVR provably preserves a student's initial policy advantage and strictly outperforms both OPD and the teacher.

Why it matters / caveats: OPDVR is a hyperparameter-free way to unify dense distillation guidance with verifiable task-level correctness, and its RLVR-compatible formulation lets it plug into standard policy-gradient algorithms (GRPO, DAPO, PPO) as demonstrated by GRPD. Caveats: evaluation is limited to six math reasoning benchmarks with two teacher-student pairs (Qwen3-4B and Qwen3-1.7B/4B-Base) trained on DeepMath/DAPO-Math-17k, and OPDVR does not uniformly beat sampled-token OPD on every individual benchmark (e.g., it trails OPD on MATH500 in the same-architecture setting, 84.7 vs 85.5, and on AIME25 in the cross-architecture setting is tied rather than improved).

Problem: Research on smart glasses is fragmented across isolated devices, tasks, and benchmarks, so the field lacks a unified way to determine whether a complete system can sustain a reliable, temporally valid, correctable, and governable perception-state-interaction-action loop rather than just recognizing, answering, remembering, or acting in isolation.

Method: The survey formalizes smart glasses as closed-loop systems via a first-person observation-stream equation (visual, audio, motion/IMU, spatial/gaze, interaction, and device-state signals) mapped through a function Fθ to feedback, updated state, and optional actions under a constrained deployment objective (latency, energy, thermal, privacy, and social-cost limits). It consolidates hardware into eight verifiable capability axes (vision capture, audio, spatial/motion sensing, near-eye feedback, hands-free interaction control, computing/connectivity, data/reproducibility interfaces, deployment/trust signals), organizes the literature into seven interdependent foundational capabilities (first-person perception, multimodal context, persistent spatial state, personal memory, situated agentic action, embodied data interfaces, cross-cutting deployment constraints), and introduces an L0-L5 cross-capability level framework (Recording & Relay, Reactive Perception, Contextual Assistance, Persistent State, Governed Action, Embodied Coupling) where L0-L4 form a wearer-facing progression and L5 is an orthogonal cross-embodiment extension.

Key results: This is a survey/framework paper, not an empirical study, so it reports no experimental benchmark scores of its own; its "results" are the taxonomy and evidence maps it produces:

• Formal claim tuple C = ⟨T, H, I, Y, τ, S, A, P, Ω, V, R⟩ defining capability as a conditional, evidence-dependent claim rather than an intrinsic device/model property.
• 8 hardware capability axes and a product hardware matrix (Table 2) rating ~19 representative devices/platforms (e.g., Xiaomi AI Glasses, Ray-Ban Meta Gen 2, Meta Ray-Ban Display, RayNeo X3 Pro, Even Realities G1, Project Aria, Pupil Labs Neon, XREAL AURA) against the 8 axes and assigning each an L0-L5 level (mostly L2 for camera/audio-first consumer glasses, "potential L3" for camera+display AR glasses, L1 for camera-free HUD glasses, "partial L5" for research sensing platforms).
• 7 foundational capabilities and an L0-L5 level table (Table 3) with defined evaluation targets per level.
• 9 application scenes (daily assistance, accessibility, industrial workflow, healthcare/caregiving, education/training, mobility/transportation safety, social collaboration, spatial AI, embodied AI, plus other emerging scenes), mapped in Tables 4-5 to required capabilities, datasets/benchmarks, and systems/products.
• A 9-dimensional deployment framework and standardized evaluation protocol (Table 6) plus a design checklist (Table 7), and an evidence ladder spanning documentation, lab measurement, benchmarks, device-stream replay, fault injection, end-to-end studies, longitudinal deployment, and privacy/security audit.
• 8 open challenges (hardware budgets, longitudinal data/annotation, grounding/memory/uncertainty, personalization/accessibility, interaction timing/proactivity, ecosystem interoperability, privacy/security/governance, evaluation/version drift) and 6 roadmap directions (reproducible device/system profiles, privacy-aware longitudinal data engines, auditable memory and continual world models, adaptive/inclusive proactivity, interoperable action ecosystems, robot-validated transfer of human experience).

Why it matters / caveats: The framework gives researchers and practitioners a common vocabulary (hardware axes, L0-L5 claims, claim-conditioned evaluation) to compare heterogeneous smart-glasses products and research systems without over-generalizing benchmark or marketing claims into deployment-readiness assertions. As a survey, it synthesizes and reorganizes existing literature and public product evidence rather than presenting new experiments, models, or quantitative benchmark results, so its contribution is structural/taxonomic rather than performance-based.

Problem: Raw in-the-wild gameplay videos entangle the underlying game world with screen-space UI overlays (HUDs, menus, notifications, watermarks), introducing game-specific biases and irrelevant dynamics that degrade their usefulness as training data for video world models.

Method: The paper introduces GameUI-Taxonomy (21 categories organizing HUD elements by semantic role and rendering behavior) and G2WEngine, a four-stage pipeline (UI taxonomy definition, UI asset extraction from real gameplay frames via GPT-5.6 Terra annotation + human verification, clean gameplay corpus curation via CLIP-based clustering/deduplication, and temporally coherent UI overlay synthesis) that produces the Game2World dataset (96K synthetic paired videos plus 1,079 in-the-wild clips from 303 games, 5,132 verified UI assets). Built on this data, GameCleaner is a mask-free gameplay UI removal model initialized from Kiwi-Edit's Stage-2/Stage-3 instruction-editing checkpoints, coupling an MLLM encoder (replacing the conventional T5 text encoder) with a video diffusion transformer (DiT); a set of learnable latent queries extracts task-relevant semantic conditioning from the MLLM via cross-attention, source-video VAE latents are injected into the noisy target latents through a timestep-dependent residual connection, and the model is fine-tuned with LoRA (rank 64) on paired UI-overlaid/clean videos at 720p using a standard flow-matching objective.

Key results:

• Pilot study (5,442 clips, Wan2.1-T2V-1.3B fine-tuned): training on UI-free (clean) gameplay vs. UI-overlaid data improves overall VideoReward by 6.83%, motion quality by 18.8%, video quality by 2.7%, aesthetic quality (LAION) by 8.59%, and motion consistency (RAFT) by 4.36%.
• MLLM-as-a-judge evaluator validated against human annotations (200 videos, 6,150 UI-element labels): 87.78% removed F1, 85.96% artifact agreement, 78.00% preservation agreement, Cohen's κ = 0.820.
• On the Game2World-S synthetic benchmark, GameCleaner achieves AAR of 95.36, outperforming the strongest mask-free baseline (Kiwi-Edit) by 57.3%, with BG (background preservation) of 99.00.
• On the Game2World-W in-the-wild benchmark, GameCleaner (w. Ref) achieves best AAR of 80.05 with BG of 99.80, exceeding the strongest baseline (Aurora, AAR 51.19) by 28.86 percentage points; GameCleaner (w.o. Ref) achieves AAR of 51.60 with BG of 99.60.
• Overall score (avg of AAR+BG across both benchmarks): GameCleaner (w. Ref) = 93.34, beating mask-assisted VACE-1.3B (69.37) by 23.97 points and mask-free Kiwi-Edit (60.58) by 32.76 points.
• Text (introduction) reports slightly different overall comparison numbers: on synthetic subset GameCleaner scores 0.5697, beating mask-free baseline LoomVideo by 40.3% and trailing mask-assisted EffectErase by only 0.0233; on in-the-wild subset GameCleaner scores 0.4886, exceeding EffectErase by 23.7%.
• Inter-rater reliability of the judge: AAR ICC(A,5)=0.926, Krippendorff's α=0.712, Kendall's W=0.772; BG ICC(A,1)/ICC(A,5)=0.834/0.962.
• Ablations: reference-drop ratio of 20% during training gives the best balance (wild AAR improves from 70.64 at 0% drop to 80.05 at 20%; degrades to 56.23/54.64 at 50%/80% drop); scaling training data from 60% to 100% raises wild AAR by 25.09 points (54.96→80.05) vs. only 2.06 points on synthetic AAR, with no sign of saturation.

Why it matters / caveats: The results support treating gameplay UI removal as a scalable, generalizable data-processing step (rather than game-specific editing) that meaningfully improves the value of abundant Internet gameplay footage for world-model training. Caveats stated by the authors: the dataset does not yet cover the full diversity of games/interfaces (currently ~100K paired videos, being scaled toward 1M), GameCleaner can produce imperfect reconstructions for large opaque interfaces or rapidly changing overlays, the downstream validation uses text-conditioned video generation as a proxy rather than a fully action-conditioned world model (reliable action-recovery/inverse-dynamics models across games remain unavailable), and the current model requires the full generation trajectory rather than being distilled for efficient few-step inference.

Problem: Open video-language datasets remain far smaller and less accessible than open text and image-text corpora (the largest, InternVid, has only 7M videos), because collecting and processing web video at scale is bottlenecked by compute, engineering, and platform access restrictions rather than by video availability.

Method: The authors extract 1.3B platform-specific (YouTube, Vimeo, Dailymotion) video URLs from CommonCrawl WAT files using yt-dlp extractors, cc2dataset, and an Apache Spark cluster, then download 80M videos (10M hours) via a distributed pipeline of 2,000 virtual servers coordinated with Celery and residential proxies. From a 2.4M-video sample they extract 55M clips using PySceneDetect content-aware scene detection (threshold 30) plus motion-based filtering of static segments, and separately extract 300M scene-changing keyframes via ffmpeg scene detection (threshold 0.1). Clips/frames are captioned with modality-specific models: Qwen3-VL-2B-Instruct (via vLLM) for video captions, Audio Flamingo 3 for audio captions, and DeepSeek-VL2-tiny for frame recaptioning. Dataset quality is validated by training ViCLIP (video-text), CLAP (audio-text), and CLIP (image-text) at multiple model/data scales, with WiSE-FT checkpoint merging applied for some evaluations.

Key results:

• Released assets: 1.3B video URLs, 80M downloaded videos (10M hours, 60% download success rate), BVD-V-55M (55M captioned clips from 2.4M videos), BVD-I-300M (300M captioned frames).
• ViCLIP (L-14): BVD-V-50M at 50M samples seen reaches an overall average of 62.0 (top-1 classification + retrieval), beating InternVid-10M-FLT (58.0) by 4.0 percentage points; with WiSE-FT merging, BVD-V-50M improves to 62.6 vs. InternVid-10M-FLT's 60.2.
• CLAP: BVD-A-10M matches or exceeds LAION-Audio at matched samples seen across model scales (e.g., HTSAT-base-Roberta-large: 46.8 vs. 44.8 for LAION-Audio), though LAION-Audio+AudioSet (curated) remains best (56.6).
• CLIP on BVD-I-300M frames: strongest MS-COCO retrieval among compared datasets (ViT-B-16, 300M samples: T2I R@5 0.63, I2T R@5 0.80, beating Re-LAION and DataComp-1B) but weaker ImageNet-1k zero-shot accuracy (0.28 vs. 0.58 for DataComp-1B and 0.51 for Re-LAION at the same scale).
• FID between ReLAION and LAION-BVD frame embeddings is 33.92 (vs. 0.16 between two independent ReLAION samples), confirming LAION-BVD frames occupy a visual distribution distinct from standard web-image corpora.
• Scaling trends (power-law fits) show LAION-BVD scales especially steeply on COCO retrieval error rate but more weakly on ImageNet-1k classification error rate compared to DataComp-1B and Re-LAION.

Why it matters / caveats: LAION-BVD substantially expands open access to large-scale multimodal video/audio/image data for reproducible foundation-model research, but captions are intentionally short and synthetically generated (introducing possible model biases), the dataset underwent minimal filtering beyond platform moderation, and the validation experiments cover only contrastive (not generative) training and evaluate video, audio, and image modalities separately rather than jointly.

Problem: Fixed-canvas masked diffusion language models (dLLMs) must be given a target output length before denoising begins, but existing masked diffusion decoding research focuses on token unmasking order and leaves this target-length decision largely unexplored, even though a too-short or too-long canvas directly causes dropped content or redundancy.

Method: The paper introduces Entropy-Valley (EV), a training-free length selector for a frozen LoRA-SFT LLaDA-8B backbone: for a source sentence it builds a small set of candidate target-canvas lengths from a fixed direction-specific source-to-target ratio grid (e.g., {0.70,...,0.90} for En→Zh), runs one all-mask forward pass per candidate, scores each canvas by mean predictive entropy over all slots except the designated EOS slot, and selects the "entropy valley" (lowest-entropy) length before running standard minimum-entropy decoding (MED) for T=32 steps on that canvas. EV adds no trainable parameters, no length predictor, and does not use reference lengths or dev-set tuning — it only changes the target length supplied to the decoder, not the backbone, training loss, or unmasking schedule. It is evaluated against a fixed training-corpus ratio baseline and a reference-length "oracle" upper bound, and compared to a matched-data LLaMA-3-8B LoRA-SFT autoregressive baseline and other masked-diffusion backbones (Dream-Base, DiffuLLaMA).

Key results:

• On WMT22 with LLaDA-8B+LoRA (3 runs, N=2037/direction), EV raises COMET-22 over the fixed-ratio baseline by +0.0172 on En→Zh (83.45→85.17) and +0.0165 on Zh→En (82.66→84.31), closing 64.9±7.4% and 65.3±0.8% of the gap to the reference-length oracle, respectively; on En→De it gains +0.0070 COMET-22 (71.70→72.40), closing 33.0±8.4% of the gap.
• Corresponding sacreBLEU gains: +1.85 (En→Zh), +1.63 (Zh→En), +0.82 (En→De).
• Paired bootstrap and Wilcoxon tests confirm EV > Ratio significantly on En↔Zh; En→De sentence-level evidence is weaker.
• Placeholder retention on En→Zh rises from 66.9% to 89.1%, and number retention from 77.6% to 81.3% (EV vs. Ratio).
• Matched-data comparison: LLaDA+EV ties LLaMA-3-8B+LoRA-SFT on En→Zh (85.17 vs. 85.07) and beats it on Zh→En (84.31 vs. 83.68); on En→De the AR baseline leads by ~8.8 COMET points over EV-LLaDA (81.19 vs. 72.40).
• Cross-backbone check (Dream-Base, DiffuLLaMA, LLaDA): EV stays above Ratio in every tested direction/backbone, though gap-closure magnitude varies by model family.
• Controlled decomposition on En→Zh: fixing length to reference and varying reveal order spans only 0.0081 COMET-22 across schedules, versus 0.0265 COMET-22 spanned by three length choices (Ratio/EV/Oracle) with order fixed to MED — i.e., length source matters more than reveal order in this setting (EV−Ratio difference ≈2.1× the tested order span).
• Human evaluation (3 bilingual expert translators, 100 stratified WMT22 sentences per direction): EV improves adequacy by +0.50 on Zh→En (majority preference 45 EV / 25 Ratio / 30 tie) and +0.18 on En→Zh (28/19/53 tie-heavy); fluency changes are small in both directions.
• EV target length error remains far from the oracle's, yet still recovers ~65% of the oracle-Ratio COMET gap — showing EV is a "denoising-friendly canvas" selector, not a reference-length predictor.
• EV outperforms the best single fixed ratio in a sweep (Appendix), outperforms DAEDAL in both tested directions and CAL on En→Zh (comparable to CAL on Zh→En), with lower measured inference cost than CAL. Extra probe passes alone (given to the Ratio baseline as a control) recover at most 0.001 COMET, far below EV's gains.

Why it matters / caveats: The paper reframes fixed-canvas masked diffusion MT decoding as fundamentally a length-selection problem rather than only an unmasking-order problem, and shows a cheap, training-free entropy-based probe can recover a majority of the achievable quality gap without reference lengths. Caveats stated by the authors: En→De remains a clear boundary case where the matched AR system and even the LLaDA oracle outperform EV substantially, so the En→De shortfall is not primarily a canvas-selection error; EV's fixed candidate ratio grid limits it to canvases within a bounded compression range, causing failures on sentences needing compression outside that range; human evaluation was conducted only for En↔Zh, not En→De or De→Fr; and cross-backbone generalization findings (Dream-Base, DiffuLLaMA) are not causally attributed to any single architectural factor.

Problem: Industrial recommender systems built as cascaded retrieval/ranking/re-ranking pipelines suffer from information fragmentation, scattered optimization objectives, rigid rule-based strategies, and weak real-time intent awareness, leaving session-level shifts between browsing, comparison, and purchase unaddressed.

Method: DREAM (Developing Recommender Engine with Agentic Methods) adds a perception-aware, orchestrable, auditable policy layer on top of the existing pipeline rather than replacing it, built from two core components: (1) a three-tier Intent Engine that fuses on-device signals via a device-cloud F1–F4 Traffic Funnel into structured L0 (Physical)/L1 (Demand)/L2 (Preference) intent representations, using a synchronous 0.8B Main Agent plus asynchronous 4B specialized agents (context subagent and expert) with on-policy distillation self-evolution, and a nightly "Dreaming Mechanism" (4B model, six operations: keep/correct/enrich/merge/add/kill) for long-term consolidation; and (2) a Meta Engine whose MetaModel (built on Qwen3) performs layered M1 (intent summarization) → M2 (strategy planning, schema-constrained JSON informed by Strategy Memory) → M3 (deterministic parameter translation) reasoning, dispatching bounded parameters through a Unified Outlet ("default fallback + personalized override") to downstream stages (retrieval, ranking, re-ranking) protected by safety guardrails (schema validation, allowlists, range checks). Both engines are continuously optimized by a Reward Dual Loop combining an offline loop (production-path replay with a binary Evaluator-based reward comparing strategy vs. default pipeline, trained via a single-step contextual RL objective) and an online loop (live user feedback deposited into Strategy Memory).

Key results:

• Large-scale A/B testing on Taobao's Homepage Feed (Guess You Like): with re-ranking control alone, DREAM improves IPV by 2.06%, Core IPV by 2.39%, GMV by 0.88%, PV by 1.03%, PCTR by 0.76%, Click UV by 0.54%, UCTR by 0.53%, Ad Cost by 0.21%.
• Extending control to fine ranking increases gains to IPV +2.71%, Core IPV +3.06%, GMV +1.31%, PV +1.04%, PCTR +1.25%, Click UV +0.68%, UCTR +0.81%, Ad Cost +0.02%.
• Intent Engine downstream A/B tests (platform-wide): Cognitive Recommendation (recall) improves IPV +0.80%, Core IPV +0.91%, PCTR +0.84%, Transaction Volume +0.36%; Recommendation Strategy Adaptation improves Core IPV +0.70%, IPV +0.52%, Transaction Volume +0.33%, with neutral PCTR (-0.02%).
• In-scenario (Heuristic Recommendation): recall integration lifts Inquiry Card Clicks +7.17% and CTR +2.00%; Explicit Text Interaction (copy optimization) lifts Inquiry Card Clicks +10.64% and CTR +3.01%.
• Multi-Scale Intent Model ablation (LLM-as-a-Judge overall score / search-term recall / filter recall): Baseline 71.32% / 50.57% / 24.25%; +Routing 78.20% / 51.68% / 26.79%; +Routing+Self-Evolution 84.74% / 56.05% / 26.40%.
• On-device Traffic Funnel expands usable behavior vocabulary from 6 to 60+ types and admits only ~15% of behavior for cloud upload at the F2 gate (abstract cites ~8.7% overall reporting-volume reduction); ~6.3% of requests are escalated to the asynchronous 4B tier.
• Dreaming Mechanism evaluation on 683 matched users shows consistent improvement across all evaluated intent fields vs. the daytime-only state, with the largest gains in Intent Type (0.583→0.680) and Priority (0.524→0.571).

Why it matters / caveats: DREAM demonstrates that an agentic meta-control layer can be added on top of an unchanged production retrieval-ranking-re-ranking pipeline and yield compounding, statistically validated gains as its control surface widens from re-ranking to fine ranking, without replacing existing models or compromising serving stability. Caveats stated in the paper: offline RL training uses a single-step, list-level binary reward (no simulation of downstream clicks/transactions/state transitions), replay treatments are separate service calls so repeated averaging reduces but cannot eliminate serving noise, and absolute metric values are omitted for confidentiality (only relative lifts reported).

Problem: Existing self-improving LLM agent systems either fix their meta-level and never revise the improver itself, or let the agent edit its own source but must freeze some driver component to stay stable, which caps realized meta-depth at roughly 2–2.5 and prevents true recursive self-improvement.

Method: Meta^n holds a single fixed, LLM-prompted meta-operation Ω constant and recurses on its own accumulated outputs instead of rewriting Ω itself: at each depth d, Ω reads the previous depth's execution traces together with the full stack of code it previously emitted, and writes a new layer C_d consisting of a strategic pre-process function and a library of callable helper functions, which a wrapper M_d composes around the existing solver stack (S_d = M_d ∘ M_{d-1} ∘ ... ∘ M_2 ∘ S_1). Depth is not fixed in advance but grows until Ω stops finding improvements (convergence-based, via linear greedy deepening or an evolutionary archive search over layer chains, with archive selection weighted by score plus an exploration bonus and optional cross-candidate inspiration). A "consolidation" mode is also introduced that inherits frozen best traces per task to guarantee monotonic (non-regressing) per-task improvement. The framework is tested with a single Ω prompt template across all benchmarks, on top of either a single-shot LLM solver or an 8-turn agentic observe/act solver.

Key results:

• Across 8 benchmark families (CO-Bench, Symptom2Disease, LawBench, AlphaEvolve Math, AlgoTune, Symbolic Regression, TerminalBench 2.0, ARC-AGI-2) and two backbones (Gemma 4 31B-IT, GPT-5.2), Meta^n outperforms Gödel Agent and OpenEvolve on every benchmark family on at least one estimator (archive-best or best-chain).
• ARC-AGI-2 (GPT-5.2): Meta^n reaches 0.331 while OpenEvolve scores 0.003 and Gödel Agent scores 0.054 — Meta^n is the only system to solve any held-out task at all.
• CO-Bench (GPT-5.2): Meta^n archive-best 0.870 ± 0.011 vs. OpenEvolve 0.702 ± 0.025 (+0.168 margin) and Gödel Agent 0.527 ± 0.033; even after boosting Gödel Agent's budget 10×, it plateaus at 0.628.
• AlphaEvolve Math (GPT-5.2): Meta^n 0.917 ± 0.016 vs. OpenEvolve 0.726 ± 0.046 vs. Gödel Agent 0.674 ± 0.068.
• AlgoTune (Gemma): Meta^n archive-best ×15.10 ± 2.4 speedup vs. OpenEvolve ×10.45 ± 1.8 vs. Gödel Agent ×13.22 ± 2.7.
• LawBench: archive-best beats Gödel Agent by +0.040 and OpenEvolve by +0.070; at compute parity with OpenEvolve's token budget, Meta^n still leads (0.784 vs. 0.745).
• Ablation (isolation of recursion): removing recursion (collapsing to depth-1) drops CO-Bench archive-best from 0.845 to 0.714 on Gemma (−0.131), from 0.886 to 0.806 on GPT-5.2 CO-Bench (−0.080), and from 0.917 to 0.759 on GPT-5.2 AlphaEvolve Math (−0.158).
• Component ablation: removing inter-layer context ("conditioning") accounts for ~72% of recursion's gain; removing the code-library injection accounts for ~15%; the remaining ~13% comes from the recursion/search machinery itself (beam, retry, inspiration).
• Meta^n is more sample-efficient than OpenEvolve on CO-Bench, matching or exceeding it using ~13× fewer candidate evaluations (29 vs. 378).
• Emergent role analysis (two independent LLM raters, Cohen's κ = 0.59 average): rollback/correction behavior is exactly 0% at depth 2 but appears at depth 3 (55% on code substrates, 33% on prompt substrates); tactical primitives peak at depth 3 (45%) and decay to 17% by depth 5 as specialized libraries take over — role differentiation emerges without being prescribed by the prompt.
• Consolidation mode (zero-regression guarantee): on an 8-task CO-Bench band, lifts per-task-best from 0.502 to 0.71 ± 0.02, beating a compute-matched best-of-4 control by +0.10 (95% CI [+0.04, +0.16]).

Why it matters / caveats: The paper argues this is the first system to demonstrate that meta-depth beyond two produces structurally distinct, useful layers rather than redundant ones, resolving a stability-vs-depth tradeoff that capped prior self-modifying agents. Caveats stated by the authors: the same model is used as both base solver and Ω at every layer, so the practical case of a stronger model driving a weaker base solver remains untested; the framework underperforms or gives negligible gains on benchmarks with little headroom above the seed solver or few distinct failure modes (e.g., AlgoTune, where extra context over-constrains an already near-optimal kernel, and SWE-Bench, where Ω never activates because the seed is already strong); and observed recursion depth in practice stops between depth 3 and 6 because Ω stops finding improvements, not due to any known capacity limit.

Problem: Outcome-supervised search agents learn when and how to retrieve evidence from terminal rewards alone, so intermediate errors in a long-horizon trajectory go unlocalized and uncorrected before they compound into failure.

Method: CAFE (Coupled Agent–Feedback Evolution) uses a single shared-parameter model that alternates between an agent role (issuing search actions and an optional <request_feedback> action) and a critic role (generating in-context corrective feedback), initialized via SFT on recovery trajectories built by preserving the base agent's own failed prefixes, inserting feedback, and appending a teacher-generated (Kimi-K2.5) successful continuation. During online RL (GRPO-based), a Comparative Feedback Estimate (CFE) shapes the reward using the prompt-level success gap between rollouts that call vs. skip feedback, combined with feedback-aware advantage shaping that reweights token-level advantages differently before and after a feedback request. Offline, Rollout-Derived Preference Optimization (RDPO) mines prefix-matched successful/unsuccessful rollout pairs from the latest on-policy trajectories and applies DPO to update the critic, with online agent updates and offline RDPO critic updates alternated iteratively (default schedule: 100 RL steps × 5 iterations).

Key results:

• On 7 agentic SearchQA benchmarks (2WikiMultihopQA, HotpotQA, MuSiQue, PopQA, TriviaQA, Bamboogle, NQ) with Qwen2.5-7B-Instruct backbone, CAFE reaches average EM 52.5 / F1 60.7, beating the strongest RL-based baseline IGPO by 2.1 EM and 1.3 F1.
• Staged gains: base model 38.1/47.9 EM/F1 → +SFT 40.8/50.1 → +SFT+GRPO 49.7/58.0 → +SFT+CAFE 52.5/60.7.
• CAFE improves over GRPO on every benchmark, including all 6 out-of-domain datasets; vs. Search-R1, average gains are 7.4 EM/5.9 F1 on four multi-hop benchmarks vs. 1.3 EM/1.3 F1 on three single-hop benchmarks.
• Online ablation: CFE alone raises EM/F1 from 49.7/58.0 to 50.8/58.4; advantage shaping alone reaches 51.2/59.4; combined reaches 51.9/60.3.
• Offline ablation (5-iteration schedule): RDPO consistently outperforms a rollout-derived SFT (RSFT) baseline that uses only positive (successful) feedback examples; full CFE+advantage-shaping+RDPO combination reaches 52.5/60.7.
• Co-evolution analysis on 2Wiki (mean of EM/F1): feedback-only optimization improves 67.7→71.3; agent-only optimization peaks at 84.2 then declines to 83.6; alternating (CAFE) reaches 86.6, beating the final agent-only checkpoint by 3.0 points — one-sided updates plateau or regress, alternating continues improving.
• Agent–critic cross-play (2Wiki) shows each agent performs best paired with the critic from its own training iteration; pairing the final agent with the iteration-5 critic vs. the SFT critic raises EM 80.6→84.0 and F1 86.6→89.2.
• Average answer-level hallucination rate: 29.9% (base) → 17.6% (GRPO) → 12.6% (CAFE), with largest CAFE-vs-GRPO reductions on NQ (10.8 pts) and MuSiQue (9.4 pts).
• Results also hold at 3B scale (comparable to several 7B baselines) and on the harder BrowseComp-Plus deep-research benchmark, where CAFE achieves the best result among training stages tested.

Why it matters / caveats: The paper's central empirical claim is that self-improvement in search agents requires jointly evolving both the policy and the feedback/critic mechanism — one-sided optimization (agent-only or critic-only) plateaus or regresses, while alternating online RL and offline preference optimization on a shared model sustains improvement and reduces hallucinations; results are demonstrated only on Qwen2.5-7B/3B-Instruct backbones and rely on a proprietary teacher model (Kimi-K2.5) for SFT data construction.

Problem: Dense transformer decoders in video-language models share the same feed-forward networks across heterogeneous procedural tasks (action recognition, forecasting, procedure prediction), entangling task behavior, while existing sparse Mixture-of-Experts decoders route at the token level, which is not naturally aligned with task-level procedural objectives.

Method: MoTE (Mixture of Task Experts) converts each LLM decoder feed-forward network (FFN) into a set of task-specific expert FFNs plus an always-active shared FFN expert, initialized by copying the pretrained dense FFN (dense-to-expert "sparse upcycling" initialization), while keeping the visual encoder, MLP projector, and attention path (GQA) shared across tasks. Each video–prompt sample is routed as a whole (sample-level routing) to one task expert per decoder layer, either via explicit task-index supervision during training or, at inference, via prompt-conditioned selection that matches the prompt embedding to registered natural-language expert descriptions using a frozen text encoder (MiniLM-L6-v2) and cosine similarity. The architecture is instantiated as VideoLLM-MoTE (SigLIP2 ViT-L/16-384 encoder + Llama-3.2-1B-Instruct decoder, LoRA fine-tuning) trained on five COIN task routes with the VideoLLM-online-style LM+streaming-silence loss, and supports modular expert addition (freeze old modules, train only new expert) and expert removal for capability withdrawal.

Key results:

• On five COIN benchmarks (Step, Next, Proc., Task, Proc.+), VideoLLM-MoTE-1B+5E achieves 62.9% average top-1 accuracy, the highest among all compared methods, versus 61.8% for the strongest 8B VideoLLM baseline (VideoLLM-online-8B-v1+ / VideoLLM-MoD) and 59.3%/59.9% for matched-scale VideoLLM-online-1B/3B-v1+ baselines.
• Efficiency: VideoLLM-MoTE-1B+5E activates only ~2B LLM parameters per sample (5.3B total) vs. 8B active/8B total for the 8B baselines, cutting TFLOPs from 98.7 to 28.9 and latency from 2.8s to 1.8s while raising decoding throughput from 27.8 to 37.7 tokens/s (vs. VideoLLM-online-8B-v1+).
• Ablation on routing granularity (same 5-expert topology): MoTE (task-index routing) = 62.9% avg vs. 61.5% for dense all-expert activation, 61.4% for learned token-level top-1 routing, and 62.1% for learned sample-level top-1 routing.
• Ablation on expert construction: full-FFN-copy experts (default) reach 62.9% avg vs. 60.3% for FFN-splitting into disjoint neuron groups (LLaMA-MoE style, on an 8B backbone).
• Modular expert addition: sequential single-task-at-a-time training reaches 61.0% average (vs. 62.9% for joint training), with earlier task accuracies unchanged across later addition/removal stages, demonstrating no forgetting.
• Cross-dataset extension: VideoLLM-MoTE-1B+6E (Ego4D narration expert + 5 COIN experts in one checkpoint) achieves the highest Fluency (51.0) on Ego4D narration validation among compared models while remaining competitive on COIN tasks, though PPL (2.57) and TimeDiff (2.25) trail Ego4D-only specialized baselines (PPL 2.40–2.43, TimeDiff 2.04–2.25).
• Cross-domain (document understanding): converting GLM-OCR-0.9B into GLM-OCR-MoTE-0.9B+2E raises SROIE KIE Micro-F1 from 55.04% to 87.90% and CORD KIE Micro-F1 from 20.07% to 95.79%, while perfectly preserving the original OCR route's performance on OmniDocBench v1.5.
• Prompt-conditioned route selection: a 22.7M-parameter MiniLM-L6-v2 selector reaches 100% test F1 across the five predefined COIN intents (50 prompt variants each), with 4.35 ms selection latency per prompt and 1.5 KiB per cached expert-description embedding.

Why it matters / caveats: Task-structured sample-level routing gives an interpretable, compute-efficient, and modular alternative to dense decoders and learned token-level MoE routing for multi-task video-language models, and the same FFN-expertization principle transfers to a non-video domain (document KIE/OCR). The authors note MoTE works best when task boundaries are meaningful and stable — the prompt-selector evaluation does not cover ambiguous, overlapping, or compositional requests, and freezing old expert routes during incremental addition trades away plasticity (61.0% vs. 62.9% average) to preserve prior task behavior.

Problem: World action models (WAMs) that predict future observations to guide robot action generation improve generalization but incur substantial test-time latency, while efficient variants that drop future imagination at inference (e.g., Fast-WAM) generalize markedly worse, especially with scarce demonstrations and out-of-distribution scenarios.

Method: LAWA reframes future imagination as prediction of compact, discrete latent actions (rather than future video) that serve as an operational "intention" representation guiding an action expert. A ViPRA-style latent action tokenizer (DINOv2 encoder + non-causal spatial/temporal transformer + vector quantization against a learnable codebook) is pre-trained on action-free robot and egocentric video with a causal forward decoder for next-frame reconstruction and an auxiliary SAM-2-supervised mask-prediction loss to bias codes toward hand/manipulator interaction regions. LAWA then jointly trains video, latent-action, and action denoising experts via flow matching under a "Multi-model Joint Attention" scheme with a structured attention mask (current observation stays clean/uncorrupted, future-video tokens see everything, latent-action tokens see current observation + latent sequence, action tokens see current observation + latent sequence + action sequence); at inference the future-video branch is discarded and only the latent-action and action experts are jointly denoised from noise, with the current-observation encoding cached.

Key results:

• RoboCasa (24 tasks): LAWA reaches 65.6% success (few-shot, 10% data) and 80.8% (full data), beating matched Fast-WAM by 9.6 and 4.5 points respectively, and matching matched Joint-WAM (64.1%/78.8%); also beats DIAL by 7.3 points in few-shot and outperforms most full-data VLA baselines (e.g., JoyAI-RA 63.2%, RLDX-1 58.7%).
• LIBERO-Plus (zero-shot transfer): LAWA gets 74.4% overall, vs. matched Fast-WAM 60.0% (+14.4 pts), Joint-WAM 70.4% (+4.0 pts), and OpenVLA-OFT 69.6% (+4.8 pts); largest gains under camera-viewpoint (+44.3 pts) and sensor-noise (+27.5 pts) shifts.
• Inference latency: LAWA = 338.5 ms/action-chunk vs. Joint-WAM 593.1 ms (42.9% lower) and Fast-WAM 196.5 ms (faster but much lower accuracy).
• Latent-action perturbation ablation: adding Gaussian noise to the latent state drops full-data RoboCasa success from 80.8% to 52.2%; temporal shuffling drops it to 56.4%, showing the executor functionally depends on the latent sequence's content and temporal structure.
• Egocentric pre-training scaling: increasing action-free video fraction from 10%→100% raises LAWA from 77.2%→80.8% (full-data) and 61.6%→65.6% (few-shot), versus only ~1.0–1.1 point gains for Fast-WAM under the same schedule.
• Component ablation (RoboCasa): Latent Action alone: 59.7%/76.3% (few-shot/full) vs. Fast-WAM baseline 54.5%/74.6%; + Egocentric Pre-training: 64.8%/79.3%; + Mask auxiliary loss: 65.6%/80.8% (best); flow-based auxiliary loss instead of mask underperforms (63.5%/78.6%).
• Real-world tasks (xArm7, 4 tasks: Gear, Battery, Block, Laboratory): LAWA beats Fast-WAM by 31.2/36.3/33.8 points at 25%/50%/100% of demonstrations respectively; with only 25% of data LAWA (40.0% avg) surpasses Fast-WAM's full-data result (33.8% avg); on long-horizon tasks (Block, Laboratory) Fast-WAM scores 0% in the 25%-data setting while LAWA reaches 45.0% and 30.0%.

Why it matters / caveats: The results suggest future imagination need not be discarded for efficiency — representing it as compact latent actions instead of full future observations offers a favorable trade-off among performance, generalization, and inference latency. A caveat noted by the authors: without egocentric pre-training, LAWA's latent-action approach alone does not yet match Joint-WAM (trails by 3.4–2.0 points), so its advantage depends on the scalable action-free video pre-training pipeline; also, code and models are stated as "to be released" (not yet available at time of writing).

Problem: The de-facto Python reference for morphological transforms, scipy.ndimage, is CPU-only, single-array, and limited in dimensionality, making it unusable inside a GPU training loop without expensive device-to-host round trips, and existing GPU vision libraries (Kornia, cuCIM, MONAI, OpenCV, scikit-image) only cover a narrow subset of these operators (typically 2-D, flat structuring elements, no exact EDT or optimal transport).

Method: TorchMorph is a lightweight PyTorch extension exposing 22 public operators across four families (binary morphology, greyscale morphology, exact/approximate distance transforms, and entropy-regularised optimal transport via Sinkhorn), implemented as fused CUDA kernels operating directly on (B, C, Spatial...) CUDA tensors with up to eight spatial dimensions. Its API mirrors scipy.ndimage argument-for-argument (size, footprint, structure, mode, cval, origin, iteration semantics), and the architecture is layered into a Python validation/normalisation layer, a pybind11 binding layer with 8 kernel entry points, and a CUDA kernel layer (6 translation units) implementing a templated fused erosion/dilation kernel, a separable lower-envelope exact Euclidean distance transform (per-scanline thread blocks), dimension-separable chamfer sweeps, a brute-force distance oracle, and a batch-tiled Sinkhorn solver (scaling and log-domain, with CUDA-graph replay for iteration chunks and custom autograd via centered dual potentials).

Key results:

• Numerical agreement: binary morphology and chamfer distance transforms match scipy.ndimage exactly (0 error); all float-valued operators have worst-case absolute error ≤ 1.8×10⁻⁶ and relative ℓ2 error ≤ 9×10⁻⁸ against scipy.ndimage/POT references (Table 2).
• Throughput vs. single-threaded scipy.ndimage: up to ~1.1×10³× on greyscale morphology (e.g., grey dilation on 256² images: 111.11 inputs/ms at B=8 vs. 0.756 inputs/ms for SciPy) and up to 350× on exact Euclidean distance transforms.
• Batching gains: grey dilation improves 10.31→111.11 inputs/ms from B=1 to B=8 (10.8× gain); grey erosion improves 16.13→83.33 inputs/ms (5.2× gain); binary morphology reaches 36–56 inputs/ms at B=8.
• Batching benefit diminishes for heavier transforms: EDT on 1024² images improves only 2.11→2.86 inputs/ms (1.4×) across batch sizes; EDT on 128³ volumes stays flat at ~1.4 inputs/ms regardless of batch size.
• Sinkhorn solver vs. POT on 32×32-grid problems (Table 4): 18.8× speed-up (scaling, 100 iterations, 23.0ms→1.2ms), 22.8× (scaling, 1000 iterations, using CUDA-graph replay), 11.8× (log-domain, 200 iterations), and up to 42.4× (batched n=16, 100 iterations, 367.3ms→8.7ms); relative transport-plan errors range from 4.45×10⁻⁷ to 1.54×10⁻³.
• Validated by 78 test functions across five modules (oracle agreement, cross-implementation consistency, contract tests, runtime/stream/device tests), run on every push/PR via self-hosted CI with a physical CUDA device.

Why it matters / caveats: By matching scipy.ndimage semantics exactly while running as fused, batched CUDA kernels, TorchMorph lets morphological/distance-transform/optimal-transport operators sit inside a GPU training loop rather than as an offline CPU preprocessing step; the authors frame its contribution as availability rather than algorithmic novelty. Stated limitations: morphology and distance-transform kernels are forward-only (not differentiable, though erosion/dilation admit subgradients), only the transport module is differentiable and falls back to pure torch on CPU (morphology/distance kernels require CUDA with SciPy as the intended fallback), computation is in float32 under fast-math with NaN propagation not guaranteed to match the reference, and reported speed-ups compare a GPU against a single CPU core rather than a well-parallelised CPU baseline.