Ground Truth.
AI, checked against the source.

AI papers — 2026-08-06

Every paper on Hugging Face's Daily Papers list, most-upvoted first — each linked to arXiv, with a plain-English summary and a technical breakdown underneath.Both are AI-written — the summary from the authors' abstract, the breakdown from the paper's full text — and are not individually fact-checked by us. The paper itself is the source.
← 2026-08-052026-08-062026-08-07 →
Jump to one of 27 papers
  1. ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment
  2. ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
  3. The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads
  4. Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes
  5. OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents
  6. GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks
  7. Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
  8. When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation
  9. NOLLI: A Difficulty-Calibrated Puzzle Benchmark for Diagnosing the English-Korean Performance Gap
  10. Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
  11. AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities
  12. When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
  13. Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance
  14. FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory
  15. K-EXAONE 2.0 Technical Report
  16. HelloWorld: Enabling Socially Interactive Characters in Video World Models
  17. WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models
  18. OPD-V: Visual On-Policy Self-Distillation with Modality Balance
  19. BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
  20. Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models
  21. UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models
  22. SKILL-KD: Contrastive Skill Distillation for LLM Agents
  23. Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming
  24. TriGlue: a Biology-Inspired Generative Model for Generating Molecular Glue-Induced Ternary Complex
  25. Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
  26. DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack
  27. Self-Evolving Coding Agents

ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment →

arXiv 2608.05102 · ▲ 49 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing training methods for long-horizon search agents apply uniform trajectory-level supervision (SFT and RL) to every step, so they cannot distinguish useful actions from erroneous or redundant ones within a trajectory.

Method: The paper proposes Answer-Backtracked Credit Assignment (ABC), which first runs Answer-Backtracked Clue Recovery — prompting an LLM (DeepSeek-V4-Flash) in a ReAct loop to trace back from a verified answer and recover a set of intermediate evidence clues — and then applies Clue-Anchored Step Scoring, a rubric-based scorer (base score 1.0, deltas for discovering/verifying clues, ruling out candidates, dismissing correct clues, or submitting correct/incorrect answers, clipped to [0, 2.0]) that converts each trajectory step into a dense reward. These step rewards drive ABC-SFT (loss reweighting via a sigmoid function of the reward) and ABC-GRPO (step-level, discounted advantages fed into a clipped GRPO objective), used to train ABSeeker, a Qwen3.5-4B-based agent, on 8.5K trajectories from OpenSeeker.

Key results:

  • ABSeeker (4B) scores 37.3% on BrowseComp and 39.1% on BrowseComp-ZH without context management; with context management (256K tokens, discard-all strategy up to 5 rounds) these rise to 55.3% and 52.9%, respectively.
  • It also reaches 77.0% on xbench-2505, 46.0% on xbench-2510, and 81.6% on GAIA-text, the best among all reported 4B search agents on every benchmark (vs. QUEST-4B 40.0% BrowseComp, DR-Venus 29.1%/37.7% BrowseComp/-ZH, AgentCPM-Explore 24.1%/29.1%).
  • ABSeeker's GAIA-text (81.6%) and xbench-2505 (77.0%) scores exceed all reported ~30B search agents (e.g., Tongyi-DeepResearch 70.9%/75.0%, OpenSeeker –/74.0%, MiroThinker-1.7-mini 80.3%/–).
  • Ablations: ABC-SFT improves over standard SFT (e.g., GAIA-text 72.8% vs. 66.0%), and ABC-GRPO improves over standard GRPO on every benchmark tested (e.g., BrowseComp 37.3% vs. 33.5%, GAIA-text 81.6% vs. 77.7%).
  • Reward-distribution analysis: ~4% of steps in successful trajectories score below 1.0 (low quality) and ~10% of steps in failed trajectories score above 1.0 (useful despite ultimate failure), motivating step-level over trajectory-level supervision.

Why it matters / caveats: The approach lets a compact 4B model match or beat several ~30B search agents by extracting denser training signal from the same trajectories, and it generalizes to xbench/GAIA despite training only on BrowseComp-style data. The authors note the method has only been validated at 4B scale due to compute constraints and has not yet been extended beyond web search to other long-horizon agent tasks.

ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation →

arXiv 2608.04436 · ▲ 44 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing agentic image-generation systems only place part of the process (search/prompt-optimization or a fixed pipeline) under agent control, so reasoning, tool invocation, and image generation are not coordinated by a single policy, limiting open-world image generation that requires external knowledge.

Method: ToolArtist post-trains Emu3.5, a Unified Multimodal Model (UMM), so that a single policy autonomously chooses among TextSearch, ImageSearch, and native image generation within one ReAct-style multimodal sequence. In SFT, a teacher agent (using an external image generator, gemini-3-pro-image-preview) produces 7,132 multi-turn trajectories that are then converted into UMM-native format by hiding the external image-generation tool call and replacing it with a native visual-caption span plus visual-token span; the SFT loss masks conditioning input and tool observations. In RL, the paper introduces Reason-Act-Draw GRPO (RAD-GRPO), a token-normalized clipped-GRPO objective combining an Intent Reward (checks if the final caption sufficiently captures user request + evidence) and a Quality Reward (a weighted world-knowledge reward model over faithfulness, visual correctness, text accuracy, and aesthetics, weights 0.1/0.4/0.4/0.1), plus auxiliary format, draw-signal, length-penalty, and no-draw-penalty terms.

Key results:

  • On WISE, ToolArtist reaches 0.79 overall, the best score among agentic image-generation models compared (Unify-Agent 0.77, GenSearcher-Qwen-Image 0.77), with 0.81 on Physics and 0.79 on Chemistry.
  • On WorldGenBench-Humanities, ToolArtist achieves the best non-proprietary average Knowledge Checklist Score (KCS) of 22.10, versus 21.76 for the strongest open-source baseline mentioned (Qwen-Image), 15.58 for Unify-Agent, and 13.66 for GenSearcher-Qwen-Image.
  • Ablation of source-aware image-search summaries drops WISE Overall from 0.79 to 0.61, with the largest single-category drop in Biology (0.75 → 0.25, a 0.50 decrease).
  • Case study (Socotra resin-collection prompt): ToolArtist's RL checkpoint scores KCS 0.625 (5/8 checklist points satisfied) versus 0.000 for Unify-Agent and 0.250 for GenSearcher.
  • RAD-GRPO training dynamics show overall reward rising from an initial dip to roughly 0.41 by the end of ~60 training steps, while policy entropy declines gradually from ~12.03 to ~11.87–11.91 without collapse.

Why it matters / caveats: Making tool use and image generation actions of the same policy (rather than delegating generation to an external model) yields better open-world knowledge grounding than fixed pipelines or prompt-optimization agents, using a relatively small (7K-trajectory) SFT set. The frontier proprietary models (Nano Banana / Nano Banana-Pro) still outperform ToolArtist on WISE overall (0.87–0.89) and on several WorldGenBench continents, so ToolArtist's gains are specifically over other open, agentic, and unified-multimodal baselines rather than over closed frontier systems.

The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads →

arXiv 2608.04570 · ▲ 35 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Personalized LLMs with persistent memory are assumed to reliably distinguish what they know about a user from what they are guessing, but this faithfulness of user modeling has not been systematically measured, and it is unclear whether models' own self-reported confidence can be trusted as a safety signal.

Method: The paper introduces MirageBench, pairing 150 user personas (balanced across stereotypical, counter-stereotypical, and neutral profiles, each with a 15-attribute ground-truth profile P and exactly 3 revealed facts E) with 6 open-ended personalization tasks (dating profile, weekend itinerary, recommendation letter, birthday gift, apartment description, stress source) spanning a groundability gradient. A four-way faithfulness taxonomy (Grounded / Reasonable / Stereotype / Fabricated) is applied via a three-stage-plus-longitudinal evaluation pipeline (Probe, Task, Judge, Accum), where an independent judge model (Claude-Opus-4-7) classifies every claim, validated against a blind human annotator on 400 claims. Over-inference (OI) rate is defined as (#Stereotype + #Fabricated) / #Total Claims, and is computed for 12 models across 7 families over 143,616 judged claims, alongside a self-monitoring gap comparing each model's own Task-stage self-audit OI to the Judge's OI.

Key results:

  • Every one of the 12 tested models over-infers between 35% and 49% of its personalized claims (cross-model macro-average OI 41.6%; claim-weighted micro-average 41.8%), with Qwen3-8B highest at 48.7% and Claude-Opus-4-6 / Gemini-3.1-pro lowest at 35.4% / 35.1%.
  • The Self-Monitoring Inversion: across the 12 models, self-reported OI is negatively rank-correlated with judge-measured OI (Spearman ρ = −0.60, p = 0.044, bootstrap CI [−0.90, +0.06]) — e.g., Qwen3-8B self-reports lowest OI (13.0%) but is judged highest (48.7%), while Kimi-K2.5 self-reports highest (58.2%) but is judged mid-pack (43.1%).
  • Within a single model, self-audit still ranks that model's own claims moderately to strongly (AUROC 0.58 for Qwen3-8B to 0.83 for Qwen3.6-plus; 9 of 12 models exceed 0.75).
  • OI is task-dependent, ranging from 27.0% (birthday gift) to 57.8% (apartment/home description), following a groundability gradient across the six tasks.
  • Judge/human agreement on the four-way taxonomy is 89.8% (Cohen's κ = 0.863), rising to 95.0% (κ = 0.900) on the binary over-inference decision, across 400 stratified claims.
  • In a multi-turn pilot (8 rounds, 2 personas), 9 of 12 models show approximately linear growth in inferred attributes (R² > 0.90, 5–15 new attributes per round), with the fastest accumulators (GPT-5.5, GLM-5.1, Claude-Opus-4-6, Qwen3.6-plus, DeepSeek-v4-flash) showing only 0.4–5% removal of prior inferences, versus 70–82% removal for the two exceptions (Qwen3-8B, GPT-5.4-nano).
  • Stereotypical personas incur 44.8% OI versus 37.0% for counter-stereotypical personas (7.8 percentage-point gap, consistent across all 12 models, per-model gaps +4.0 to +10.5 pp).

Why it matters / caveats: The findings imply that no tested model is safe from fabricating user attributes, and that choosing or trusting a personalization model based on its own self-reported confidence is actively misleading at the cross-model level, even though within-model self-audit remains useful for internal filtering — arguing for external verification and provenance tracking over self-report. The authors flag that the multi-turn accumulation pilot uses only 2 personas and a memory prompt biased toward retention, so it should be read as suggestive rather than definitive, and the OI metric relies on a single judge model (cross-judge robustness left to future work).

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes →

arXiv 2608.05000 · ▲ 32 on Hugging Face · HF page · PDF

Technical breakdown

Problem: The design space of unified multimodal pretraining (jointly modeling language, visual understanding, and visual generation in one model) is largely navigated by heuristics, with the mechanisms governing cross-modal knowledge transfer, synergy versus competition, and optimal training timing remaining empirically underexplored.

Method: The authors pretrain Llama-3-style decoder-only Transformers (1.5B/2.3B parameters, GQA, SwiGLU, RoPE, QK-norm) using the Transfusion framework, which unifies discrete next-token prediction for text with continuous flow matching (rectified flow, x-prediction, 25-step Euler sampling) for images, across four visual tokenization schemes (RAE with a frozen SigLIP-2 encoder, Raw Pixels, CLIP+VAE, and an autoregressive UniTok variant). They run controlled mixture experiments on real-world data (DCLM for language, ~350M Shutterstock image-text pairs) and a procedurally generated synthetic CLEVR-based testbed with concept ablation to isolate knowledge flow, ablate Transformer components (FFN, attention, normalization sharing/splitting) to study modality synergy versus competition, and vary unification timing/sequencing to study early-vs-late fusion, ultimately deriving an asymmetric data-mixing recipe validated on 13.5B-parameter MoE models (256 experts, top-16 routing, 2 modality-fixed experts) trained on 2T tokens.

Key results:

  • Knowledge flow is asymmetric: increasing language ratio from 0% to 80% monotonically improves all visual understanding and generation metrics; visual understanding strongly benefits generation (e.g., diffusion loss and GenEval improve as understanding-token ratio rises), but visual generation gives near-neutral backward transfer to language/understanding.
  • On synthetic CLEVR, low-level concepts (color, shape) show zero-shot transfer failure in both directions, but fine-tuning recovery shows generation-learned priors accelerate understanding acquisition by +0.133 to +0.273 mean accuracy, while understanding-learned priors give ~0 to +0.221 boost to generation.
  • Architecture ablations: fully shared "dense" parameters degrade language PPL by +0.272 and vision diffusion loss by +0.0537 versus unimodal baselines; splitting only FFNs (split_ffn) yields the best synergy, improving language PPL by -0.211 and vision loss by -0.0168, while fully isolating everything (split_all) reverts to baseline performance.
  • Data mixing grid search finds a 70% language / 25% understanding / 5% generation (L70/U25/G5) split near-optimal, achieving GenEval 0.482 and DPG 0.689 with generation using only 5% of visual training data — a "5% of the compute budget" recipe that still slightly beats a balanced 50/25/25 recipe on FID (5.234 vs. 5.131 in the reverse comparison).
  • At 13.5B-MoE/2T-token scale, the asymmetric recipe outperforms a balanced-mix baseline on language accuracy (54.31% vs 52.86%) and visual understanding average (43.08% vs 41.42%); the MoE architecture beats a 3.5B dense model on language accuracy (52.86% vs 52.03%) and vision diffusion loss (0.261 vs 0.266); early unification beats late-fusion on language accuracy (52.86% vs 51.78%) and diffusion loss (0.261 vs 0.269).

Why it matters / caveats: The findings offer concrete architectural recipes (shared attention/normalization with modality-specific FFNs, early joint training, asymmetric data mixing) that let unified multimodal models achieve strong generative performance while spending most compute on language and understanding data, reducing the cost of adding native generation. The study is confined to static text-image pretraining at up to 13.5B/2T-token scale; the authors note that video/audio modalities and >1T-parameter frontier scales remain unvalidated extrapolations.

OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents →

arXiv 2608.05013 · ▲ 24 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Long-horizon, cross-environment, multimodal "everyday" agent tasks cause LLM agents to suffer from compounding failure modes — goal drift, context/state loss, and context-window overflow — that prior work addresses individually but rarely jointly within a single backend-agnostic harness.

Method: The paper introduces OneDayAgent, a harness built around three coupled capabilities layered on a ReAct execution loop: (1) task decomposition by a Planner that breaks a request into an ordered set of bounded subtasks while preserving the original request as global intent; (2) global verification and repair, where a Verifier checks the synthesized candidate deliverable against the original request and a targeted ReAct-style Repair stage patches specific defects; and (3) execution memory combining summarized truncation of tool observations, subtask-boundary state passing (compact answer/result-file checkpoints), and automatic context compression (LLM-generated summaries at a configurable context threshold, e.g. 0.9x budget, with deterministic emergency pruning as fallback). Tool access is unified across web search/visit, academic search (Google Scholar, OpenAlex), Python/command execution, file read/write/edit, and multimodal (image analysis/generation) tools. The harness is evaluated on the AgentIF-OneDay benchmark (104 tasks, 767 scoring points) across five backend LLMs (GLM-5.2, Gemini-3.1-Pro-Preview, Qwen3.5-397B-A17B, Qwen3.5-9B, Qwen3.6-27B).

Key results:

  • With the GLM-5.2 backend, OneDayAgent achieves a new state-of-the-art overall score of 0.821 on AgentIF-OneDay, exceeding baselines including AutoClaw (0.799), Gemini-3.1-Pro-Preview backend variant (0.743), Codex/GPT-5.5 medium (0.664), Manus (0.645), and Minimax-Agent (0.562), while leading across all task types (OWE, LII, IR), domains, and rubric dimensions.
  • Ablation (2x2 over decomposition and verification, GLM-5.2 backend): DIRECT (both disabled) scores 0.771; DECOMP alone reaches 0.804 (+3.3 pp); VERIFY alone reaches 0.804 (+3.3 pp); FULL (both) reaches 0.821 (+5.0 pp) but at 53.6 minutes average latency versus 27.6 minutes for DIRECT and only 29.7 minutes for VERIFY alone.
  • Execution behavior: most tasks (88/104) are decomposed into 2-5 subtasks; five-subtask tasks cost 117.2 minutes and 156 tool calls on average versus 20.6 minutes/17 tool calls for one-subtask tasks; 95/104 tasks pass verification on the first attempt, 9 enter repair, of which 6 are recovered and 3 still fail; 35/104 tasks trigger context compression with near-zero correlation (r ≈ -0.034) between compression count and final score.
  • Across five backend LLMs (9B to 744B-parameter scale), the same unchanged harness produces overall scores ranging from 0.613 (Qwen3.6-27B) to 0.821 (GLM-5.2), showing a weak but non-monotonic parameter-scaling trend (e.g., Qwen3.6-27B does not beat Qwen3.5-9B) alongside distinct backend execution-style profiles (GLM-5.2: 53.6 min/51.6 tool calls/585.7 KB context per task; Gemini-3.1-Pro-Preview: 21.4 min/18.7 tool calls/118.1 KB context; Qwen3.6-27B has the highest repair rate at 56.7%).

Why it matters / caveats: The results suggest a single harness design (decomposition + verify/repair + execution memory) can transfer across model families without backend-specific tuning, while cost/latency/quality tradeoffs mean the "always enable every module" configuration is not universally optimal (VERIFY-only matches DECOMP's score at much lower latency in the ablation, and simpler variants beat FULL on a nontrivial subset of tasks). The authors note findings are specific to the AgentIF-OneDay benchmark and the current implementation runs without workspace isolation, a stated security limitation.

GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks →

arXiv 2608.03764 · ▲ 20 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing benchmarks for evaluating agent self-evolution (learning from prior task experience and transferring it to new tasks) provide limited coverage of economically valuable enterprise domains, often fail to construct train-test splits that make test-time gains attributable to training experience, and remain vulnerable to data contamination.

Method: The paper presents GDPevo, an evolution-native benchmark built by a fully automated three-stage agent pipeline (seed scenario discovery from real-job benchmarks like GDPval, SOP-Bench, and JobBench; task group generation via an orchestrator agent plus environment/task builder agents; and calibration/review by six independent reviewer agents), all powered by the same Codex+GPT-5.5 harness/model run as independent instances. Its core mechanism, rule hybridization, decomposes each enterprise workflow into atomic business rules, scatters subsets of these rules across five training tasks per task group, and recombines untested rule combinations in five held-out test tasks, so genuine rule learning (not memorization) is required to succeed. Each task uses a deterministic rule-based grader (an LLM converts rubric points into code-based test cases rather than acting as judge). The evaluation compares four agents (harness+model combinations: GPT-5.5/Codex, Opus-4.8/Claude Code, GLM-5.2/Claude Code, DeepSeek-V4-Pro-Preview/Claude Code) under four supervision types — base (no evolution), fewshot (gold answers, analogous to SFT), reflect (three rounds of self-attempt plus grader feedback, analogous to RL), and self (unsupervised exploration only) — with a skill-based evolution method (a portable SKILL.md artifact) as the persistent state.

Key results:

  • GDPevo V1 contains 120 tasks in 12 groups (CRM, ERP, finance); the fully automated pipeline expanded this to V2's additional 120 tasks in 12 groups (healthcare, legal, data-centric) within two days, giving 240 tasks across 24 groups total.
  • Self-evolution consistently improves held-out accuracy over base by 2.59 to 16.44 percentage points across all four agents; fewshot is the most reliable supervision type, attaining the highest accuracy for every agent tested.
  • The best evolved agents remain far below a fully informed oracle ceiling of 91.6% accuracy (the score achieved when a model is given all hidden rules directly), indicating substantial unrealized self-evolution headroom.
  • DeepSeek-V4-Pro-Preview with fewshot reaches 48.79% accuracy, comparable to GPT-5.5's base accuracy of 49.37%, at roughly 1/28 of the amortized end-to-end cost; GLM-5.2 fewshot (67.07%) surpasses GPT-5.5 base (49.37%) and Opus-4.8 base (50.63%) by 10.72 and 9.46 pp respectively at roughly half the cost.
  • Cross-domain transfer (CRM/ERP/Finance 3x3 matrix): fewshot behaves like SFT and overfits its source domain, with 5 of 6 off-diagonal cells negative (worst case -5.0 pp); reflect-3 transfers more robustly, with half of off-diagonal cells positive (up to +6.5 pp transferring ERP to Finance) and a worst case of only -1.0 pp.
  • Controlled comparison of five skill creators (Naive, Claude Code, Codex, DeepAgents, OpenCode) holding the model fixed shows the minimal "Naive" creator performs on par with or better than more elaborate creators (e.g., +15.46 pp vs. +12.49 to +13.03 pp for GPT-5.5/Codex), indicating evolution magnitude is driven mainly by the underlying model's capability rather than the sophistication of the evolution method.

Why it matters / caveats: GDPevo demonstrates that rule hybridization can make test-time gains from self-evolution attributable to specific training experience rather than memorization, and its full automation offers a practical, fast (two-day) response to benchmark contamination. The reported oracle ceiling of 91.6% versus best evolved scores in the 55-67% range indicates current agent self-evolution methods capture only a fraction of the learnable business rules; the paper's cross-domain and skill-creator experiments were restricted to a subset of domains (three V1 groups) and two agents respectively, limiting generality of those specific findings.

Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning →

arXiv 2608.05139 · ▲ 18 on Hugging Face · HF page · PDF

Technical breakdown

Problem: LLMs that perform well on single-skill benchmarks remain brittle on long-horizon tasks that require switching between different reasoning skills mid-chain (e.g., math derivation feeding into scheduling), and no existing benchmark or training signal measures or targets this skill-switching difficulty directly.

Method: The paper defines Skill Entropy, SkE(s_a, s_b), a directed pairwise measure of how hard it is to switch from skill s_a to s_b, computed as a Laplace-smoothed (α=0.1) ratio between average single-skill accuracy and cross-skill accuracy under a fixed reference model (Claude-opus-4.7); task-level skill entropy is the average pairwise entropy along a task's skill sequence. Building on this, Skill²-Bench is a benchmark of cross-skill long-horizon tasks constructed from 558 labeled skills across 9 domains (math from OpenR1-Math, science from MMLU-Pro, coding from LiveCodeBench, logic from ZebraLogicBench/Guru-RL-92k, information extraction from WikiTable/WebSRC, planning from NaturalPlan, plus creative writing, context retrieval, and instruction following), with tasks sorted into low/medium/high skill-entropy tiers. Skill-Entropy RL is a two-stage GRPO-based training pipeline: an SFT warm-up on skill-annotated traces (from a Qwen3-8B teacher) followed by GRPO with a reward r = λ_ans·r_ans + λ_ent·r_ent (λ_ans=0.7, λ_ent=0.3), where r_ans is per-step correctness and r_ent grades the predicted skill sequence against the gold sequence via rank alignment of task-level skill entropy, using a "<skill>...</skill><answer>...</answer>" response format.

Key results:

  • Evaluating 8 frontier models (Claude-haiku/sonnet/opus, Gemini-3.1-flash/pro, GPT-5.4-mini, O4-mini, GPT-5.5) and 4 open-source models (Qwen3-4B/8B/32B, Olmo-3-7B-Think) shows accuracy drops of −4% to −13% when the same skill is tested in a cross-skill task versus single-skill mode.
  • Skill²-Bench performance decreases nearly monotonically from low to high skill-entropy tiers for nearly all models (e.g., Claude-opus-4.7: 72.2% low → 68.6% high; GPT-5.4-mini: 42.6% low → 42.7% high with larger swings for weaker models).
  • On Qwen3-4B-Instruct, Skill-Entropy RL raises Skill²-Bench score from a 34.4% base to 68.4%, beating vanilla GRPO (58.8%) by +9.6% and the strongest skill-aware baseline STAT (61.4%) by +7.0%.
  • On Qwen3-1.7B, Skill-Entropy RL raises the score from 14.6% to 40.1%, beating GRPO (32.2%) by +7.9% and STAT (33.0%) by +7.1%.
  • Applied to off-the-shelf OpenR1-Math data (no built-in skill structure), Skill-Entropy RL improves over GRPO by +1.9% on average across six math benchmarks and over the base model by +7.7%, while vanilla GRPO's reward plateaus during RL training.

Why it matters / caveats: The work shows that per-skill competence does not predict skill-switching ability, and that models tend to default to reusing the previous step's skill/answer modality rather than switching, roughly halving accuracy when the wrong skill is picked; the skill-entropy reward is shown to transfer to open-ended domains not seen during RL training and to five external benchmarks (MuSR, LongBench-MuSiQue, GPQA-Diamond, MMLU, IFEval), though the skill entropy scale itself is calibrated against one fixed reference model (Claude-opus-4.7), which may limit its generality as an absolute difficulty measure.

When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation →

arXiv 2608.03632 · ▲ 18 on Hugging Face · HF page · PDF

Technical breakdown

Problem: In on-policy distillation (OPD), existing methods for selecting which token-level teacher signals to trust (based on confidence, informativeness, or learnability) assume the teacher's judgment is reliable, ignoring that a teacher's token-level judgments can be driven by input-agnostic language priors, formatting conventions, or stereotyped reasoning templates rather than actual task evidence — producing "spurious signals" that yield large but misleading gradients.

Method: The paper proposes SA-OPD (Spurious-Signal-Aware On-Policy Distillation), which decomposes the sampled teacher-student divergence A_t at each token into a grounded component and an input-agnostic prior component, and defines an Input-Grounding Gap ΔIG_t = A_t^full − A_t^res, the difference between the teacher-student divergence computed under the original prompt versus a residual no-prompt context (same generated prefix, prompt removed). SA-OPD filters tokens satisfying both low input-groundedness (bottom-p1 of ΔIG_t) and high optimization impact (top-p2 of |A_t^full|), using a dynamically adapted (p1, p2) constrained by a target filtered-loss-mass ratio (FLMR ≤ β, best β=1.8), and trains the reverse-KL objective (L_SA) only on the retained tokens. Experiments use Qwen3-4B-Instruct→Qwen3-1.7B and Qwen3.5-35B-A3B→Qwen3.5-2B as teacher-student pairs (plus DeepSeek-R1-0528-Qwen3-8B→Qwen3-1.7B and Qwen3.5-9B→Qwen3.5-2B for scalability checks), trained on DeepMath (math) and on VERO-600K/MMRL30k subsets (VLM visual understanding/reasoning), compared against Vanilla OPD, ExOPD, TIP, and FiRe-OPD baselines.

Key results:

  • On visual understanding benchmarks (EvoChart, MMIFEval, CountQA) under Qwen3.5-35B-A3B→Qwen3.5-2B, SA-OPD improves average score from 50.5 (Vanilla OPD) to 54.0, with +7.2 pts on CountQA specifically.
  • On visual reasoning benchmarks (MathVision, Geo3K, MathVista), SA-OPD raises the average from 60.4 to 63.5, with +5.0 on Geo3K and +2.2 on Geo3K/+1.2 MathVision noted as largest gains over the strongest prior baseline.
  • On math reasoning (Math500, AMC23, AIME25, AIME24, MinervaMATH) under Qwen3-4B-Instruct→Qwen3-1.7B, SA-OPD improves average from 28.5 (Vanilla OPD) to 30.4, including a +3.0-point gain on Math500 over the strongest prior OPD baseline (TIP).
  • Ablations show single-criterion filters underperform: Divergence-only and Input-Groundedness-only filters each help less than combining both; on the dynamic-threshold sensitivity study, the counting (CountQA) score rises from 30.8 (without dynamic thresholding) to 33.6 at the best β=1.8.
  • Filtered-token analysis on 500 sampled visual-understanding examples shows content words make up 69.9% of filtered tokens, with 60.2% being tokens seen before and 27.9% repeated bigrams, and filtered tokens span both high- and low-entropy regions (contradicting the idea that entropy alone captures spurious signals).

Why it matters / caveats: The paper identifies input-groundedness (not just divergence magnitude, confidence, or entropy) as a previously unaddressed axis for filtering harmful OPD supervision, requiring no external verification labels; the mutual-information formalization of input-groundedness has no closed form and is approximated by a lightweight no-prompt-removal proxy, and results are demonstrated only on Qwen and Qwen3.5 model families as both teacher and student.

NOLLI: A Difficulty-Calibrated Puzzle Benchmark for Diagnosing the English-Korean Performance Gap →

arXiv 2608.04397 · ▲ 15 on Hugging Face · HF page · PDF

Technical breakdown

Problem: When an LLM performs worse in Korean than English, existing multilingual benchmarks collapse language-specific failures into a single undifferentiated performance gap, leaving it unclear whether the gap stems from presentation language itself, from Korean's writing system (Hangul jamo structure), or from Korean-specific cultural/linguistic knowledge.

Method: The paper introduces NOLLI, a procedurally generated English-Korean benchmark of 15 puzzle types (25 tasks after language split; 7,500 items total, 100 seed-regenerable instances per difficulty tier), organized into three tiers: 8 direct-translation types (e.g., Sudoku, SAT Puzzle, Minesweeper) using identical generators/parameters in both languages to isolate presentation-language effects; 2 script-adaptation types (Cipher, Cryptarithmetic) that operate over Hangul jamo rather than being direct translations, isolating writing-system demands; and 5 Korean-only types (Kinship, Saju, Time, Jamo Composition, Korean Units) grounded in Korean culture/orthography. Difficulty is calibrated behaviorally rather than structurally: generator parameters are swept until a fixed reference model (Gemini 3 Flash, medium reasoning effort) lands in target accuracy bands of 75±10% (Easy), 50±10% (Medium), 25±10% (Hard), with adjacent tiers separated by at least 10pp; each instance is verified to have a unique solution via deterministic solvers (backtracking, constraint propagation, exhaustive filtering, optimal assignment, or exact calendar/script computation) and scored by exact match with no LLM judge. Statistical comparisons use Holm-corrected TOST (two one-sided tests) for equivalence testing and McNemar tests for paired item-level contrasts.

Key results:

  • Across 15 evaluated models, GPT-5.5 achieves the highest macro accuracy (84.9%), followed by Opus 4.8 (78.3%) and Gemini 3.1 (73.6%); Korean-developed models underperform similarly-sized international open-weight models (EXAONE-4.0: 25.1% vs. Qwen3.5-27B: 44.4%), and three models (EXAONE-3.5, Mi:dm-2.0, Llama-3.1-8B) fall below a 3% accuracy floor.
  • On direct-translation tasks, English-Korean gaps across 12 non-floor models range from only −2.7 to +5.0 pp and are statistically equivalent within a ±10pp margin under Holm-corrected TOST, indicating presentation language alone costs little.
  • Korean Cipher (a script-adaptation task over jamo) shows drops of up to 68.7 pp (DeepSeek-V4-Flash: 74.3% EN vs. 5.7% KO), while Cryptarithmetic over the same jamo shows no systematic penalty (6 of 12 non-floor models score higher in Korean); Jamo Composition accuracy predicts Korean Cipher accuracy with R²=0.91 (partial r=0.87 controlling for Direct-KO accuracy).
  • The calibrated Easy>Medium>Hard difficulty ordering transfers across models: accuracy decreases from Easy to Hard in 255 of 300 non-floor model-task pairs (85%; 94% including ties), despite a salient structural size measure failing to grow from Easy to Hard in 7 of 15 puzzle types.
  • The Kinship task shows a positive deficit (task accuracy below the model's own Direct-Korean baseline) in all 12 non-floor models, ranging from +4.7 pp (Gemini 3.1) to +36.7 pp (GPT-5.5), and does not shrink with overall model capability (r=0.16, p=.62), unlike rule-application tasks (Saju/Time/Korean Units) whose deficit varies in sign across models.

Why it matters / caveats: By separating presentation-language, writing-system, and cultural-knowledge effects into distinct diagnostic contrasts, the benchmark shows Korean underperformance is concentrated in sub-syllabic (jamo-level) multi-step manipulation and in culturally-specific lexical mapping rather than in language presentation per se; the authors note these are diagnostic, not causal, comparisons since script-adaptation and Korean-only tasks differ from direct translations in content and procedure, and difficulty calibration is anchored to a single reference model (Gemini 3 Flash) whose absolute band membership need not transfer to other models or reasoning-effort settings.

Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data →

arXiv 2608.02580 · ▲ 15 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Robot manipulation policies need large-scale, diverse demonstration data, but collecting robot teleoperation data is expensive and limited, while it remains unknown whether converting abundant egocentric human videos into robot-format data can provide pretraining benefits (rather than just per-task gains) for vision-language-action (VLA) models at scale.

Method: Ego2Robot is a three-stage pipeline: (1) action alignment retargets 21 MANO hand keypoints (via WiLoR per-frame estimation and DynHaMR temporal optimization for unannotated video) into parallel-gripper end-effector trajectories using a virtual-fingertip TCP formulation, Savitzky-Golay/SLERP smoothing, and per-source frame subsampling to match robot speeds; (2) visual alignment uses SAM 3 for arm segmentation, ProPainter for hand removal/inpainting, an IK-feasibility-based grid search in MuJoCo for robot base placement, and depth-aware compositing to render one of 15 robot morphologies into the scene; (3) multi-level quality curation (L1 pipeline-internal IK/collision checks, L2 statistical outlier/discontinuity filters, L3 Qwen3.5 VLM semantic-consistency audit) filters the output. Training uses a VLA with a Qwen3.5-4B vision-language backbone and a Diffusion Transformer (DiT) action head predicting 32-step camera-frame relative end-effector action chunks, evaluated on an extended RoboTwin2.0 benchmark with disentangled visual/scene/embodiment/task perturbation axes plus EBench.

Key results:

  • The pipeline converts ~1,940 hours of egocentric video (ANT, EgoDex, ViTRA, EgoVerse) into 18,561 hours of synthetic robot training data across 15 morphologies (~9.6x amplification), the largest ego-to-robot dataset reported to date.
  • Co-training Ego2Robot data with robot data (1:1 ratio) raises RoboTwin2.0 Randomized success from 50.9% (robot-only) to 53.5%, with visual-perturbation success rising from 61.4% to 67.3% and EBench average from 39.6% to 49.8% (best EBench score of 51.7% at 3:1 ratio, +12.1 points).
  • Cross-embodiment transfer improves with more morphologies: ARX success rises from 44% to 51% and UR5 peaks at 31% (from 20%) at 1:1/3:1 ratios; unseen-object task success improves from 29% to 40% (+11 at 3:1) and paraphrased-instruction success reaches 69% (vs. 63% baseline).
  • Ablations show pipeline processing alone (single morphology) lifts ego-only pretraining success from 28.1% (raw ego) to 31.7%, and scaling to 15 morphologies further raises it to 33.5%/37.3%; on a real ARX ACone robot across 5 long-horizon tasks, mixing teleoperated demos with Ego2Robot-converted play video gives the largest real-robot gains (+14 on Put Blocks, +13 on Insert Screw) over robot-only pretraining.

Why it matters / caveats: The results indicate ego-to-robot synthesis can be scaled to serve as VLA pretraining data rather than only per-task augmentation, with gains concentrated in visual, embodiment, and semantic out-of-distribution robustness rather than raw trajectory coverage. The authors note limitations: retargeting only supports parallel-jaw grippers (discarding dexterous finger articulation), visual compositing can introduce artifacts under heavy occlusion, and evaluation is confined to RoboTwin2.0's task scope.

AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities →

arXiv 2607.24821 · ▲ 15 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing video-editing benchmarks evaluate visual transformations on silent clips or audio editing in isolation, leaving models' ability to execute audio-visually coupled editing instructions, preserve non-target content, and maintain cross-modal consistency largely unmeasured.

Method: AVE-Compass is a benchmark of 145 curated source videos (from OmniVideoBench, UltraVideo, in-the-wild web video, and AIGC-generated video) and 196 human-verified audio-visually coupled editing instructions spanning 28 operation types across joint audio-visual, speech, video-only, and audio-only branches, generated via an LLM-generator-plus-critic-plus-human-review pipeline and decomposed into 2,688 fine-grained Yes/No checklist items. Evaluation combines MLLM-as-Judge scoring across four dimensions (Editing Intent = Instruction Following × Fidelity Preserving, plus Realism) with seven automated metrics (Lip Sync via SyncNet, AV Sync via Synchformer, Video Aesthetic, Subject Consistency via DINOv2, Motion Smoothness, Audio Aesthetic, Speech Quality/MOS). The paper also proposes AVE-Agent, a three-agent framework (Gemini-based planner building a subtask dependency DAG, an executor with a tool-use self-check reflection/improver loop, and a mixed evaluator that triggers remix/regeneration/replanning) built on Wan2.7 as the base editing model.

Key results:

  • AVE-Agent (Wan) achieves the best Overall Editing Intent score of 59.8 (+17.4 over base Wan2.7's 42.4) and the best Overall Instruction Following of 77.3 (vs. 69.3 for Wan2.7), with a +25.4-point audio Editing Intent gain over the base model.
  • Across six evaluated systems (Wan2.7, HappyHorse, Gemini-Omni, Seedance, LTX2, AVE-Agent), Editing Intent scores range from 15.2 (LTX2) to 59.8 (AVE-Agent), showing that no baseline exceeds 43 on this composite metric.
  • Human-LLM agreement on the checklist-based judging protocol averages close to 90% across all four dimensions (89.9-91.3%), and repeated-run stability shows automated metrics vary by less than 0.01 and MLLM-judge scores by less than 1%.
  • Response-gating analysis shows some models inflate Fidelity Preserving/Realism via non-response rather than successful editing (e.g., Gemini-Omni's audio response rate is only 22.0%, and Seedance's video response rate is 69.1%), while AVE-Agent shows the smallest score drop after gating (video response rate 92.7%, audio 94.4%).

Why it matters / caveats: The benchmark demonstrates that current audio-visual editing systems trade off Instruction Following against Fidelity Preserving (e.g., LTX2 follows instructions but largely regenerates the input, hurting preservation) and that perceptual Realism scores lag behind automated aesthetic scores, indicating persistent physical-world/audio-visual-consistency failures. A caveat noted by the authors is the benchmark's relatively compact scale (196 instructions), justified by the cost of running larger-scale MLLM-based evaluation for routine use.

When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents →

arXiv 2608.04574 · ▲ 12 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Memory-augmented VLM agents rely on persistent spatial memory that can silently become stale as an environment changes, and it is unclear whether current models can detect a memory-observation conflict before it causes a safety-relevant mistake.

Method: The paper introduces SpatialSTALE, a dynamic FrozenLake testbed (8x8 grid, Gymnasium-based) where each cell carries one persistent natural-language memory claim (SAFE/DANGER) that can silently flip (Thaw/Freeze) under three controlled change regimes: L1 (random spot changes, one-shot detection), L2 (spatially clustered changes, one-shot detection), and L3 (incremental online changes with repeated detection). Models are tested on entry-level staleness detection (text vs. rendered-image observation) and on downstream navigation using four memory strategies—NoMemory, NoFilter (raw stale memory), SelfVerify (one-shot self-check), and OMCD (Overconfident Memory Conflict Detection), a batched (B=10) event-triggered read-time audit that removes model-flagged-stale entries before they reach the navigator—plus an Oracle ablation using ground-truth stale labels through the same filtering interface. Six models are evaluated: GPT-4o, Claude-Sonnet-4.6, Qwen3.6-Plus, GLM-5.1, InternVL3-2B, and InternVL3-8B.

Key results:

  • Text-mode staleness detection is near ceiling for capable models (average F1 above 0.88 for GPT-4o, Claude-Sonnet-4.6, Qwen3.6-Plus, and GLM-5.1), but vision-mode F1 on identical grids ranges from 0.887 (Qwen) down to 0.067 (GLM), a roughly 13x spread, while InternVL3-2B/8B lag in both modalities (~0.22/~0.31 text F1).
  • On primary GPT-4o L2 text navigation, trusting raw stale memory (NoFilter) yields 74.4% death and 14.4% success versus 28.0% death and 28.8% success with no memory at all—stale memory is 2.7x deadlier than no memory; OMCD reduces GPT-4o's L2 death rate to 31.6% and improves success over NoFilter by 14.4pp (L1) to 24.4pp (L3), all p<0.001, with average death-rate reductions of 38% (Claude), 45% (GLM), and 43% (Qwen) relative to NoFilter.
  • Oracle (ground-truth) stale labels raise L2/L3 success by 18-24pp over raw memory (p<10^-5) but show no detectable further gain over OMCD's learned labels (|delta|<=0.02, p>0.2), and per-seed detection F1 does not correlate with OMCD navigation success (Pearson r in [+0.005, +0.060], all p>0.67).
  • Outcome-conditioned trajectories show NoFilter deaths terminate in about half the steps of successful runs (7.8-8.5 vs. 14.5-14.6 steps) on GPT-4o and Claude, the signature of memory directing the agent straight into a hole; in exploratory 10-seed vision navigation, OMCD provides no consistent improvement for GPT-4o, Claude, or GLM when the underlying visual audit is unreliable.

Why it matters / caveats: The study frames spatial-memory staleness as a distinct safety failure mode for memory-augmented agents and shows that a text-based read-time audit (OMCD) removes most of the safety tax, but reliable visual grounding of the audit and downstream action selection over filtered memory remain open problems, since even oracle-quality labels do not close the residual gap. The authors note their numbers are lower bounds from a fully-observable, symbolic 8x8 grid rather than field estimates, and results for vision navigation and mechanistic analyses are drawn from smaller-scale or single-model (GPT-4o) exploratory settings.

Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance →

arXiv 2608.00782 · ▲ 10 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Group Relative Policy Optimization (GRPO) loses all gradient signal on "zero-variance" prompts where every rollout in a group gets the same reward, and naively summing GRPO with on-policy distillation (OPD) to recover that signal actually degrades performance because not all samples benefit from distillation, the student converges prematurely to the teacher, and OPD's token-level advantages are asymmetrically negative.

Method: The paper proposes RSTG (Recovering Learning Signals via Adaptive Teacher Guidance), which restricts OPD exclusively to negative zero-variance prompts (all rollouts wrong) and reweights the OPD advantage by the teacher's mean@8 confidence score (a hybrid GRPO/OPD advantage function); applies token-level selection via a Soft-OR combination of student entropy and teacher-student KL divergence (inspired by TIP) to mask gradient updates to only high-value tokens; and adds an auxiliary SFT loss on teacher-generated correct trajectories (shortest correct response from 8 teacher samples) restricted to the same negative zero-variance prompts, with the OPD and SFT terms sharing a linearly annealed coefficient β (βinit=5e-3, βmin=1e-3). Training uses GRPO as the base RL objective and is built on the VeRL framework, with teacher-student pairs from Qwen3 and Qwen2.5 families trained on filtered DeepMath (57K math samples) and Eurus-RL-Code (25K code samples).

Key results:

  • RSTG outperforms naive GRPO+OPD by +4.02% on math and +3.05% on code (averaged across benchmarks), and surpasses baselines GRPO, OPD, ReLIFT, and RL-ZVP on nearly all metrics.
  • For Qwen3-1.7B-Instruct→Qwen3-4B-Instruct-2507, RSTG reaches 42.98 (AIME24), 31.87 (AIME25), 89.36 (MATH500), 57.36 (OLYMPIAD), averaging 55.39 vs. 51.37 for naive GRPO+OPD and 51.57 for GRPO.
  • Ablations show each RSTG component adds value: starting from GRPO+OPD on negative zero-variance prompts (51.37 avg), adding advantage weighting (+52.69), token selection (+53.24), and SFT (+55.39) on Qwen3-1.7B math benchmarks.
  • Only 3.63% of the 57K training set (Dswtr: samples where the student fails all 8 rollouts but the teacher succeeds all 8) yields the best OPD performance, better than training OPD on the full dataset or only student-failure samples.
  • RSTG slows convergence toward the teacher (65.81% top-k token overlap at step 200 vs. 69.7% for OPD and 67.6% for naive GRPO+OPD) and keeps response length comparable to GRPO, avoiding the abrupt length inflation seen with plain OPD.

Why it matters / caveats: The work identifies concrete failure modes of combining RL with distillation and shows that precisely scoping distillation (by sample, token, and via auxiliary SFT) can recover otherwise-lost gradient signal without capping performance at the teacher's level. The authors note experiments are limited to math and code domains and to model scales feasible on their compute, leaving agentic tasks and larger-scale validation as future work.

FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory →

arXiv 2608.04530 · ▲ 9 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing latent-memory approaches for GUI agents map each retrieved trajectory to one fixed memory block trained mainly via next-action supervision, causing important details to be lost during compression, forcing the same memory block to serve unrelated decision stages, and allowing irrelevant retrieved trajectories to mislead the agent.

Method: The paper introduces FocusMem, a frozen-policy latent-memory framework built on a shared Q-Former-style latent compressor that factorizes memory into three components: (1) a role-aware content basis using role-specific base queries trained with a semantic KL-distillation loss (Lsem) and a functional pseudo-target loss (Lfunc) so episodic and working memory retain different recoverable information; (2) a state-conditioned readout in which a lightweight state–item adapter (Aη) generates a residual over the base queries to produce decision-specific dynamic memory blocks; and (3) a block-level trust gate (Tξ) trained with a straight-through HardConcrete mask and an Lgate loss on injected irrelevant episodic items, which suppresses memory blocks deemed irrelevant to the current step. Training proceeds in two stages (fixed-query Stage A, then dynamic-readout/gate Stage B) atop a frozen Qwen3-VL-8B policy and UGround-V1-7B grounding model, using action supervision from MolmoWeb and Qwen3.6-27B rollouts (23,438 trajectories).

Key results:

  • Across five GUI-agent benchmarks (MMInA-Wikipedia, MMInA-Shopping, DeepShop, WebVoyager, Online-Mind2Web), FocusMem achieves the highest overall interactive task success rate (SR) of 48.3%, versus 33.7% for its matched Action-only Fixed baseline and 42.5% for the strongest prior method (Mem-W-style).
  • FocusMem improves over the fully matched Action-only Fixed baseline by +10.8, +13.5, +18.0, +17.0, and +13.9 points on the five benchmarks respectively, and beats Mem-W-style by +4.5 to +8.0 points.
  • A cumulative ablation on MMInA-Shopping shows monotonic gains: No Memory (41.0% SR) → +Fixed memory (45.5%) → +Content Basis (50.5%) → +Dynamic Readout (54.5%) → +Evidence Gate (59.0%), an 18-point total improvement over No Memory.
  • Content-basis recoverability probes show combining semantic and functional supervision ("Both") outperforms either alone, e.g., 74.6/74.1/70.3 (Semantic/Aligned/Transfer) for episodic trajectories vs. 54.2/50.8/47.1 for action-only supervision.
  • In oracle-centered evidence expansion, from Oracle Core to full trajectory, fixed-query readout drops 22–25 points in action accuracy across latent budgets B=4/8/16, whereas dynamic readout drops only 6–12 points; under injected irrelevant episodic contamination, full FocusMem stays 9–11 points stronger than a no-gate variant, and gate passage rate for contaminated blocks falls from 86% to 35%.

Why it matters / caveats: The results indicate that latent GUI memory benefits from separating what is retained, what is exposed to a given decision, and what is allowed to influence the policy, rather than simply compressing more past interaction. The authors note the evaluation is limited to a frozen Qwen3-VL-8B policy on web benchmarks (transfer to other backbones or mobile/desktop environments is untested), the trust-gate diagnostic only covers injected clearly-irrelevant trajectories rather than subtler mismatches, and all scores depend on a single judge model (Gemini-3.1-Pro), which may introduce judge bias.

K-EXAONE 2.0 Technical Report →

arXiv 2608.04505 · ▲ 9 on Hugging Face · HF page · PDF

Technical breakdown

Problem: LG AI Research needed to scale its domestically-developed K-EXAONE foundation model toward global frontier-scale capability and broader deployment-oriented capabilities (agentic coding, long context, multilingual, safety) without discarding the parameters, compute, and training investment already sunk into the predecessor model.

Method: K-EXAONE 2.0 is produced by upcycling K-EXAONE into a larger fine-grained sparse Mixture-of-Experts model (78 layers, 256 experts per MoE layer vs. 48 layers/128 experts), growing total parameters from 236B to 750B (37B activated) via block-level depth up-scaling (repeating LLLG blocks from the middle of the stack) and width up-scaling (duplicating experts with norm-preserving random rotation noise to break symmetry), followed by continued pre-training with Clamped SwiGLU (clip value 7.0) applied to the last 16 MoE layers for stability. It retains the hybrid sliding-window/global attention architecture and adds a DSpark semi-autoregressive drafter alongside the existing Multi-Token Prediction (MTP) module (refined with a total-variation loss) for two self-speculative decoding paths. The training pipeline comprises continual pre-training (with Active Reading synthetic data and Korean/multilingual data covering 10 languages), difficulty-focused mid-training (context extension 8K→64K→256K with long-context and agentic/SWE data), and post-training combining SFT with preference learning using the internally developed AGAPO and GROUPER (Group-wise SimPER) optimization objectives, plus a Korea-Augmented Universal Taxonomy (K-AUT-V2) safety framework refined via a Safety Teacher Advisory Council.

Key results:

  • DSpark outperforms the MTP drafter on identical draft budgets (γ=7), improving acceptance length by 32–66% and end-to-end speedup from 1.27–1.77× to 1.81–2.57× across GSM8K, MATH-500, AIME 2026, HumanEval, and MBPP.
  • On agentic coding, K-EXAONE 2.0 scores 68.2 on SWE-Bench Verified (+18.8 points over K-EXAONE's 49.4) and 43.8 on Terminal-Bench 2.1 (+13.5 points over 30.3).
  • On long-context understanding, OpenAI-MRCR jumps from 52.3 (K-EXAONE) to 94.4, with perfect Needle-in-a-Haystack retrieval up to 256K tokens; AA-LCR reaches 56.2 and KO-LongBench reaches 89.6 (vs. 53.5 and 86.8 for K-EXAONE).
  • On safety, K-EXAONE 2.0 scores 99.8 on KGC-Safety (vs. 96.1 for K-EXAONE, and 69.3–92.0 for competitor models Qwen3.5/GLM-5.1/DeepSeek V4 Pro) and 89.5 on ROK-Fortress (vs. 60.9 for K-EXAONE and 47.6–86.1 for competitors).
  • Multilingual math (PolyMath) improves 13.9 points to 71.3; instruction-following gains are +2.7 (IFEval, to 92.4) and +5.3 (IFBench, to 72.6); world-knowledge benchmarks GPQA-Diamond (+3.1 to 82.2) and Humanity's Last Exam (+4.7 to 18.3) also improve, while some benchmarks (e.g., MMLU-Pro at 83.5, HMMT Feb 2026 at 78.4) trail larger competitor models like GLM-5.1 and DeepSeek V4 Pro.

Why it matters / caveats: The report demonstrates that upcycling (rather than training from scratch) can scale a sovereign, open-weight (Apache 2.0) MoE model 3.2x in parameters while achieving an average improvement of over 10% across 24 benchmarks, with the largest domain-specific gains (~30% on three benchmarks) in coding/agentic coding. The authors explicitly state K-EXAONE 2.0 does not lead on every benchmark against larger competitors (e.g., DeepSeek V4 Pro at 1.6T total params) and frame this release as a step toward, not the endpoint of, trillion-parameter-scale frontier models.

HelloWorld: Enabling Socially Interactive Characters in Video World Models →

arXiv 2608.05070 · ▲ 9 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing interactive video world models let users control cameras or trigger events in generated worlds, but offer no way for users to socially interact with the in-world characters (e.g., have a character turn, wave, or greet the viewer on command).

Method: HelloWorld finetunes a pretrained video generation model (LTX-2.3) with a self-distillation pipeline: the frozen base model first generates its own interaction-rich training clips, from which Pi3X recovers first-frame point clouds and camera trajectories that are re-rendered into "warp videos"; a rank-32 LoRA on the self-attention projections of the video DiT is then trained under a flow-matching loss to reconstruct the clips conditioned on visible warp-video tokens (with the camera prompt excluded so warp video alone drives camera control). At inference, a training-free temporal cross-attention mask restricts the interaction-related text tokens to attend only to frames inside a user-specified "button press" window (applied to both video and audio streams), giving explicit temporal control over when the character responds.

Key results:

  • On the newly built HelloWorldBench (400 samples, 120 source images, 264 character instances, 101 interaction types), HelloWorld reaches TimeAcc of 81.7% versus baselines around 30–52% (near the 33.3% random-guess level for the three-way timing question).
  • HelloWorld attains GazeDev of 40.2° and CamCtrl of 82.9, both the best among compared methods (WorldPlay, Matrix-Game 3.0, LingBot-World, SANA-WM, Warp-as-History), while maintaining top video quality (BgCons 96.9, Aesthetic 5.27).
  • Ablating the temporal cross-attention mask shows TimeAcc jumps from 36.7% (no mask) to 80.9% (video mask only) to 81.7% (video+audio mask), with SpeechInWin rising from 52.5% to 69.1%.
  • In a 30-rater user study (41 samples), HelloWorld is preferred over all four competing methods on naturalness, interaction, and scene quality, with every 95% confidence interval above 66% (e.g., 88.5% preferred over a real-video-trained LoRA on the interaction question).
  • Adding warp-video conditioning increases inference time by only ~20% (50.3s to 60.2s for a 10-second clip) and FLOPs by ~36% (6.9 to 9.4×10^15) relative to the base model.

Why it matters / caveats: The self-distillation approach requires no manually collected or annotated data, since both the training videos and their camera/interaction structure are synthesized by the base model itself. The authors note HelloWorld does not yet support real-time interaction (generation is driven by pre-specified camera trajectories and interaction scripts) and does not model persistent character identity across multi-round interactions.

WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models →

arXiv 2608.04964 · ▲ 8 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Interactive video world models accumulate compounding errors over long autoregressive rollouts, and reinforcement-learning post-training cannot correct this because, for an arbitrary action sequence, there is no ground-truth future state available to measure long-horizon drift.

Method: WorldCycle exploits reversible action cycles (e.g., forward-then-backward camera motion) whose composed transformation is analytically the identity, giving an annotation-free verification target. It constructs a spatial closure reward that compares mirrored forward/reverse frame pairs (via CoTracker point tracking) at every intermediate depth of a cycle, and a temporal consistency reward that compares phase-aligned frames across K repeated executions of the same cycle; these are combined with an action-following reward and an HPSv3 visual-quality reward and optimized with a DiffusionNFT-style negative-aware regression objective, following a warm-up-and-combine schedule (spatial-only warm-up for tw=300 steps, then joint optimization) and multi-scale hierarchical cycle sampling to prevent shortcut learning. The model is post-trained from the 8B WorldPlay checkpoint on ~4,000 image-caption pairs using AdamW (learning rate 1×10^-5), rollout group size G=16, up to K=8 repeated cycles, on 8 H200 GPUs for 3 days.

Key results:

  • On the authors' new CycleBench (47 action trajectories, 380 initial frames, 4 tasks × 4 horizon/complexity settings), WorldCycle reduces Reverse-Path Symmetry (RPS) error by up to 44% and Endpoint State Closure (ESC) by up to 32% relative to WorldCompass at short-term (125 frames).
  • On out-of-distribution composite actions (simultaneous translation+rotation), action accuracy rises from 0.136 (base WorldPlay) to 0.553 (WorldCycle), a roughly 4x improvement, and +11% over WorldCompass (0.499).
  • At long-term horizon (381 frames), Repeated-Cycle Stability (RCS) drops by 34% versus WorldCompass, the largest relative gain across horizon settings.
  • Visual quality (HPSv3) is not sacrificed: WorldCycle achieves 10.42 vs. 8.98 for the base WorldPlay model (a 16% gain) and is on par with or better than baselines across all settings.
  • Ablations show removing the temporal reward most hurts long-term ESC (0.163→0.212) and composite generalization, while removing the spatial reward degrades performance broadly; a joint (spatial+temporal) warm-up-and-combine schedule outperforms both sequential and joint-from-scratch training schedules.

Why it matters / caveats: The cycle-consistency framing turns a previously unverifiable long-horizon supervision problem into an exact, label-free target for a specific but "ubiquitous" class of dynamics (reversible trajectories like camera ego-motion), and it generalizes to composite action combinations the base model handles poorly without any ground-truth video. The approach is inherently restricted to reversible/closed action sequences; the paper notes extending the cycle-return principle to approximate reversibility and other action modalities (e.g., robotic manipulation) as future work.

OPD-V: Visual On-Policy Self-Distillation with Modality Balance →

arXiv 2608.05131 · ▲ 8 on Hugging Face · HF page · PDF

Technical breakdown

Problem: In on-policy self-distillation (OPSD) for multimodal large language models, carefully designed privileged information for the teacher is underused because textual context can dominate generation (Modality Imbalance), limiting how much the visual input actually informs the distilled supervision.

Method: OPD-V treats "Modality Balance" itself as privileged information by constructing a Positive Teacher conditioned on a Zoom-In Image (an evidence-centered crop magnifying the task-relevant region) and a Negative Teacher conditioned on a Mask Image (the Zoom-In Image with a random rectangular region blacked out); both score the same student-generated on-policy tokens under matched textual query and prefix, sharing EMA-updated teacher parameters. A tokenwise Modality-Balance Logits Margin (difference between Positive and Negative Teacher log-probabilities) defines a Modality-Balance Trust Region of tokens with positive margin, and only within this region is the Positive Teacher distribution distilled into the student via Jensen-Shannon divergence (with top-K=100 logit approximation for memory efficiency), scaled by the margin itself. Training uses Qwen3.5-4B/9B and Qwen3-VL-4B/8B-Instruct backbones on a 6.2K sample visual reasoning dataset from Vision-OPD, 8 rollouts per prompt, EMA teacher update rate 0.05.

Key results:

  • OPD-V raises Qwen3.5-4B's average accuracy across 6 benchmarks (V* Bench, ZoomBench, HR-Bench 4K/8K, MME-RealWorld EN/CN) from 64.30% to 80.01%, a 15.71 percentage-point absolute gain, beating the strongest matched baseline Vision-OPD (77.10%) by 2.91 points.
  • The resulting 4B model (80.01% average) outperforms every closed-source model reported, including Gemini-3.5-Flash (79.24%) and Gemini-3.1-Pro (79.25%), and much larger open models such as the 397B Qwen3.5 (77.44%) and 1T-parameter Kimi-K2.6 (72.81%).
  • OPD-V also improves Qwen3-VL-4B from 68.00% to 74.14%, Qwen3-VL-8B from 68.93% to 73.03%, and Qwen3.5-9B from 69.75% to 77.63%.
  • Training is more efficient: mean step latency drops from 352s to 240s (31.8% reduction) on the 4B backbone and from 451s to 340s (24.7%) on 9B, driven by 74.5% shorter mean response length (140.9 vs. 553.8 tokens) and cheaper teacher-input construction (15.3s vs. 79.2s on 9B).
  • Ablations show the Positive Teacher alone reaches 74.62% and the Negative Teacher alone 71.98%, while combining both gives 80.01% (complementary, not redundant); among tested image-operation pairs, Zoom-In + Mask is the strongest (vs. 75.95% for Zoom+Repeat, 72.17% for Zoom+No-Image).

Why it matters / caveats: The results indicate that explicitly correcting for modality imbalance in the teacher signal, rather than just enriching privileged information, is what unlocks OPSD's effectiveness, letting a 4B model surpass 1T-parameter systems on this benchmark suite. All comparisons are within a specific evaluation suite oriented toward visually demanding/high-resolution tasks and a single training dataset (6.2K samples from Vision-OPD), so generalization to other reasoning domains is not directly demonstrated in the text provided.

BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation →

arXiv 2608.05042 · ▲ 7 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing 3D vision-language-action (VLA) models for robot manipulation remain data-hungry, generalize poorly under distribution shift, and predict each action from only the current observation, so they lack explicit memory for tasks whose correct action depends on past interaction history or occluded scene geometry.

Method: BridgeVLA++ extends the authors' prior BridgeVLA framework, which projects point-cloud observations into three orthographic (top/front/right) views and uses a PaliGemma backbone (SigLIP encoder + Gemma transformer) with convex-upsampling decoders to predict per-view 2D translational heatmaps, pre-trained on the 120K-sample RoboPoint object-grounding split and fine-tuned with a coarse-to-fine heatmap loss (Ltrans+Lrot+Lgripper+Lcollision). BridgeVLA++ adds a unified spatio-temporal memory: a temporal memory (initial anchor views A0, recent neighboring keyframes, and adaptively selected sub-goal keyframes via a learned retention module Dϕ) injected into the coarse stage via cross-attention/self-attention blocks (Ftemp), and a spatial memory that re-renders the stored initial point cloud P0 under the current zoom/waypoint and injects it into the fine stage (Fspa) to recover geometry occluded by the gripper or object. The memory modules add ~269.77M parameters (9.2% over the 2.92B backbone) and extend naturally to bimanual manipulation via arm-specific action heads sharing the same memory-conditioned backbone.

Key results:

  • RLBench (18 tasks, 100 demos/task): BridgeVLA reaches 90.5% average success rate (vs. SAM2Act 86.8%), and BridgeVLA++ reaches 93.7%, with gains concentrated in occlusion-heavy tasks (Sort Shape +16.8%, Place Cups +18.4%).
  • COLOSSEUM (14 OOD settings): BridgeVLA achieves 64.0% average success (>7 points over RVT-2's 56.7%); BridgeVLA++ matches/slightly improves this at 65.2%. GemBench: BridgeVLA 50.0% vs. BridgeVLA++ 51.1%.
  • RMBench (memory-dependent dual-arm, 9 tasks): memory-free BridgeVLA drops to 18.9% overall, while BridgeVLA++ reaches 96.0%, beating the strongest prior memory baseline MemoryWAM (83.0%) by 13.0 points and Mem-0 (42.0%) by 54.0 points; MemoryBench success rate is 99.7±0.3%.
  • Real-world (Franka, 10 demos/task): BridgeVLA outperforms a strong baseline (RVT-2) by an average of 32% across generalization settings, and with only 3 demos/task still reaches 95.4% success. On real memory-dependent Dobot tasks, BridgeVLA improves from 20.0% (BridgeVLA, memory-free) to 93.3% (BridgeVLA++), a threefold gain over memory-augmented SAM2Act+ (30.0%).
  • Ablations: removing heatmap decoding drops RLBench success from 90.5% to 31.4%; feeding explicit 3D positions into the VLM backbone drops it to 56.2%; removing the memory extension entirely drops RMBench overall success from 96.0% to 18.9%.

Why it matters / caveats: The results show that aligning VLM pre-training and 3D action prediction in a shared 2D heatmap space, rather than adding explicit 3D encoders or token-based action prediction, is what drives data efficiency, and that a lightweight scene-level memory (rather than storing every frame, as in SAM2Act+) can add memory-dependent reasoning without eroding this efficiency or generalization. A caveat is added inference cost: BridgeVLA++ takes 0.57s per prediction step versus 0.35s for BridgeVLA on an RTX 4090, though the paper argues this is minor relative to motion execution time.

Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models →

arXiv 2608.04349 · ▲ 7 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Leading text-to-image models have complementary strengths (e.g., FLUX.1-dev's aesthetics vs. Z-Image's compositional accuracy), but their different autoencoders, denoisers, and noise schedules make it impossible to transfer these strengths into one compact, switchable model using standard latent-space distillation.

Method: Poly-OPD performs heterogeneous on-policy distillation into a 2.5B SD3.5-Medium flow-matching student by bridging incompatible latent spaces through pixels: the student's own rollout is decoded to an image, re-encoded by a selected teacher's (FLUX.1-dev or Z-Image) VAE, re-noised to a level matched by magnitude rather than timestep index, and refined by the frozen teacher for r steps (Eq. 3), with supervision applied via a cosine loss in frozen DINOv2 CLS-feature space (Eq. 4) rather than raw pixel/latent coordinates. Capability-selectable adapters share a single attention LoRA across teachers (chosen via a gradient-compatibility diagnostic showing attention gradients align across modes while FFN gradients conflict) while keeping teacher-specific feed-forward LoRA adapters, so switching capability costs only an adapter swap. Gap-aware adaptive sampling reallocates compositional training prompts toward categories (counting, position, two-object, color, attribute) with the largest teacher-student residual, using EMA-smoothed per-category gap scores and softmax reweighting, annealing to uniform as gaps close; training proceeds via a 500-step warm-start stage (off-policy imitation of teacher samples) followed by 800 on-policy distillation (OPD) steps.

Key results:

  • GenEval overall accuracy rises from 67.30 (SD3.5-Medium base) to 73.30 with Poly-OPD, exceeding both the Z-Image teacher (69.40) and the FLUX.1-dev teacher (65.20); DrawBench HPSv3 rises from 9.341 to 11.354, and ImageReward rises from 0.922 to 1.168.
  • Per-category GenEval gains concentrate on structured categories: two-object composition +14.93 points (82.07→97.00), attribute binding +8.25 (58.75→67.00), position +7.20 (24.00→31.20), versus +3.48 for color and +0.84 for counting; attribute binding ends above both teachers (67.00 vs. Z-Image's 52.80 and FLUX's 44.30).
  • DPG-Bench overall improves from 84.51 to 85.80, within 0.28 points of the Z-Image teacher (86.08), with the best relation sub-score (85.20) among compared models.
  • Ablations on DrawBench/GenEval: removing capability-selectable adapters (w/o CSA) costs 11.0 GenEval points, 1.24 HPSv3, and 0.364 ImageReward; removing gap-aware sampling costs 4.8 GenEval points overall and disproportionately hurts two-object (88.4 vs 97.0), color attribution (54.8 vs 67.0), and counting (53.8 vs 59.9); removing warm start degrades GenEval by 24.1 points and HPSv3 by 1.526.
  • Replacing the DINOv2 supervision space with ConvNeXt or SigLIP costs 7.1–7.5 GenEval points, and a direct MSE loss in student latent coordinates collapses training outright.

Why it matters / caveats: The method demonstrates that a 2.5B student distilled on-policy from two architecturally incompatible 12B/6B teachers can exceed both teachers on their respective specialty benchmarks (GenEval, ImageReward, UR-Alignment) within a single switchable backbone, which the authors attribute to on-policy training not being capped by the teacher's own sample distribution (unlike off-policy imitation). The approach is demonstrated with only two teachers/modes and evaluated at 512x512 resolution; per-metric trade-offs remain (e.g., PickScore stays flat versus the base student, and DrawBench LPIPS/UR-Coherence gains are comparatively small).

UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models →

arXiv 2608.04701 · ▲ 6 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Generating photorealistic, geometrically consistent novel views under large camera-baseline changes from a single monocular video or image is difficult because reconstruction methods (NeRF, 3D Gaussian Splatting) degrade under sparse input coverage, and prior generative approaches rely on inaccurate or implicit geometric guidance, especially where naive point-cloud rendering produces ambiguous visibility/occlusion.

Method: UniWorld-View lifts a source video/image into a point cloud via feed-forward depth/geometry estimation, then renders novel views using an occlusion-aware point cloud rendering strategy combining Triple-Reprojection-based Occlusion Disambiguation (back-project to target view, re-render to intermediate view, then back-project and re-render again to the source view to build a cumulative visibility mask Mvis via Eq. 4) and Normal-based Visibility Correction (culling back-facing points via a cosine-similarity threshold on surface normals, Eq. 5). These occlusion-aware renders and validity masks, plus the original source video, condition a dual-stream video diffusion model built on VACE (fine-tuned from WAN2.1-14B): VACE's Context Blocks digest the spatially-aligned but incomplete point-cloud renders, while newly introduced reference-conditioned DiT (Ref-DiT) blocks use cross-attention (novel-view features as queries, source-video features as keys/values) to inject visually-complete but spatially-misaligned source appearance. Training uses a two-stage curriculum: Context Blocks are trained on 100K self-supervised dynamic monocular pairs (via a self-supervised triple-reprojection degradation of OpenVid-1M using VideoDepthAnything), then Ref-DiT blocks are trained on 100K static multi-view triplets from DL3DV and RealEstate10K (reconstructed via VGGT). For 4D scene generation, the model separately reconstructs foreground/background depth layers (via SAM2 segmentation, video inpainting, and Stream3R/VideoDepthAnything depth alignment) and generates synchronized multi-view videos in a two-stage (static-anchor then dynamic) procedure to supervise downstream 4D Gaussian Splatting reconstruction.

Key results:

  • WorldScore benchmark: UniWorld-View achieves the best WorldScore-Static (85.53) among 9 compared model families, and the highest scores on all three Controllability metrics (Camera Control 97.72, Object Control 88.98, Content Alignment 86.61); it achieves the second-best WorldScore-Dynamic score (76.09, versus top WorldScape-0.2's 76.23).
  • Zero-shot novel view synthesis on RealEstate10K, CO3D, and DL3DV (vs. See3D, GEN3C, Uni3C, SEVA): UniWorld-View achieves the best PSNR and SSIM on all three datasets (e.g., RealEstate10K PSNR 21.7261 / SSIM 0.7833 vs. next-best Uni3C's 21.5379 / 0.7530), and best LPIPS on CO3D (0.3082).
  • On RealEstate10K and DL3DV, GEN3C/Uni3C obtain lower (better) LPIPS than UniWorld-View, indicating a trade-off rather than a clean sweep on perceptual metrics.

Why it matters / caveats: The occlusion-aware rendering (triple-reprojection plus normal filtering) targets a specific failure mode of prior point-conditioned diffusion methods — foreground-background tearing and back-face leakage under large-baseline shifts — and the dual-stream design lets a frozen geometric-conditioning path coexist with a separate appearance-reference path without retraining the whole backbone. A caveat is that WorldScore-Dynamic performance is only second-best, and LPIPS is not uniformly best across datasets, suggesting the geometric fidelity gains do not universally translate into the sharpest perceptual quality.

SKILL-KD: Contrastive Skill Distillation for LLM Agents →

arXiv 2607.28048 · ▲ 6 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Existing skill-acquisition methods for LLM agents treat skills as experience summaries or memory entries derived from an agent's own trajectory (or from a teacher's demonstration alone), which leaves frozen, weaker student agents without a way to translate a stronger teacher's behavior into guidance they can actually use.

Method: SKILL-KD is a contrastive skill distillation framework that keeps the student agent's weights frozen and instead edits an external, prompt-time skill library (a shared Markdown document). It contrasts a student's failed trajectory with a teacher's trajectory on the same task instance via a "consolidation agent" that proposes a candidate skill patch (Adaptive Skill Distillation), validates the patch by rerunning the student and iterating up to n=3 rounds until success or a round limit, and then applies Drift-Aware Skill Consolidation using trace-linked edit histories and a two-tool interface (trace-link and patch/add-modify-delete-skip) to decide whether to add, modify, delete, or skip a rule. It is evaluated in two teacher-student configurations: Qwen3.5-4B student with Qwen3.7-plus teacher, and Qwen3.6-35B-A3B student with ChatGPT-5.5 teacher, across five benchmarks (SearchQA, SpreadsheetBench, DocVQA, LiveMath, ALFWorld) against baselines EvoSkill, Trace2Skill, SkillGen, and SkillOpt.

Key results:

  • Group 1 (Qwen3.5-4B/Qwen3.7-plus): SKILL-KD reaches a macro-average of 66.8% across the five benchmarks vs. 43.5% for No Skill and 63.5% for the best baseline (SkillOpt); per-benchmark scores are 79.2% (SearchQA), 24.3% (SpreadsheetBench), 89.3% (DocVQA), 54.8% (LiveMath), 86.6% (ALFWorld).
  • Group 2 (Qwen3.6-35B-A3B/ChatGPT-5.5): SKILL-KD reaches a macro-average of 74.6% vs. 57.9% (No Skill) and 68.6% (SkillOpt best baseline), with ALFWorld improving to 96.3%.
  • On ALFWorld's held-out (unseen-environment) test split, SKILL-KD improves Qwen3.5-4B from 30.6% to 86.6% (+56.0 points) and Qwen3.6-35B-A3B from 59.7% to 96.3% (+36.6 points).
  • Ablation (Group 1): SKILL-KD's full method scores 66.8 average using only 38 rules/3,010 words, versus Student-only (60.1 avg, 96 rules, 6,821 words), Teacher-only (58.3 avg, 80 rules, 5,683 words), Batch (59.4 avg, 46 rules, 3,849 words), and Pairwise (59.0 avg, 85 rules, 6,966 words).
  • Removing drift-aware consolidation (edit history/trace-link access) drops SpreadsheetBench from 24.3% to 14.6% and LiveMath from 54.8% to 27.4%, while SearchQA stays tied at 79.2%.
  • The teacher succeeds on only 46.1% of student-failed training instances on average, yet 38.5% of accepted patches (on average) come from cases where the teacher itself failed, showing the method does not require oracle teacher supervision.
  • Adaptive rounds show diminishing returns: cumulative training success rises from 58.9% (n=0) to 67.7% (n=1), 70.8% (n=2), and 72.7% (n=3), with round 1 providing the dominant gain.

Why it matters / caveats: The approach offers a training-free way to transfer procedural knowledge across heterogeneous model families (different tokenizers/pretraining) without fine-tuning, and the ablations show that naive teacher-trajectory summarization or single-shot contrastive patches are insufficient — behavioral validation via student reruns and trace-linked consolidation are what prevent "skill drift" (case-specific, bloat, and destructive-update drift) and keep the library compact. Results rely on a fixed teacher/student pairing and evaluator feedback specific to each benchmark, and reported numbers come from single evaluation runs.

Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming →

arXiv 2608.05108 · ▲ 5 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Automated prompt-injection red-teaming methods for LLM agents either rely on reinforcement learning, which needs tens of thousands of target-model queries and generalizes poorly to new target LLMs, or on search-based methods that restart from scratch on every sample and therefore underperform RL-based attacks.

Method: PIMiner is an agentic red-teaming system built from four components — a strategy library (Markdown strategy files with target-LLM scope, task scope, injection template, in-context examples, and failure conditions), a strategy router that selects a Top-K candidate set of strategies per sample, an iterative attack module that refines injection prompts over up to Nmax=10 iterations using three memory tiers (long-term routed strategies, intra-dataset memory curated across prior samples, and intra-sample attack history), and an experience digester that updates the strategy library after each dataset-model pass by classifying successful attacks (covered/broadened-scope/new-mechanism) and downgrading strategies via miss-pattern analysis on failures. Each agent role is instantiated as an independent Claude Code session (claude -p) orchestrated by a Python/Bash harness; the default backbone is Claude-Opus-4.7, trained on 8 (dataset, target-model) pairs from AgentDojo and IPIArena training splits.

Key results:

  • On IPIArena, PIMiner attains ASR@10 of 76.2% against Gemini-2.5-Pro, 61.9% against GPT-5.1, and 42.9% against Claude-Sonnet-4.5; on AgentDojo it attains 86.7% against Gemini-2.5-Pro, 53.3% against GPT-5.1, and 40.0% against Claude-Sonnet-4.5.
  • Claude-Opus-4.5 is the most robust target evaluated, with only 4.8% ASR@10 on IPIArena and 3.3% on AgentDojo; GPT-4o-mini (81.0% on IPIArena) and DeepSeek-V4-Pro (93.3% on AgentDojo) are the most vulnerable.
  • On IPIArena vs. human red-teamers (from prior work), PIMiner's ASR@1 of 28.6% against Claude-Sonnet-4.5 vastly exceeds the reported human ASR@1 of 1.0% on average.
  • On InjecAgent, PIMiner matches the best RL-based baselines (RL-Hammer, PISmith) with ASR of 1.0 on GPT-4o-mini, GPT-4.1-nano, and GPT-5-nano, while search-based baselines TAP and PAIR reach only 0.08 and 0.01 respectively on GPT-5-nano.
  • On AgentDojo, PIMiner outperforms PISmith on GPT-5-nano (0.53 vs. 0.38 ASR) and is comparable on GPT-4o-mini (0.73 vs. 0.78), though lower on GPT-4.1-nano (0.63 vs. 0.81) — notably without training a separate attacker per target model, unlike PISmith.
  • Ablations show combining the strategy library and intra-dataset memory improves average ASR over a vanilla PAIR-style iterative attacker by 19.8 points on Claude-Haiku-4.5 and 17.8 points on Claude-Sonnet-4.6; the router also cuts attacker input length by 43% (Haiku-4.5) and 61% (Sonnet-4.6) with at most a 1.2-point ASR drop (and up to a 7.5-point ASR gain on Sonnet-4.6).

Why it matters / caveats: The hierarchical memory design lets a search-based (training-free) attacker transfer accumulated strategy knowledge to previously unseen target LLMs with far fewer per-sample queries (about 10) than RL-based attackers require during training, closing much of the effectiveness gap while avoiding costly attacker fine-tuning — useful both for red-team evaluation and for generating training data for defenses. The method still assumes grey-box access (observable intermediate agent outputs) during training and depends on a strong, costly backbone (Claude-Opus-4.7) for generating high-quality strategies; robustness varies sharply by target model (e.g., far lower ASR against Claude-Opus-4.5), so results are not uniformly transferable across all frontier models.

TriGlue: a Biology-Inspired Generative Model for Generating Molecular Glue-Induced Ternary Complex →

arXiv 2607.22143 · ▲ 5 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Computational design of molecular glue degraders is largely unaddressed because, unlike conventional structure-based drug design that assumes a predefined binding pocket on a single protein, molecular glue design must simultaneously model ligand generation, protein-protein docking, and ternary complex assembly around an unknown, ligand-induced protein-protein interface.

Method: TriGlue is a biology-inspired generative framework that factorizes ternary complex generation into two coupled stages: (1) an SE(3)-equivariant interface estimation module that uses an EGNN protein encoder and cross-protein attention to predict K "virtual interface points" per protein, which are aligned to a Gaussian-ellipsoid-parameterized representation (mean/covariance) of the true interface via an ellipsoid alignment loss (Lellip); and (2) an interface-conditioned ternary flow matching network with three coupled flows — a Rigid Transformation Flow (initialized via a Differentiable Kabsch Model and refined by an iterative Lie-algebra residual module, trained with LRT), a Coordinate Flow for ligand atom positions (using bi-level intrinsic/extrinsic message passing over an Invariant Point Attention-based Latent Ternary Encoder, trained with Lcoor), and an Atom Type Flow modeled as flow matching on the probability simplex (trained with Ltype). All components are jointly trained with a weighted sum Lflow = (λ1,λ2)·LRT + λ3·Lcoor + λ4·Ltype on the TernaryDB dataset (22,303 PDB-derived ternary complexes, sequence-clustered via MMseqs2 to prevent leakage), using AdamW with cosine or linear learning-rate schedules on 8 NVIDIA L40 GPUs.

Key results:

  • On molecule generation/complex reconstruction (Table 1), TriGlue achieves the lowest average ligand RMSD (5.22) versus AR (8.16), TargetDiff (6.27), and PocketXMol (6.74), despite not being given binding-pocket information (unlike the baselines).
  • TriGlue attains the best average Vina score (-6.23) and Vina dock score (-7.37, median -8.85) among compared methods, and the highest QED (0.67) for drug-likeness, though its SA score is worse than PocketXMol's (attributed to PocketXMol's much larger training set, ~10^8 molecules).
  • For protein-protein docking accuracy against DeepTernary, TriGlue achieves a higher DockQ acceptance rate (35.2% vs. 30.8%), though DeepTernary shows a higher best-case score with a heavier long-tail distribution.
  • On held-out interface-ellipsoid prediction (Table 2), TriGlue outperforms PeSTo and ScanNet on all four geometry/distribution metrics: JSD 0.47 vs. 0.68/0.68, Wasserstein-2 distance 12.65 vs. 58.26/58.96, Center-L2 10.99 vs. 57.39/57.57, and LogDet error 3.83 vs. 8.00/10.20.
  • Ablation: removing the ellipsoid interface prior ("w/o elli") or the interface correction step ("w/o corr") both reduce docking performance; removing interface correction is more damaging, dropping average DockQ from 0.22 to 0.15, and TriGlue wins the RMSD comparison against "w/o corr" and "w/o elli" in 84% and 78% of cases, respectively.

Why it matters / caveats: TriGlue is presented as the first framework to jointly perform de novo ligand generation and ternary complex assembly for molecular glues without requiring a predefined binding pocket, potentially accelerating discovery in a space where only a few hundred molecular glues have been experimentally identified to date. Evaluation is entirely computational (Vina docking scores, RMSD, DockQ, QED/SA) on a curated PDB-derived benchmark (TernaryDB) with no baseline that performs the full joint task, so comparisons are made piecewise against pocket-aware ligand generators and sequence/structure-based docking methods rather than an end-to-end competitor; no wet-lab validation is reported.

Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning →

arXiv 2608.04926 · ▲ 4 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Cross-representation understanding among chart images, tabular data, and visualization code is hard to supervise because the mappings between representations are inherently one-to-many, labeled correspondences are costly and ambiguous, and task-specific training objectives do not generalize across directions (e.g., chart-to-table vs. table-to-chart) or to unseen representation pairs.

Method: The paper introduces CoCoEvolve, a consistency-driven co-evolution framework built on a chart-table-code representation cycle (fθ: chart→table, gψ: table→code, h: code→chart, a deterministic code executor). It grounds one-to-many mappings into one-to-one correspondences via an auxiliary constraint space S, then trains two coupled policies (Mθ, Mψ) with a hierarchical reward combining a code consistency reward (execution success + cosine-similarity-based code embedding agreement), a visual consistency reward (CLIP, SSIM, OCR, DINO metrics on rendered charts), and a format reward, used as a drop-in signal for RL algorithms (GRPO, DAPO, GSPO) — this is CoCoEvolve@Train. The same consistency objective is reused at inference for test-time co-optimization (CoCoEvolve@Test), and an optional teacher module with linearly annealed teacher-guidance fraction extends the framework to supervised co-learning (Rs via SentenceBERT cosine similarity, Rt via schema/value F1). The authors also build CoCoEvolve@Eval, a rule-as-judge plus LLM-/MLLM-as-judge evaluation suite spanning all six cross-representation tasks, and train/evaluate on data sampled from ChartCoder plus out-of-domain sets from ChartMimic, ChartNet, and Chart2Code.

Key results:

  • On code generation, CoCoEvolve@Train+@Test reaches up to 100.00% sandbox execution success (e.g., Qwen3-8B, Llama3.2-3B, Qwen3-1.7B, Qwen3-4B all hit 100%), with gains up to +17.48% on rule-as-judge and +26.57% on LLM-as-judge metrics over baselines.
  • Chart evaluation shows gains of at least +6.79% (rule-as-judge) and +3.33% (MLLM-as-judge) across models; table evaluation gains reach up to +15.64% (rule-as-judge) and +24.72% (MLLM-as-judge).
  • Generalization to out-of-domain benchmarks: up to +37.97% on ChartMimic chart→code reproduction, +34.65% on chart+data→code reproduction, +37.91% on Chart2Code chart→code reproduction, +46.88% on chart+figure→code modification, +27.85% on chart+table→code modification, and +45.44% on chart+instruction→code modification; also up to +35.68% transfer to ChartNet.
  • Test-time-only teacher guidance improves performance on all six cross-representation tasks by at least +2.54%; on the non-overlapping ChartCoder test set, CoCoEvolve improves performance by at least +2.41%.
  • Applying CoCoEvolve on top of GRPO, DAPO, and GSPO (with Mθ=Qwen3-VL-4B, Mψ=Qwen3-4B) improves performance regardless of RL algorithm, with gains up to +18.71% (execution), +6.96% (code), +12.15% (chart), and +3.36% (table); DAPO gave the most consistent improvements.

Why it matters / caveats: The approach removes the need for costly one-to-many annotated correspondences by using cross-representation agreement itself as a self-supervised training and test-time signal, and it is reported to plug into multiple RL algorithms (GRPO, DAPO, GSPO) without redesign. Reported gains are largest on execution/functional metrics (e.g., sandbox execution success reaching 100%), while gains on more semantic/perceptual dimensions (e.g., visual style, data accuracy) are comparatively smaller, and results depend on the specific rule-based and LLM/MLLM judges (GPT-5-mini, Gemini-3-Pro) used for evaluation.

DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack →

arXiv 2608.03207 · ▲ 1 on Hugging Face · HF page · PDF

Technical breakdown

Problem: Flow-matching vision-language-action (VLA) models such as π0 and π0.5 have been reported as robust to adversarial perturbations, but this claim rests on attacks that ignore the multi-step denoising ODE these models actually use to generate actions, leaving open whether a deployed, frozen flow-matching VLA can be broken by a practical test-time patch.

Method: The paper introduces DRIFT (Denoising Redirection via Input perturbation of the Flow-matching Trajectory), a universal adversarial patch (32x32 px, ~2% of the 224x224 wrist image) placed on the robot's gripper in the wrist camera view, optimized offline with projected gradient descent (PGD, 500 iterations, step size 0.01) against a velocity-divergence objective computed only at the first Euler denoising step (k=0) of the policy's flow-matching ODE (K=10 steps for π0/π0.5). The objective maximizes the squared difference between the clean and patched velocity field vθ at that single step, exploiting an "early injection, full-path amplification" cascade effect; the authors show analytically and empirically (via per-step gradient cosine similarity) that attacking additional early steps causes gradient conflict (cos(g0,g9)≈-0.35) that weakens rather than strengthens the attack. They evaluate on π0 and π0.5 (LIBERO-finetuned checkpoints) across the four LIBERO suites (Spatial, Goal, Object, Long), comparing against a random patch, UADA (action-space attack), and EDPA (embedding-space attack) baselines under the same patch-size budget.

Key results:

  • DRIFT achieves 99.8% average Attack Success Rate (ASR) on π0 across four LIBERO suites (100.0% on Spatial and Object, 99.7% on Goal, 99.6% on Long), versus 13.2% average ASR for UADA and 25.3% for EDPA (EDPA required a 64px patch, 2x DRIFT's, to reach even that).
  • On π0.5, DRIFT reaches 99.3% average ASR with a 64px patch versus 30.5% (UADA) and 1.5% (EDPA).
  • The single-step attack (M=1) outperforms wider windows: attacking 3 or 5 early steps drops average ASR to 77.5% and 79.7% respectively on π0, confirming the "fewer steps, stronger attack" finding; gradient survival ratio R falls from 1.00 (M=1) to 0.45 (M=10) as the attack window widens.
  • Under an l-infinity perturbation budget sweep, ASR shows a sharp phase transition, jumping from near 0% at epsilon<=0.05 to 97% at epsilon=0.1; a grayscale (stealthier) patch needs roughly 2x the budget of a color patch to reach the same effect.
  • Cross-model transfer is weak: a patch trained on π0 transfers to π0.5 at only 0.4% (32px) or 8.9% (64px) ASR, while π0.5-to-π0 transfer reaches 24.7%, indicating patches are largely model-specific despite the shared vulnerability.
  • The failure mode is a "phantom grasp": the patched policy closes the gripper within a mean of 3.8 control steps (vs. ~46 for the clean policy) without approaching the object, and attention analysis shows the "bowl" token's attention on the patch region jumps from a below-chance 1.7% (clean) to 28-33% (patched, chance level 3.5%).
  • JPEG compression as a defense reduces ASR to near zero (0-12%) on Spatial/Goal/Object suites at quality Q<=50-60, but plateaus at a markedly higher 16-21% ASR on the Long suite even at the strongest compression tested (Q=10).

Why it matters / caveats: The results indicate that flow-matching VLAs' previously reported adversarial robustness is largely an artifact of attacks that ignore the denoising ODE structure, and that a single small, physically realizable sticker can break nearly all originally-solvable LIBERO tasks under a white-box, offline-optimization threat model. The attack requires white-box gradient access to the frozen policy for optimization (though not at deployment), patches do not transfer well across model versions, and the one tested defense (JPEG compression) is only partially effective and notably weaker on long-horizon tasks.

Self-Evolving Coding Agents →

arXiv 2608.03392 · HF page · PDF

Technical breakdown

Problem: Most deployed LLM-based coding agents remain static after deployment even though software engineering is a dynamic, feedback-rich process (evolving repositories, changing dependencies, failing tests, reusable repair experience), and there is no systematic account of the emerging class of "self-evolving coding agents" that adapt from their own coding experience.

Method: This is a survey, not an empirical method paper: it defines self-evolving coding agents (distinguishing them from static coding agents like SWE-agent/OpenHands and from general self-evolving agents) and proposes an object-centered taxonomy of five categories of what evolves — agent framework self-evolution (e.g., self-modifying scaffolds like SICA, SIFT, STOP; archive-based evolution like the Darwin Gödel Machine, Mendel/Huxley Gödel Machine), memory self-evolution (experience/repository memory systems such as SWE-Exp, EvoCoder, EvoRepair, Repository Memory, SAGE), skill and tool self-evolution (CODESKILL, GSkill, Socratic-SWE, EffiSkill, Live-SWE-Agent), model self-evolution (coder-verifier co-evolution and self-play RL such as ReVeal, CURE, ZeroCoder, Sol-Ver, ACE, Self-play SWE-RL, Agent-RLVR), and workflow/topology self-evolution (SEW, AFlow, EvoAgentX, SEMAG, EvoMAC, AgentConductor). It complements this with two orthogonal axes — evolving time (task-time, post-task, stage-wise) and evolving evidence (outcome evidence, environmental feedback, trajectory-derived evidence) — and reviews evaluation practices across benchmarks such as SWE-bench/SWE-bench Pro/SWE-Gym for repository-level issue resolution and HumanEval/MBPP/APPS/CodeContests/LiveCodeBench for function-level and competitive programming.

Key results:

  • Not stated: as a survey, the paper does not report its own experimental accuracy/benchmark numbers; it instead classifies and cites numerical results from ~50 surveyed papers (e.g., benchmark solve rates, pass@k) without aggregating them into comparable summary statistics in the extracted text.
  • Table 2 in the paper classifies each of roughly 20 representative systems (e.g., SICA, SWE-Exp, CODESKILL, Self-play SWE-RL, AFlow) along the dimensions of main evolving object, evolution timing, evidence type, and target software-engineering task/domain.

Why it matters / caveats: The survey's contribution is a conceptual taxonomy (what evolves, when, and from what evidence) plus a discussion of open challenges — reproducibility and benchmark overfitting, feedback reliability and safety when agents self-modify tools/scaffolds, staleness and quality control in long-term memory/skill libraries, and the current lack of evaluation beyond short-horizon pass/resolve rates (e.g., maintainability, security, cross-domain generalization). Because it is a synthesis rather than a new method or benchmark, there are no first-party quantitative results to report, and the field is explicitly described by the authors as "rapidly emerging" with "fluid" conceptual boundaries.

← 2026-08-052026-08-062026-08-07 →