AI papers — 2026-07-16

Problem: Modifying a large, tightly-coupled AI agent harness is bottlenecked by "behavior localization" — finding every scattered code location that implements a described behavior — since repositories are organized by files/functions while modification requests describe behaviors.

Method: The paper introduces Harness Handbook, an automatically-synthesized behavior-centric representation with an L1–L3 document tree (system overview, component/stage overview, source-grounded unit deep-dive) plus a cross-stage state-register view, built via static program analysis (function/call-graph extraction) and LLM-assisted behavioral structuring in two leaf modes (function-as-leaf using a seed skeleton, or file-as-leaf inferring the skeleton). Modification uses Behavior-Guided Progressive Disclosure (BGPD) to navigate coarse-to-fine from the handbook to verified source locations, followed by edit planning, execution, and automatic handbook resynchronization after every non-empty diff. Evaluation uses a read-only planner built with NexAU powered by DeepSeek-V4-Pro, tested on two open-source harnesses (Terminus-2 and Codex) with 30 modification requests each (Query, Cross-file, Search-Hostile types), judged by GPT-5.5, Opus 4.8, and DeepSeek-V4-Pro.

Key results:

• Handbook-Assisted planning raised overall plan-quality win rate from 28.3% to 38.3% on Codex (+10.0 pts) and from 26.7% to 45.6% on Terminus-2 (+18.9 pts), consistent across all three judges.
• Planner token use fell simultaneously: 0.102M→0.089M tokens/request on Codex (-12.7%) and 0.058M→0.053M on Terminus-2 (-8.6%).
• Against independent reference plans from Opus 4.8 and GPT-5.5, all 24 Recall/Precision/F1 comparisons improved (F1 gains 5.0–18.8 points); on Terminus-2, Handbook-Assisted reached file-level F1 of 84.7–89.3% and symbol-level F1 of 77.1–89.3%, with precision up to 93.3% against GPT-5.5.
• "Wrong" (zero-overlap localization failures) never increased and fell by up to 25.9 points.
• Gains held across all six harness-by-request-type comparisons (16.3–33.3 pts) and all six harness-by-difficulty comparisons (3.7–33.3 pts).

Why it matters / caveats: The results suggest a weaker planner model, when guided by a behavior-linked repository representation, can match stronger models' implementation-site localization while using fewer tokens — indicating that where to edit, not just how to generate edits, is a key lever for evolving complex agentic systems. Evaluation is limited to two open-source harnesses and 30 requests each, with plan quality judged by LLMs rather than ground-truth human verification.

Problem: Open-source unified multimodal image generation/editing models still lag behind closed-source systems like Nano-Banana-Pro and GPT-Image-2, largely because those closed systems achieve strength through undisclosed system-level integration (heavy comprehension modules, agentic inference) rather than a single disclosed model.

Method: Boogu-Image-0.1 is a model family (Base, Turbo, Edit, Edit-Turbo, plus Thinking variants) built around a 10-billion-parameter Diffusion Transformer (32 single-stream + 8 dual-stream layers, ~10,292.56M DiT params, ~11,022.55M total with prompt tuning) conditioned by a Qwen3-VL-8B Instruction Encoder (frozen, used as a "sensor" for language/vision) and an optional lightweight 3-layer Prompt Tuning Transformer; a separate ~32B Instruction Reasoner rewrites/decomposes user instructions before generation, decoupling comprehension from the DiT to save compute versus their initial 72B-VLM-driving-10B-DiT design. Training emphasizes data quality over quantity (a curated, long-tail-balanced corpus of 208.62 million unique images) plus an agentic inference-time pipeline (prompt rewriting, Base/Turbo model routing, and a Reflection step, orchestrated with Deepseek-V4-Flash) and a proposed classifier-free-guidance variant called Boosted Orthogonal Guidance that exploits the DiT output's 2D spatial structure.

Key results:

• On the authors' own Boogu Arena (Elo, blind pairwise human votes, ~4,000+ votes across 9 models), Boogu-Image-0.1-Turbo-Thinking scores 1048 overall, leading all open-source models (Z-Image-Turbo 960, HiDream-O1-Image 868) and approaching closed-source GPT-Image-2 (1196) and Nano-Banana-Pro (1087); Boogu Arena correlates with LMArena at Pearson r=0.986 / Spearman ρ=1.0.
• On Qwen-Image-Bench, Boogu-Image-0.1-Base-Thinking achieves the best overall score among open-source models: 53.57 (Chinese) and 53.73 (English), beating Qwen-Image-2512 (52.06/51.32) and HunyuanImage-3.0 (50.81/51.35).
• On LongText-Bench (text rendering), Boogu-Image-0.1-Turbo-Thinking reaches 0.971 average (0.985 on Chinese), ranking second overall behind closed-source Seedream-4.5 (0.988).
• On ImgEdit-Bench, Boogu-Image-0.1-Edit-Thinking attains the best overall score of any evaluated model, 4.64, ahead of JoyAI-Image-Edit (4.57), Seedream-5.0-Lite (4.42), and Nano-Banana-Pro (4.37).
• Achieved with an estimated total base-model training cost of roughly $400K.

Why it matters / caveats: The paper argues standard academic T2I benchmarks are saturated and poorly correlated with human preference (GPT-Image-2 ranks only mid-pack on GenEval/DPG-Bench despite being strongest by human preference), motivating Boogu Arena as a proxy; the authors caution against over-trusting ImgEdit-Bench, and note weaker world knowledge than closed-source frontier systems, text rendering limited to Chinese/English, anatomical inconsistencies in complex scenes, and detail bottlenecks from reusing the open-source FLUX.1 VAE.

Problem: Existing "zero RL" (RLVR training directly from a pretrained base model without human-annotated CoT data) has only been studied at small model scales, leaving the training dynamics, emergent capabilities, and reasoning quality of zero RL at the trillion-parameter scale unexplored.

Method: The authors build a four-stage zero-RL pipeline applied to Ling-2.5-1T-Base (1T-parameter MoE, 63B activated) and Ling-2.5-flash-Base (104B MoE, 7.4B activated): First Stage RL uses a clipped importance-sampling policy gradient (with training-engine-logit ratio correction between Megatron and SGLang, KL penalty, token-level loss) to elicit reasoning from scratch; a Self-Distillation phase compresses and shortens CoT traces to reset the training-inference numerical gap; Second Stage RL switches to sample-level loss normalization without KL penalty; and Third Stage RL introduces tier-based training (Low/Medium/High difficulty tiers with distinct system prompts and token budgets). Infrastructure includes FP32 mixed-precision for attention softmax/LM head and tailored context-parallelism, trained on 320 H200 GPUs via the Areal framework. They also propose a three-axis CoT quality evaluation framework (comprehensibility, reproducibility, efficiency).

Key results:

• Ring-2.5-1T-Zero (First Stage RL) reaches 84.2% on AIME 2026 from scratch with no human-annotated data; after all stages it reaches 93.2% on AIME 2026 and 92.5%/93.5% on AIME 2024/2025.
• The 1T model shows superior sample efficiency: after 3,600 steps it hits 89.06% (AIME 2024)/83.28% (AIME 2025) versus only 71.72%/63.54% for the 104B flash model even after 5,200 steps.
• Distilling from only 100K CoT samples (vs. DeepSeek-R1's 800K) yields a 5.8-point gain on Qwen-32B (78.4 vs. 72.6) and 4.5-point gain on Llama-70B (74.5 vs. 70.0) over DeepSeek-R1-distilled students.
• On mutually-solved AIME problems the model averages 6,368 tokens per correct trace, less than half the token count of baselines (GLM-5.1, MiniMax-M2.7, Qwen3.5-397B, Kimi-K2.6, ~14k-17k tokens).
• Their custom Megatron-numerator ratio correction keeps training stable past 3000+ steps versus baseline collapse at ~800 steps and IcePop collapse at ~2700 steps.

Why it matters / caveats: The paper argues empirically for the "bitter lesson" — scaling to 1T parameters dominates hand-engineered heuristics, with emergent behaviors (anthropomorphism, structured formatting, self-verification, parallel reasoning) arising without explicit supervision. A caveat: the Third Stage (tier-based) RL shows a slight drop versus the Second Stage peak, due to limited high-quality ultra-long reasoning data and negative transfer across jointly trained difficulty tiers.

Problem: End-to-end document parsing models (single-pass image-to-Markdown) still lag behind multi-stage pipeline methods in accuracy on mainstream document parsing benchmarks, despite being simpler to deploy.

Method: OvisOCR2 is a 0.8B-parameter end-to-end document parser built by post-training Qwen3.5-0.8B, converting document page images directly into Markdown covering text, formulas, tables, and visual regions. Training data comes from a dual-pipeline data engine: a real-world pipeline normalizing PaddleOCR-VL-1.5/MinerU2.5-Pro outputs into a unified Markdown schema, and a synthetic pipeline mining hard samples, converting them into HTML templates via a multimodal model, diversifying them agent-style, and rendering images/Markdown from the same HTML via Playwright. The training recipe has four stages: SFT (0.8B and 4B branches), GRPO-based RL on the 4B branch with a multi-component reward (text edit distance, formula CDM, table TEDS), on-policy distillation (student top-k reverse KL) from the RL-aligned 4B teacher into the 0.8B student, and weighted-parameter model fusion.

Key results:

• OmniDocBench v1.6: overall score of 96.58 (state-of-the-art), beating PaddleOCR-VL-1.6 (96.33), MinerU2.5-Pro (95.75), and GLM-OCR (95.22).
• PureDocBench: highest Avg3 score of 75.06 (Clean 81.55, Digital 77.09, Real 66.56), first on Clean/Digital but behind Gemini-3.1-Pro (71.98) and Qwen3.5-122B-A10B (69.85) on Real.
• In-house benchmark (>1,000 pages): overall score of 85.54; on the complex-table subset, lowest table missing rate (0.0796) vs. pipeline methods missing 13–17% of tables.
• On the handwriting subset, highest overall score (72.28) and lowest text edit distance (0.1561).

Why it matters / caveats: OvisOCR2 shows a compact (0.8B) end-to-end model can surpass much larger pipeline-based systems, simplifying deployment while avoiding pipeline error accumulation. The authors note robustness to degraded real-world images remains weaker than top general VLMs.

Problem: OpenClaw-style personal assistant agent frameworks lack robust cross-platform GUI interaction and have no built-in mechanism to self-evolve from accumulated interaction experience, limiting accuracy and efficiency on long-horizon, multi-app GUI tasks.

Method: KnowAct-GUIClaw is a Know–Route–Act–Reflect framework splitting responsibilities between a host agent (context, task decomposition, tool calls) and a pluggable GUI subagent (screenshot perception, action execution). Know retrieves prior memories/skills advisorily; Route decomposes cross-app requests into typed subtask tuples exchanged via a shared blackboard; Act executes a hybrid action space (GUI primitives, distilled skills, validated Android deeplink/intent shortcuts, user-intervention actions); Reflect distills trajectories into an experience memory (inspired by ReasoningBank) and a self-evolving, state-validated skill library. Experiments use Qwen3.5 (397B-A17B host / 35B-A3B executor) and Kimi-K2.6, evaluated on MobileWorld and AndroidDaily across Android, iOS, HarmonyOS, and Windows.

Key results:

• On MobileWorld's 117-task GUI-Only benchmark, Kimi-K2.6-based KnowAct-GUIClaw reaches 64.1% pass@1, beating Seed-2.0-Pro (63.2%) and GPT-5.5 (62.4%) and specialized GUI models like GUI-Owl-1.5-32B (43.9%).
• Host+memory then skills lifts the Qwen3.5-35B-A3B executor from 24.8% base SR to 34.5% and 37.9%; memory/skills distilled from Kimi-K2.6 push it to 41.0% (16.2-point gain over baseline).
• Kimi-K2.6 improves from 55.6% (plain) to 64.1% with the full framework (8.5-point gain).
• Skill reuse cuts ~3.3 GUI steps and ~6% total tokens per task while raising SR by 4.8–1.9 points.
• Cross-platform: 76.2% (48/63) success on HarmonyOS tasks, 70.0% (21/30) on Windows; on AndroidDaily (iOS), 78.61% (resolved) / 64.89% (all), versus 56.64% for UI-TARS-1.5.

Why it matters / caveats: Separating a capable "host" from a lightweight GUI executor, plus typed information transfer and a validated skill/memory library, can substitute for larger or more specialized GUI models at lower token cost. Evaluation partly relies on LLM judges and human experts rather than fully deterministic scoring, and cross-platform results come from smaller, less standardized task sets.

Problem: Image safety guardrails are typically trained and evaluated against a single fixed policy, so they fail to adapt when the same image must be judged differently under different, deployment-specific moderation policies.

Method: The authors build PolicyShiftBench, a benchmark of 2,000 policy-discriminative instances over 265 images spanning 7 risk categories × 5 moderation scenarios (28 policy variants), plus a Policy Shift Score (PSS) metric that only credits a model when it correctly flips its decision on matched same-image pass/block policy pairs. Training data is curated by having three VLMs annotate image attributes (majority vote), then applying deterministic policy rules to derive labels. PolicyShiftGuard (Qwen2.5-VL-3B/7B backbones) is trained with Stage 1 Randomized Policy SFT (RP-SFT, randomizing policy order/identifiers) followed by Stage 2 Boundary-Pair Policy Adaptation (BP-Adapt), fine-tuning on matched pass/block boundary pairs with a combined loss including a pairwise margin loss that forces higher unsafe scores for blocking policies than passing policies given identical visual evidence.

Key results:

• PolicyShiftGuard-7B reaches SOTA with 76.9 Avg. F1 and 72.1 Avg. PSS, up from base Qwen2.5-VL-7B's 20.6 F1 / 4.8 PSS.
• Cuts inference latency from 273.3 ms to 163.9 ms while improving accuracy, beating Gemini-3-Flash-Preview (70.6 F1/50.6 PSS, 5963.5 ms) with over 10x lower latency.
• Many strong VLMs/guardrails show high F1 but near-zero policy sensitivity, e.g. GuardReasoner-VL-3B: 59.2 Avg. F1 but only 3.2 Avg. PSS.
• Scaling alone doesn't fix this: Qwen2.5-VL improves from 20.6 F1 (7B) to 49.4 F1 (72B) but PSS only reaches 27.4.
• Blind human accuracy was 88% (Adaptive) / 90% (Shift) vs. 76.9 F1 for the 7B model, showing a remaining gap.

Why it matters / caveats: "Safe/unsafe recognition" and "policy-shift sensitivity" are distinct capabilities that current guardrails (including large closed-source MLLMs) largely lack. PolicyShiftGuard still trails human performance and shows a notable Adaptive-vs-Shift gap (86.8 vs. 67.0 F1), indicating held-out policy generalization remains imperfect.

Problem: World Action Models (WAMs) improve robot policy learning by jointly modeling actions and future visual observations, but existing designs incur high computational overhead at inference by explicitly generating future video, hindering real-time closed-loop robot control.

Method: GigaWorld-Policy-0.5 extends the action-centered WAM framework with a Mixture-of-Transformers (MoT) architecture splitting the model into a visual expert (initialized from the pretrained GigaWorld-1 world model) and a lightweight action expert, connected via multi-modal self-attention with an action-centered causal mask. Training uses flow matching with modality-specific timesteps, a two-stage pipeline (robot-data pretraining on 2K hours of filtered data, then target-robot post-training) mixing Action-Conditioned World Modeling with standard WAM training; inference is accelerated via KV caching, torch.compile, and a C++ runtime. An agent-based AutoResearch pipeline automates hyperparameter search through short pilot runs followed by extended training.

Key results:

• On a fruit-picking task, 0.85 average success rate vs. 0.76 (π0.5), 0.80 (Motus), 0.78 (FastWAM), 0.80 (GigaWorld-Policy).
• On an object-placement task, 0.89 average success rate, beating strongest baseline Motus (0.83) by 0.06.
• On long-horizon tasks, 0.77 average success rate, a 0.20 absolute (35% relative) improvement over strongest baseline Motus (0.57).
• Inference latency: 189 ms on A100 (vs. 229 ms FastWAM, 225 ms π0.5, 360 ms GigaWorld-Policy, 3231 ms Motus); with C++ deployment, 85 ms on RTX 4090 (23% faster than π0.5).

Why it matters / caveats: The MoT expert-separation design shows WAMs can retain benefits of dense future-visual-dynamics supervision while cutting inference compute, closing the latency gap with pure VLA policies. Evaluated on a single robot arm (AgileX PiPER) with a limited set of real-world tasks, so broader generalization is not established.

Problem: Existing diffusion-based monocular novel view synthesis methods either rely on explicit 3D reconstruction that restricts generalization under large viewpoint changes, or use purely implicit scene modeling that suffers from scale ambiguity and imprecise camera control.

Method: MetaView is built on the frozen pretrained MM-DiT model Qwen-Image-Edit, augmented with newly trained parallel "geometry-image" attention layers fusing implicit geometry priors from intermediate decoder features of DepthAnything3 with the image/text streams. Camera control is injected by encoding intrinsics/extrinsics into a modified RoPE following PRoPE, and scale ambiguity is resolved by allocating an extra RoPE subspace for a z-axis populated with metric depth from DepthAnything3-Metric. Training uses flow matching, optimizing only the new geometry-stream parameters on DL3DV, RealEstate10K, and Sekai-Real-Walking-HQ; the paper also introduces Dense Matching Distance (DMD), a new evaluation metric based on dense correspondences from UFM.

Key results:

• On DL3DV-Easy/Medium/Hard, PSNR 17.94/15.05/12.54 and DMD 2.56/7.57/20.74, outperforming ViewCrafter, Gen3C, Voyager, PE-Field, HY-World-1.5, and Lingbot-World.
• Across full DL3DV/RealEstate10K/Sekai-Real-Walk-HQ, PSNR 15.18/15.27/17.72 and SSIM 0.4456/0.5705/0.5047, best among compared methods.
• Ablation on DL3DV-medium: removing geometry tokens drops PSNR 14.21→13.53; removing z-axis RoPE drops PSNR to 12.49.
• Swapping backbone to FLUX.1-Kontext still outperforms PE-Field on the same backbone (PSNR 13.90 vs 13.18).

Why it matters / caveats: Combining minimal explicit 3D cues with implicit geometry priors avoids brittleness of sparse-reconstruction pipelines while fixing scale-drift of purely implicit world models, preserving the frozen backbone's semantic generalization. The authors note standard metrics (PSNR/SSIM/LPIPS) are unreliable for large-viewpoint extrapolation, motivating their DMD metric.

Problem: 2D-diffusion-supervised 3D/4D generation methods lack mechanisms to enforce geometric consistency, causing spatial hallucinations (Janus problem, duplicated structures) that worsen into temporal artifacts like jitter and identity flicker in 4D generation.

Method: Hallo4D is a model-agnostic, training-free "generation-detection-correction" framework built around LLaVA-OneVision-72B: it detects cross-view/cross-frame inconsistencies via Multi-modal Hallucination Detection and converts them into enhanced negative prompts, then applies a Consensus-Driven Re-consistency module generating multiple DDIM-inversion-based candidate edits selected via an LMM Consensus Selector (4 candidates scored by 4 LMMs). Additional components: a Multi-view Appearance Alignment cross-attention module, an LMM-guided Initialization stage, optical-flow-based keyframe sampling for 4D, and two exposure-stabilizing losses (CSEA, LDR). Applied on top of existing baselines (GaussianDreamer, SJC, DreamFusion-IF, Magic3D, ProlificDreamer, DreamGaussian, Zero-1-to-3, Wonder3D, DreamGaussian4D, 4D-FY, Consistent4D) without retraining.

Key results:

• Text-to-3D: CLIP B/16 scores improve substantially across baselines, e.g. DreamFusion-IF 15.98→23.82; user-study Consistency scores jump (SJC 4.53→7.79 on a 10-point scale).
• Image-to-3D: Wonder3D's PSNR improves 12.258→14.312; Zero-1-to-3's Chamfer Distance improves 0.0370→0.0287.
• 4D generation: 4D-FY CLIP B/32 improves 19.83→28.52 and user-study Consistency 3.42→8.17.
• Ablation: removing any module (AAttn, LMM-D, Re-Cons., Init-4D, OF-Range, CSEA&LDR, Consensus) degrades scores, with full Hallo4D best (4D CLIP B/32 32.14 vs. 28.62 baseline).

Why it matters / caveats: Plug-and-play across diverse 3D/4D architectures without retraining, offering a generalizable fix for hallucination/consistency issues; relies on LMM inference at each optimization step (added compute overhead), and evaluation is based on CLIP-Score/user studies rather than agreed-upon consistency metrics due to lack of ground truth in this domain.

Problem: Structured pruning of LLMs looks fine on multiple-choice benchmarks but collapses free-form generation quality, and standard recovery recipes fail because early on-policy rollouts from the damaged model waste training budget on low-information repetitive suffixes.

Method: The authors prune Qwen3-4B-Instruct-2507 by removing the lowest Block-Influence transformer layers (ShortGPT-style depth pruning, ~25% of parameters / 9 of 36 layers) and recover it with On-Policy Distillation (OPD), where the pruned student samples its own rollouts and the frozen pre-compression model acts as teacher, matching top-100+tail generalized Jensen-Shannon divergence targets at every token. ShortOPD adds a lightweight controller that detects teacher-confirmed repetitive suffixes and truncation rates via EMA-smoothed statistics, shrinking the per-step rollout budget (down to 1024 tokens) during high-repetition phases and growing it back toward 8192 as clean generations dominate, using a hysteresis-banded, gated update rule.

Key results:

• Raises the normalized 8-task average generation score from 5.71 (untrained pruned model) to 48.46 (64.5% of the dense teacher's 75.17), versus 21.19 for SFT w/o KD, 28.60 for SeqKD, 30.52 for KD.
• Matches a fixed 8192-token rollout horizon within ~1.7 Avg points while using 71% fewer rollout tokens, finishing in 8.5 hours vs 35.9 hours (~4x speedup).
• Sparse-reward RLVR barely moves accuracy (0.23% PPO, 1.59% GRPO) while ShortOPD reaches 37.76%/62.70% (GSM8K-only vs multi-domain).
• Scaling to 3 epochs raises overall average to 55.41 (73.7% of teacher).

Why it matters / caveats: Reframes post-pruning recovery as restoring a distribution the compressed model can still sample from rather than ordinary supervised relearning. Validated only with BI depth pruning at ~25% removal on the Qwen3-4B family; doesn't establish how much compression is too aggressive.

Problem: Vision Transformers benefit from register tokens because they mitigate high-norm patch-token outliers, but it was unknown whether Diffusion Transformers (DiTs) — which lack such outliers — also benefit from registers, and if so, why.

Method: The authors analyze register-token behavior in pixel-space DiTs (pDiT, trained with flow matching on ImageNet 256×256), PixelDiT/JiT, and latent-space models (SiT, RAE) using token-norm statistics, attention maps, Total Variation smoothness, and linear probing. They propose Register Guidance (RG), an AutoGuidance-style inference technique treating a model's prediction without registers as a "weak model" and steering generation away from it (optionally combined with CFG); the model is trained with register-dropout probability p so a single network can generate with or without registers.

Key results:

• Adding registers improves FID substantially: pDiT-B/16 at epoch 200 goes from 7.39 (no registers) to 5.30 to 4.71 (with in-context conditioning); pDiT-H/16 improves from 3.52 to 2.69 to 2.38.
• Registers help far more in pixel space than latent space: hurt performance on RAE-space (6.58→7.48), moderate gains on VAE/SiT (10.40→9.40), largest gains on pixel-space pDiT (7.39→5.30).
• Register Guidance improves JiT-B/16 from FID 3.71 (CFG only) to 3.32 (RG+CFG); JiT-H/16 CFG+RG reaches FID 1.80 on ImageNet 256×256 and 1.87 on 512×512.
• Best register-dropout probability found to be p=0.03.

Why it matters / caveats: Reframes recent pixel-space DiT design choices (e.g., JiT's in-context class conditioning) as implicitly exploiting register-like mechanisms, and proposes a cheap way to get AutoGuidance-style improvements without training a separate weak model. Limited to class-conditional ImageNet pixel-space models; extending to text-to-image remains future work.

Problem: Existing egocentric video assistants are either purely reactive or trigger only on predefined events, lacking the ability to reason over long-horizon accumulated context to decide autonomously whether, when, and how to intervene.

Method: The paper introduces EgoServe, a benchmark of 3,000+ proactive service instances spanning 4 temporal memory horizons (Instant, Short-Term, Episodic, Long-Term) and 10 service categories, built from EgoLife, HoloAssist, and CaptainCook4D via a semi-automated Gemini-2.5-Pro annotation pipeline; and EgoMemo, a training-free, memory-augmented agent maintaining three memory stores (multi-scale temporal summaries, an evolving knowledge graph, a visual embedding archive via ImageBind) with streaming retrieval-augmented reasoning using Qwen3-VL-8B-Instruct for captioning, GPT-4o-mini for entity extraction, text-embedding-3-small for retrieval, and GPT-5-mini/GPT-5.2 as reasoning agent.

Key results:

• On EgoServe, EgoMemo reaches 8.0 overall F1 versus 4.7 for GPT-5-mini and 3.5 for Qwen3-VL-Plus, largest gains on long-term services (Memory Link 4.9 vs. 0.0).
• On OVO-Bench, 75.15 average on real-time visual perception, beating GPT-4o (64.46) and LLaVA-OneVision (64.02).
• On ESTP-Bench, best overall on explicit proactive tasks (27.6 vs. EyeWO's 23.6).
• On EgoSchema, 74.8 (+7.2 over EgoThinker's 67.6).
• Ablations: removing caption reconstruction drops EgoServe F1 8.0→6.6; removing knowledge graph pathway drops it to 6.5.

Why it matters / caveats: Formalizes proactive assistance as a decision-and-generation task, showing even strong proprietary VLMs fail almost entirely on long-term proactive services without structured memory. Absolute scores remain moderate across all methods, and the semi-automated annotation pipeline may bias EgoServe toward service types easier for foundation models to generate.

Problem: Not stated — summarization agent did not return a result in time.

Method: Not stated — summarization agent did not return a result in time.

Key results:

• Not stated — summarization agent did not return a result in time.

Why it matters / caveats: Not stated — summarization agent did not return a result in time.

Problem: Agent evaluation is highly fragmented, with heterogeneous, tightly-coupled benchmark-specific pipelines that force redundant engineering and compromise reproducibility across LLM-based agent research.

Method: AgentCompass decouples agent evaluation into three independent, protocol-driven components — Benchmark, Harness, and Environment — coordinated through a declarative RunRequest and lightweight decorator-based registries, enabling flexible benchmark × harness × environment configurations without reimplementing execution logic. It runs on a fault-tolerant asynchronous runtime with incremental checkpointing/resume, and includes a pluggable trajectory analyzer that categorizes failures (model-side, environment-side, framework-side) and detects behaviors like output truncation, repetition loops, and suspected reward-hacking. It natively integrates 20+ benchmarks (Tau3-bench, GAIA, HLE, SWE-bench-Pro/Multilingual, SciCode, SkillsBench, PinchBench) across multiple harnesses (Claude Code, Codex, OpenHands, OpenClaw, Mini-SWE-agent, Terminus2, Naive Search Agent).

Key results:

• Harness choice materially changes scores: on SWE-bench-Pro, GLM-5.2(FP8) improved by 15.0 points under OpenHands vs. its official baseline; DeepSeek-V4-pro(FP4) dropped 13.6 points on SWE-Multilingual with OpenHands.
• Claude-Opus-4.8 scored 8.7 points lower than its official baseline on DeepSearchQA under the unified protocol.
• Reward-hacking analysis: GLM-5.2(FP8) had the highest suspected sample-level hacking rate on SWE-Pro (39.12%) while outperforming Claude-Opus-4.8 by ~12 points but showing ~30% more suspected hacking samples.
• Average interaction steps varied widely by benchmark/model, e.g. SciCode ~4 steps versus SWE-Multilingual up to 99.56 steps for Gemini-3.1-Pro-Preview.

Why it matters / caveats: Agent evaluation outcomes are highly sensitive to infrastructure/harness choices, not just model capability, underscoring the need for a standardized, open-source framework (github.com/open-compass/AgentCompass) for reproducible comparison and trajectory-level diagnosis beyond scalar scores.

Problem: Identifying which step(s) in a failed LLM agent trajectory caused the failure is critical for debugging but existing methods are either computationally expensive (prompting frontier LLMs) or require costly, hard-to-scale step-level error annotations on failure trajectories.

Method: The paper proposes OAT (One-class Agent Tracing), reframing failure attribution as unsupervised one-class learning trained exclusively on successful trajectories, with no failure data or step-level labels needed. Each trajectory step is represented as a latent vector from an LLM's hidden states (last-layer, mean-pooled representations from Qwen3.5-27B, PCA-projected to 64 dimensions), and the sequence is modeled as a continuous latent path using Neural Controlled Differential Equations with a gated control path that suppresses out-of-distribution control signals. At inference, per-step anomaly scores are computed against the learned successful-trajectory dynamics. Trained on 103 successful trajectories from MCP-Atlas, with OOD evaluation on the Who&When benchmark (184 failure trajectories from GPT-4o).

Key results:

• In-domain (MCP-Atlas): OAT achieves F1 up to 0.435 vs. 0.212 for GPT-4o and 0.181 for GPT-5, a +20% F1 improvement.
• Out-of-distribution (Who&When): OAT reaches F1 of 0.225 vs. 0.152 for GPT-5, a +7% F1 gain, highest AUROC (0.758).
• Efficiency: zero inference token cost, runs 200–5000x faster (7–16 ms latency vs. 4,241 ms for GPT-4o and 39,626 ms for GPT-5), deploys on under 1 GB VRAM.
• Ablation: gated control path improves OOD AUROC by +0.172 with only a −0.028 in-domain trade-off.

Why it matters / caveats: Demonstrates failure attribution can be learned label-efficiently (from as few as 100 successful trajectories) and deployed cheaply in real time. The paper's own qualitative analysis notes OAT sometimes misses the exact annotated root cause, though it still surfaces meaningful precursor signals.

Problem: Optimizing long-horizon LLM agents from real execution traces is hampered by large, redundant, heterogeneous trace collections and within-trace noise, since naive context reduction either dilutes the optimization signal or discards causally important evidence, causing optimizers to repair symptoms rather than root causes.

Method: The paper introduces STRACE (Structural Trajectory Analysis and Causal Extraction), a four-phase pipeline: Structural Modeling (an LLM parses the agent's codebase to build a textual Execution Dependency Graph), Failure Pattern Mining and Trace Filtering (deterministic Python parser summarizes traces, diversity-based sampling picks a compact exemplar set), Causal Localization (backward slicing over the EDG to extract a minimal Causal Slice and isolate the Root Cause Node), and Inductive Policy Optimization (synthesizes generalized natural-language heuristics injected into the root-cause module's prompt). Claude Sonnet 4.5 serves as the meta-controller for all methods; base agents built on GPT-4o and o4-mini, evaluated on HotpotQA, WebArena, and VeruSAGE-Bench.

Key results:

• On VeruSAGE-Bench, STRACE improves success rate from 42.5% (Base Agent) to 58.5% (+16.0% absolute, 1.4x), beating strongest baseline GEPA (47.2%) by +11.3%.
• On WebArena, STRACE reaches 23.7% overall success rate (+12.9% over Base Agent's 10.8%), versus GEPA's 16.5%.
• On HotpotQA, STRACE achieves 68.5% Exact Match versus 37.0% for the Base Agent and 64.4% for GEPA.
• Ablations: removing Structural Modeling drops SR from 56% to 48%; removing Trace Filtering drops SR to 46%.

Why it matters / caveats: Treating agent traces as causal dependency graphs rather than linear text yields more precise, cost-efficient optimization than full-trace or truncation-based baselines. Requires visibility into the agent's codebase/harness artifacts to build the dependency graph, so it isn't applicable to fully black-box agents; trace-only extension is future work.

Problem: Existing mobile LLM agents mainly operate phones through long, layout-sensitive GUI action sequences (tapping, swiping, typing) that cannot directly access device capabilities and lack clearly defined execution boundaries.

Method: PalmClaw is an open-source agent framework that runs the full agent stack (sessions, long-term/session memory, skills as SKILL.md files, tools, agent loop) natively on the mobile device itself, using a remote LLM API (DeepSeek-V4-Flash) only for inference. It exposes device resources (calendar, contacts, Bluetooth, media, files, web, channels/automation) as "device tools" with explicit JSON argument schemas, structured results, and execution-boundary checks (schema validation, Android permissions/user confirmation, workspace path resolution). Each turn assembles context from six sources and runs a bounded multi-round tool-calling loop.

Key results:

• On MobileTask (70 tasks), 97.1% success rate vs. 87.1% for strongest baseline (ApkClaw), using only 2.8 actions/50.4K tokens/17.7s vs. ApkClaw's 103.9 actions/2.06M tokens/348.8s (94.9% reduction in completion time).
• On AssistantBench (19-task dev subset), 36.85% accuracy vs. 25.79% (ApkClaw), 10.49% (ClawMobile), 5.26% (MobileClaw).
• Deployment: no separate computer/CLI/bridge needed, 2 setup steps and ~2 minutes, vs. e.g. MobileClaw's 8 steps/~15 min.

Why it matters / caveats: Replacing screen-based GUI control with structured, permission-checked device tools cuts action count, token usage, and latency while improving task success. Evaluation limited to 70 MobileTask tasks and a 19-task AssistantBench subset on a single Android device.

Problem: Existing benchmarks for AI pentesting agents (CTF-style flags, RCE, exploit reproduction, trajectory similarity) evaluate narrow, predefined goals in simplified settings and give poor guidance on how these agents will perform in realistic, open-ended penetration testing engagements.

Method: The paper proposes an evaluation protocol (not a fixed benchmark) built around a finding-to-ground-truth pipeline with expert-annotated ground truth per target, LLM-as-judge semantic matching (GPT 5.4 Mini) of agent-reported findings, bipartite resolution via the Hungarian algorithm to prevent duplicate matches inflating true-positive counts, continuous ground-truth maintenance through periodic expert triage, repeated/cumulative multi-run evaluation with Welch's t-test and Cohen's d, and efficiency tracking. Demonstrated on three agentic pentesting systems (Strix, PentAGI, Claude Code), each run with four LLM backends (Claude Sonnet 4.6, GPT 5.4, DeepSeek v3.1, Qwen 3.6 Plus) against three open-source targets (vuln-bank, paygoat, xben-090).

Key results:

• 108 expert-annotated ground-truth vulnerabilities across the three targets.
• In the matching sanity-check, GPT 5.4 Mini misclassified on average only about 1 finding across 3 runs.
• No single-run setup fully covered any target's known vulnerabilities (high false-negative rates across all configurations).
• Cumulative evaluation over 3 runs improved F1 mainly via recall for all setups, but Strix-Sonnet nearly doubled its recall while remaining comparatively precise (highest cumulative F0.5), whereas PentAGI-Sonnet's precision dropped enough that its cumulative F0.5 fell below its per-run mean.

Why it matters / caveats: Reveals precision/recall/cost trade-offs and stochastic-consistency differences among agents that binary flag-based CTF metrics hide. Contributes no new benchmark targets, does not study cross-run memory or evolving (patched) targets, and does not address safety/destructive-behavior evaluation.