AI papers — 2026-05-04

Problem: Existing video diffusion models train separate networks per task with fixed input-output modality mappings, preventing cross-modal correlation and requiring redundant model families.

Method: UniVidX is a unified multimodal video generation framework built on the Wan2.1-T2V-14B backbone with three novel components. Stochastic Condition Masking (SCM) randomly partitions modalities into clean conditions and noisy targets at training time, enabling omni-directional generation across Text→X, X→X, and Text+X→X paradigms. Decoupled Gated LoRA (DGL) attaches per-modality rank-32 LoRA adapters (385M trainable params) that activate only when a modality is the generation target, preserving VDM priors. Cross-Modal Self-Attention (CMSA) shares keys and values across modalities while keeping modality-specific queries for inter-modal alignment. Two instantiations are released: UniVid-Intrinsic (RGB + albedo/irradiance/normal) trained on 924 videos, and UniVid-Alpha (blended RGB + RGBA layers) trained on 484 videos.

Key results:

• Normal estimation: Mean Angular Error 11.09°, beating specialized NormalCrafter trained on 45× more data (860K vs 19K frames)
• Video matting: MAD 4.24, Grad 1.86 (state-of-the-art)
• Albedo inverse rendering: PSNR 16.89 dB, LPIPS 0.2248, SSIM 0.7812
• Temporal consistency (RGB): Flickering score 0.9876
• Both models trained on <1,000 videos each

Why it matters / caveats: Demonstrates extreme data efficiency by leveraging strong VDM priors instead of task-specific training, enabling 15 distinct conditional generation tasks in two models. Currently limited to 4 modalities, 21 frames, 480p resolution due to memory constraints; fails on transparent glass and semi-transparent mattes not in training distribution; requires a 14B backbone with substantial VRAM.

Problem: Agentic web search systems fail at structured extraction tasks requiring schema-aligned outputs across hundreds of entities from heterogeneous sources, while single-agent systems saturate context with error compounding.

Method: Web2BigTable is a bi-level multi-agent framework combining an upper-level orchestrator that decomposes queries into sub-problems and up to 10 parallel lower-level worker agents that resolve them. Coordination uses a shared Markdown workboard with read-write asymmetry, allowing workers to see peer progress and avoid redundant exploration. A closed-loop Run–Verify–Reflect training pipeline distills execution errors into monotonically-growing SKILL.md files (8,000+ cloud skills retrieved via BM25 + ChromaDB embeddings), with no parameter fine-tuning — all learning is skill-level external memory. Backbone: GPT-5 mini + Gemini 3 Flash.

Key results:

• WideSearch Success Rate: 38.50 (7.5× second-best at 5.10)
• WideSearch Row F1: 63.53 (+25.03 over second-best)
• WideSearch Item F1: 80.12 (+14.42 over second-best)
• XBench-DeepSearch accuracy: 73.0% (vs. frontier single-agent at 72.0%)
• Removing learned orchestrator skills alone drops Success Rate from 38.50 → 7.00

Why it matters / caveats: Shows that bi-level decomposition + workboard coordination + runtime skill evolution outperform frontier single-agent systems on structured extraction without any gradient updates. Evaluation relies on proprietary APIs (GPT-5 mini, Gemini 3 Flash), limiting reproducibility; baseline discrepancies noted on XBench-DeepSearch suggest setup sensitivity.

Problem: Generalist Vision-Language-Action (VLA) policies pretrained offline fail to adapt to distribution shifts, long-tail failures, and novel task variations encountered during real-world deployment.

Method: Learning While Deploying (LWD) is a fleet-scale offline-to-online RL framework for continual post-training of VLA policies. Distributional Implicit Value Learning (DIVL) replaces scalar expectile regression (IQL) with a categorical distribution over action-values, using τ-quantile bootstrap targets with adaptive τ scheduling based on distributional entropy. Q-learning via Adjoint Matching (QAM) reformulates critic gradient backpropagation into stable step-wise supervision along reference flow trajectories, enabling policy extraction from flow-based VLA action generators without expensive multi-step denoising. Architecture uses Gemma-3 + SigLIP backbones; the VLM backbone is frozen during online updates. Deployment loop: offline pretraining → rollout collection across 16 dual-arm robots → buffer mixing → policy update → redeployment.

Key results:

• Average success rate across 8 real-world tasks: 95%
• Long-horizon task step-wise score: 0.91 (SFT: 0.68, RECAP: 0.77, HG-DAgger: 0.73)
• DIVL alone: +9.7% improvement over expectile regression on long-horizon tasks
• QAM online stage: +16.7% improvement over scalar baseline
• Cycle time reduction: 23.75 seconds faster mean execution vs. reference policy
• ~60 hours of fleet data collected within a 4-hour wall-clock window

Why it matters / caveats: Creates a self-improving data flywheel: deployed robots generate training data that updates shared policies at fleet scale, with the largest gains on complex 3–5 minute manipulation tasks. Lacks explicit safety modeling; online update scheduling is simple and may not scale optimally to larger fleets; grocery restocking baseline was already near ceiling, limiting measurable improvement there.

Problem: Existing 3D world generation methods are constrained to grid layouts and produce inconsistent object scales across large scenes, with no support for user-defined irregular boundary shapes.

Method: Map2World generates 3D worlds conditioned on arbitrary-shape segment maps via a pipeline built on TRELLIS (a pretrained 3D asset generator). It uses Latent Fusion for Rectified Flow with overlapping 64³-voxel windows (50% overlap) and Gaussian-weighted velocity aggregation to merge cube-wise generations into coherent global scenes. Segment-Map-Guided Conditioning applies multi-label weighted velocity fusion with time-dependent Gaussian smoothing (σ_t) for stable region boundaries. For sparse structures, Spectral-Domain Optimization uses FFT parameterization of initial latents. A Detail Enhancer Network — an MLP layer concatenating noise with condition latents, feeding frozen TRELLIS flow Transformers — adds fine-grained detail without disrupting global structure. Trained on 16,000 cube pairs from 35 filtered Objaverse scenes.

Key results:

• World Quality (WQ) score: 7.76/10 vs. SynCity 7.25/10 and GaussianCube 5.08/10
• GPTScore average: 7.93/10 (SynCity: 7.48/10)
• Detail Enhancer: PSNR 22.53, LPIPS 0.2137, FID (Inception v3) 16.98
• Spectral parameterization: IoU/Dice ≈0.9 achieved in 5 optimization steps

Why it matters / caveats: First method to generate 3D worlds from arbitrary-shape semantic segment maps with global scale consistency, enabling district-boundary-aware city-scale synthesis. Inherits TRELLIS's absolute position encoding, causing structure drift when merging cubes; only trained on simple Objaverse meshes so photorealism is limited; no classifier-free guidance during sampling due to performance gap.

Problem: LLM agent skills are stored as text-heavy SKILL.md artifacts where invocation interfaces, execution structure, and side-effect evidence are entangled in natural language, making skill discovery and risk assessment expensive and imprecise.

Method: The Scheduling-Structural-Logical (SSL) representation is a three-layer JSON schema inspired by Schank & Abelson's Memory Organization Packets, Script Theory, and Conceptual Dependency. The Scheduling Layer captures skill-level interface signals, goals, and dependencies. The Structural Layer is a directed graph of execution "scenes" (typed phases: PREPARE, ACQUIRE, REASON, ACT, VERIFY, RECOVER, FINALIZE). The Logical Layer encodes atomic actions using closed vocabularies (READ, SELECT, INFER, WRITE, CALL_TOOL, etc.) with resource scopes (MEMORY, LOCAL_FS, CREDENTIALS, NETWORK). An LLM-based normalizer converts SKILL.md files to SSL via four passes: skill extraction, scene decomposition, logic-step expansion, and validation. All evidence remains grounded in source artifacts with no inferred behavior.

Key results:

• Skill Discovery MRR: 0.573 (text-only) → 0.707 (+23.4%) with SSL
• Skill Discovery NDCG@10: 0.610 → 0.750
• Risk Assessment macro F1: 0.744 → 0.787 (+5.8%)
• Best per-dimension gains on data exfiltration, destructive behaviors, credential access

Why it matters / caveats: Makes agent skill collections machine-searchable and auditable without re-parsing free-text documentation each time. Cannot detect dynamic runtime actions (downloaded payloads, constructed shell commands); LLM normalizer may omit facts or over-regularize underspecified skills; risk labels come from model-mediated voting rather than expert audits.

Problem: Existing CLIP-style contrastive pretraining for vision encoders uses dual-encoder architectures misaligned with the autoregressive nature of LLMs, requiring large batch sizes and producing representations suboptimal for multimodal LLM integration.

Method: GenLIP is a minimalist single-transformer generative pretraining framework for ViTs. It concatenates image patches and text tokens into one sequence ([v₀…vₘ, t₀…tₗ]) processed by a single Transformer using Prefix-LM Attention (bidirectional for image tokens, causal for text). Multimodal Rotary Position Encoding (MRoPE) handles the concatenated visual-linguistic sequence. A Gated Attention Mechanism (G = σ(XWg + bg) with element-wise multiplication) prevents attention sink — information collapsing into early tokens — stabilizing training. Training objective is standard autoregressive language modeling on text only. Two-stage training: Stage 1 on 8B samples from Recap-DataComp-1B at 224×224; Stage 2 on 37M samples (Infinity-MM + BLIP3o captions) at native aspect ratios.

Key results:

• Frozen features (Qwen2.5-1.5B): GenLIP-g/16 achieves 65.2 ALL AVG vs. SigLIP2-g/16 61.5, using 5× less data (8B vs. 40B)
• Doc&OCR average: 53.2 vs. SigLIP2's 47.3 (+5.9 points)
• LLaVA-NeXT setting: GenLIP-So/16 at 70.3 ALL AVG (729 patches)
• ImageNet-1K: 85.2% vs. CLIP 85.1% (SigLIP2: 88.9%)
• Consistent data scaling gains 1B → 8B samples

Why it matters / caveats: Shows that removing contrastive complexity — no batch construction, no separate text decoder — achieves competitive or superior multimodal benchmark results with 5–75× less pretraining data. Validated only in LLaVA-NeXT academic setup; discriminative performance lags SigLIP2 on ImageNet by 3.7 points, indicating a generative/discriminative trade-off.

Problem: Autoregressive image generation with 2D grid tokenizers creates bidirectional token dependencies misaligned with causal AR modeling, and two-stage training that freezes the tokenizer before generative supervision prevents it from learning generation-optimized representations.

Method: EOSTok introduces a 1D Vision Transformer Tokenizer that flattens 2D patches and concatenates them with learnable query tokens, passed through a causal ViT encoder with IBQ (Index-Back Quantization). A symmetric decoder reconstructs images from mask tokens. The key innovation is end-to-end joint training with four loss components: reconstruction loss (L1/L2 + perceptual + GAN), next-token prediction for AR supervision, an Autoregressive Prediction Reconstruction (APR) loss that decodes teacher-forcing AR predictions back to pixel space to directly supervise tokenizer from generation quality, and a semantic alignment loss using implicit alignment of hidden patch embeddings to DINOv2 features (without enforcing 2D structure).

Key results:

• ImageNet 256×256 gFID: 1.48 without guidance (EOSTok-H, 644M params) — state-of-the-art for AR models
• ImageNet 512×512 gFID: 1.98 without guidance
• Reconstruction rFID: 0.71–0.74 across model sizes
• Codebook utilization: 99.7% (vs. 51.8% without APR loss)
• Sampling speed: 20–100× faster than DiT-XL/2

Why it matters / caveats: Demonstrates that pixel-space feedback during joint tokenizer-generator training substantially closes the gap between discrete token prediction loss and visual quality. Training overhead is ~15–18% vs. two-stage; performance is sensitive to nested dropout probability (optimal at 0.5) and sequence length.

Problem: Decision trees and diffusion models are treated as entirely separate model classes despite both operating as generative models, preventing cross-paradigm knowledge transfer and missing a shared theoretical foundation.

Method: The paper establishes a formal mathematical correspondence between hierarchical decision trees and diffusion/flow processes in appropriate limiting regimes, identifying Global Trajectory Score Matching (GTSM) as the shared optimization principle underlying both. It shows that idealized gradient boosting is asymptotically optimal for GTSM. Two practical instantiations: TreeFlow, which uses the tree-diffusion duality for tabular data generation, and DSMTree, a distillation method that transfers hierarchical decision logic into neural networks.

Key results:

• TreeFlow: competitive tabular generation quality with 2× computational speedup over diffusion baseline
• DSMTree: matches teacher model performance within 2% on many benchmarks
• Not stated (full quantitative tables require paper access; HTML version unavailable)

Why it matters / caveats: Reveals a fundamental mathematical bridge between two important model families, enabling efficient knowledge distillation from tree to neural models. Practical scope currently limited to tabular data generation and tree-to-neural distillation; broader applicability to other domains is not demonstrated.

Problem: VLMs trained on supervision from stronger teacher models develop a Supervision-Perception Mismatch — learning to reproduce fine-grained details beyond their perceptual capacity — causing hallucinations that offline distillation cannot fix.

Method: OSCAR (Online Self-CAlibRation) combines Monte Carlo Tree Search (MCTS) at sentence-level granularity using the PUCT selection criterion, a Dual-Granularity Reward Mechanism comprising a discriminative process reward ("Is there hallucinated content?" → P("No")) and a gated outcome reward that evaluates full trajectories only when extracted objects match ground truth, and Direct Preference Optimization (DPO) trained on preference pairs extracted from MCTS trees at both global-path and sibling levels. The framework iterates: each model version generates new MCTS-based preference data for the next DPO round.

Key results:

• LLaVA-1.5-7B CHAIR_s: 49.0 → 27.6 (−43.7% hallucination rate, 3 iterations)
• LLaVA-1.5-7B CHAIR_i: 14.3 → 8.2 (−42.7%)
• AMBER Hal: 31.2 → 17.2 (−44.9%)
• LLaVA-1.5-13B CHAIR_s: → 5.4 (−87.9%)
• MM-VET general capability: +6.5% (+2.1 absolute)
• Teacher-distilled fine-tuning confirmed to worsen performance vs. OSCAR

Why it matters / caveats: Demonstrates that self-improvement respecting a model's own perceptual boundaries outperforms offline teacher distillation, achieving SOTA on hallucination benchmarks while preserving general multimodal capability. MCTS computational overhead vs. baselines is unreported; process rewards rely on discriminative prompts that may carry their own biases; gated outcome reward requires COCO object annotations.

Problem: Distributed black-box consensus optimization with multi-agent systems relies on handcrafted update rules and static cooperation patterns that struggle to balance local adaptation, global coordination, and communication efficiency across heterogeneous nonconvex environments.

Method: LAC-MAS (Learning to Act and Cooperate for Multi-Agent Systems) combines an Adaptive Swarm Execution Layer — agents maintain particle populations with divergence-based adaptive coefficients selecting from three behavioral modes (wi,1, w₀, wi,2) based on population dispersion Di(t) — with LLM-Guided Trajectory Learning using DeepSeek-R1:14B locally. The LLM provides sparse guidance on two tracks: Learning to Act (refining internal velocity coefficients from historical trajectories) and Learning to Cooperate (adjusting row-stochastic neighbor mixing matrices A(t) via trajectory descriptors of fitness, divergence, and state variation). A Phased Cognitive Guidance (PCG) scheduler controls refresh timing at calibrated horizon T.

Key results:

• 10 standard benchmark functions, 100 dimensions, 20 agents
• F1 mean fitness: 2.21E+04 vs. MASOIE 6.81E+04 (67.5% improvement)
• F5: 1.55E+07 vs. 2.14E+07 (27.6% improvement, p<0.01)
• F4: no statistically significant improvement (p=0.245)
• Ablation: act-only +46.96%, coop-only +15.56%, combined +67.54% on F1
• WSN localization: consistently lower estimation error at Nt=3–7 targets

Why it matters / caveats: Shows that LLM-guided trajectory learning can improve distributed swarm optimization without gradient access or parameter training. Gains are modest or non-significant on some functions (F3, F4); requires local LLM deployment with unquantified compute overhead; PCG scheduling requires pre-experiment horizon calibration; only tested on fixed communication topologies.

Problem: Off-the-shelf speaker encoders conflate speaker identity with language/script, causing the same voice to map to different embeddings when audio switches between scripts — WavLM-base-plus-sv loses 0.082 cosine similarity and ECAPA-TDNN loses 0.105 on Western-accented cross-script pairs.

Method: LASE is a projection head over a frozen WavLM-base-plus backbone producing T×768 frame features, followed by a two-layer MLP (768→512→256, ReLU) outputting 256-dimensional embeddings. A gradient-reversal language classifier with reversal weight −λ_t enforces language-invariance while a supervised contrastive loss preserves speaker identity across scripts. Trained on 1,118 quality-gated synthetic cross-script pairs from 8 commercial multilingual voices (ElevenLabs Multilingual v2) spanning English, Hindi, Telugu, and Tamil.

Key results:

• Western-accented residual cross-script gap: 0.082 → 0.013 (84.3% reduction; 95% CI [−0.02, 0.05] includes zero)
• Cross-script margin amplification: 0.245 → 0.662 (2.7× over WavLM-SV baseline)
• Indian-accented gap: 0.026 (95% CI [−0.04, 0.08])
• Diarization cross-script speaker recall: 0.788 vs. ECAPA-TDNN 0.789 (matched at 100× less training data)

Why it matters / caveats: Enables practical cross-script voice cloning and diarization for code-switching Indian-language scenarios (customer support, healthcare) with minimal training data. All evaluation uses synthetic speech only — real-world cross-script speech generalization unvalidated; held-out set overlaps 8 training voices so new-voice generalization is deferred; ECAPA still outperforms on general speaker discrimination (ARI 0.693 vs. 0.640).

Problem: Existing dual-branch diffusion transformers for joint audio-video generation couple modalities throughout the entire denoising process, unnecessarily entangling high-level semantics with low-level signal rendering and producing fixed-length outputs that truncate speech.

Method: Talker-T2AV uses a two-stage autoregressive diffusion architecture. Stage 1: a shared autoregressive language model backbone processes temporally aligned audio-video patches at 25 Hz, fusing embeddings via element-wise summation per position and using Patch Transformer Encoders that compress 4 consecutive frames into single tokens for left-to-right generation. Stage 2: two independent lightweight Diffusion Transformer heads use optimal-transport conditional flow matching to decode shared hidden states into modality-specific audio (via WhisperX-VAE) and video (via LIA-X motion autoencoder) latents. An MLP stop predictor enables variable-length generation.

Key results:

• Joint Text-to-Audio-Video WER: 0.055 (UniAVGen baseline: 0.302)
• Joint Text-to-Audio-Video FVD: 246.39 (baseline: 298.27)
• SyncNet Confidence: 6.330 (baseline: 2.555)
• Video Dubbing (Chem benchmark) WER: 6.33% (previous SOTA: 12.60%)
• Video Dubbing UTMOS: 3.256 (best)

Why it matters / caveats: Shows that decoupling joint semantic reasoning from modality-specific rendering beats fully-entangled architectures on lip-sync accuracy, video quality, and speech clarity. Autoregressive error accumulation degrades quality on long sequences; video fidelity is bounded by LIA-X autoencoder capacity; trained on ~1M clips with further scaling expected to improve.

Problem: Analog circuit IP reuse is blocked by heterogeneous representations (SPICE netlists, schematics, text descriptions) with no cross-modal retrieval system, forcing engineers to rely on manual single-modality keyword matching.

Method: AnalogRetriever is a tri-modal retrieval framework mapping SPICE netlists, schematics, and text descriptions into a shared embedding space. Schematics and descriptions are encoded with CLIP (ViT-L/14) with the bottom 16 of 24 ViT blocks frozen. Netlists are encoded by a Port-Aware Relational Graph Convolutional Network (RGCN) with 20 hand-defined edge types capturing MOSFET terminal roles (drain/gate/source/bulk). All modalities are aligned via InfoNCE-style contrastive loss across all six cross-modal directions, augmented by auxiliary classification across 19 circuit topology categories. Training uses three-phase curriculum learning: RGCN warm-up with frozen CLIP, transition unfreezing CLIP, and hard-negative mining with ratio α increasing 0.05→0.3. The Masala-CHAI base dataset was repaired from 22% to 100% netlist compile rate via a two-stage pipeline, yielding 6,354 verified triplets.

Key results:

• Average Recall@1 across 6 cross-modal directions: 75.2%
• Text→Code Recall@1: 75.6% vs. CROP baseline 9.5% (+66.1 pp, >15× improvement)
• RAG integration with AnalogCoder: +5.6% average functional correctness across 8 LLMs
• Claude Sonnet 4.6 + AnalogRetriever RAG: 86.7% functional correctness (new SOTA)
• Dataset DC pass rate: 11.4% → 99.7%

Why it matters / caveats: First unified cross-modal retrieval system for analog circuit design, enabling engineers to search by schematic sketch, text description, or netlist fragment interchangeably. Limited to 19 canonical topologies; excludes mixed-signal, RF, and power-management circuits; port-type vocabulary is hand-defined rather than learned; industrial-scale deployment requires efficient approximate nearest-neighbor search.

Problem: Code generation post-training is bottlenecked by execution-only feedback that requires self-contained executable code, cannot assess non-functional quality criteria (efficiency, security, readability), and lacks multilingual coverage — no robust multi-criteria code reward models exist.

Method: The paper releases two artifacts. Themis-CodeRewardBench is an 8,900-pair evaluation benchmark across 5 criteria (functional correctness, execution efficiency, memory efficiency, readability/maintainability, security hardness) and 8 programming languages (C, C++, C#, Go, Java, JavaScript, Python, Ruby). Themis-RM is a suite of reward models (0.6B–32B, based on Qwen3 dense architectures) trained with two-stage Bradley-Terry learning: pre-training on 110K+ general preferences, then preference modeling on 350K+ code-specific pairs mined from BigQuery GitHub commits (pre-March 2019) via ModernBERT classifiers + multi-LLM consensus filtering. Criteria-specified system prompts (65% single-criterion, 20% generic, 15% none) enable flexible scoring without per-criterion model specialization.

Key results:

• Themis-RM 32B: 91.82% average accuracy (94.27% FC, 84.95% EE, 95.16% ME, 87.59% R&M, 94.55% SH)
• Themis-RM 0.6B: 79.26% — outperforms Qwen2.5-Math-RM-72B (69.06%) and Nexusflow/Athene-RM-70B (81.19%) at 100× smaller
• AceCodeRM-32B (existing SOTA): only 49.55% on security hardness vs. Themis-RM 32B 94.55%
• Multi-criteria system prompts: 89.78% vs. criteria-level model merging 77.07% (−12.7 pp)
• Strong positive cross-lingual transfer despite training data imbalance

Why it matters / caveats: Opens post-training beyond functional correctness to non-functional code quality, with compact models (0.6B) that beat 70B+ general-purpose reward models at 100× less compute. Benchmark accuracy does not directly measure downstream alignment improvements; training data cutoff (pre-March 2019) may miss modern idioms and frameworks; synthetic inverse instruction generation for unlabeled commits introduces quality variability.

Generated on 2026-05-04 from HuggingFace Daily Papers API. PDFs sourced from arxiv.org.