ArXiv Domain 2026-10-16

数据来源：ArXiv Domain

LLM Domain Papers

1. Following the Flow the Law Requires: Training Voice Agents to Transfer to Unseen Regulated Procedures in Debt Collection

Abstract:Voice agents built on large language models (LLMs) now handle customer calls, but in regulated applications completing the task is not enough: the agent must follow a conversational flow set by law. In debt collection, it must confirm it is speaking to the debtor before mentioning the debt, identify itself and the creditor, state the amount, and avoid prohibited language. Every creditor turns these rules into its own procedure and revises it. We ask whether a small, self-hosted policy can be trained once and then follow regulated procedures it has never seen. One machine-readable specification per deployment, FlowSpec, generates the policy instruction, gates the tools, supplies the supervision and defines the tests; the agent speaks only approved catalogue sentences. We fine-tune a 9-billion-parameter model on 2,506 conversations from 297 randomized procedures and evaluate a Thai outbound-call agent on four unseen deployment procedures (three debt collection, one non-debt), one from a wholly withheld lineage. Documented checks, fixed by the procedure, rise from 67—71 percents to 93—100 percents, while grounded checks, fixed by the conversation, already pass at 97—100 percents before training; scenario pass rate rises from 15/43 to 40/43. An audit of the shared specification finds 11 defects, 3 of which its own metric did not expose.

中文摘要

摘要：基于大型语言模型（LLMs）的语音代理现在可以处理客户电话，但在受监管的应用中，仅仅完成任务是不够的：代理必须遵循法律规定的对话流程。在债务催收中，它必须在提及债务之前确认正在与债务人通话，识别自己和债权人，说明金额，并避免使用禁止的语言。每个债权人都会将这些规则转化为自己的流程并进行修订。我们探讨一个小型、自托管的策略是否可以只训练一次，然后遵循它从未见过的受监管流程。每次部署一个机器可读规范——FlowSpec，生成策略指令、控制工具、提供监管并定义测试；代理只使用经过批准的目录句子。我们对一个90亿参数的模型进行了微调，使用来自297个随机化流程的2,506次对话，并在四个未见过的部署流程（其中三个为债务催收，一个非债务流程）上评估泰语外呼代理，其中一个流程完全保密。依据文档的检查项（由流程固定）从67%-71%提升到93%-100%，而依据上下文的检查项（由对话固定）在训练前已通过97%-100%；场景通过率从15/43上升到40/43。对共享规范的审计发现了11个缺陷，其中3个未被其自身指标发现。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18881 (HTTP 429)

Authors: Petmongkon Pornpichitsuwan, Krittin Pachtrachai, Touchapon Kraisingkorn

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.18881.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18881

Published: 2026-10-16T01:39:08.169Z

2. Evaluating Large Language Models for Multilingual Abstractive Summarization: A Comprehensive Study of English and Bengali

Abstract:Abstractive summarization remains challenging across languages, where linguistic quality and factual consistency may diverge, particularly in low-resource settings. We present a standardized zero-shot benchmark of eight closed-source and open-weight LLMs on size-matched English and Bengali XL-Sum test sets. Our evaluation combines lexical and semantic metrics with reference-free factuality verification and multidimensional LLM-as-a-Judge assessment, providing a comprehensive analysis of both surface-level quality and factual reliability in multilingual summarization. Results reveal substantial cross-lingual variation: GPT-5.6 Luna leads several English lexical metrics but does not maintain the same advantage in Bengali, where LLaMA 3.3 70B achieves the highest ROUGE and BLEU scores. Gemini 2.5 Flash obtains the highest observed Bengali factuality score. Furthermore, evaluation methods produce different model rankings, highlighting the limitations of relying on a single metric for multilingual summarization assessment. These findings demonstrate that strong English summarization performance does not necessarily transfer to Bengali and emphasize the need for comprehensive evaluation frameworks for low-resource languages.

中文摘要

摘要：抽象摘要在多语言环境下仍然具有挑战性，其中语言质量和事实一致性可能存在差异，尤其是在低资源环境中。我们提出了一个标准化的零样本基准，对八个封闭源代码和开放权重的大型语言模型（LLM）在尺寸匹配的英文和孟加拉文 XL-Sum 测试集上进行了评估。我们的评估结合了词汇和语义指标、无参考事实性验证以及多维度的“LLM 作为评审”评估，提供了对多语言摘要的表层质量和事实可靠性的全面分析。结果显示显著的跨语言差异：GPT-5.6 Luna 在多项英文词汇指标中领先，但在孟加拉文中并未保持相同优势，在孟加拉文中，LLaMA 3.3 70B 达到最高的 ROUGE 和 BLEU 分数。Gemini 2.5 Flash 获得观察到的最高孟加拉文事实性评分。此外，评估方法会产生不同的模型排名，这凸显了依赖单一指标进行多语言摘要评估的局限性。这些发现表明，英文摘要性能强并不一定能传递到孟加拉文，并强调了为低资源语言建立综合评估框架的必要性。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19003 (HTTP 429)

Authors: Khan Mahi Al Atahar, Nazifa Tasneem, Ajwad Abrar

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19003.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19003

Published: 2026-10-16T01:39:08.169Z

3. Can LLMs Access their World Knowledge for Event Prediction?

Abstract:Recent work has questioned whether large language models (LLMs) can be useful for temporal event prediction. Despite vast pretrained world knowledge, LLMs often underperform simpler specialized models. We ask whether this knowledge is truly inaccessible, or whether current methods fail to present event prediction in a form LLMs can use. We argue for the latter and show that LLMs become effective event forecasters when the interface is designed around two choices. The first grounds each event in a natural-language description, and the second separates event semantics from the prediction space to prevent semantic bias from label associations. To test this, we introduce DescribeEvent, a benchmark of ~150K events across seven datasets and six domains with event-level descriptions. We find that capable LLMs substantially benefit from descriptions and symbolic labels, outperforming description-free variants and non-LLM baselines, while weaker models gain little. These results suggest that accessing LLM world knowledge for event prediction is an interface problem, and that event descriptions offer a practical way to elicit the knowledge LLMs already encode.

中文摘要

摘要：最近的研究质疑大型语言模型（LLMs）是否能在时间事件预测中发挥作用。尽管拥有大量预训练的世界知识，LLMs 的表现常常不如一些更简单的专门模型。我们提出疑问：这种知识是否真正无法访问，还是现有方法未能以 LLMs 可用的形式呈现事件预测。我们认为是后者，并展示了当接口围绕两个选择设计时，LLMs 会成为有效的事件预测者。第一个是将每个事件基于自然语言描述进行表述，第二个是将事件语义与预测空间分离，以防止标签关联造成的语义偏差。为了测试这一点，我们引入了 DescribeEvent，这是一个覆盖七个数据集和六个领域，包含约 15 万个事件及事件级描述的基准测试。我们发现，有能力的 LLMs 在拥有描述和符号标签的情况下受益显著，表现优于无描述的变体和非 LLM 基线，而较弱的模型获益不大。这些结果表明，访问 LLM 世界知识以进行事件预测是一个接口问题，而事件描述提供了一种实用的方法来引出 LLM 已经编码的知识。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19021 (HTTP 429)

Authors: Zijun Wu, Zhitian Zhang, Hossein Hajimirsadeghi, Nikita Dvornik, Alexander Pashevich

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19021.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19021

Published: 2026-10-16T01:39:08.169Z

4. Joint Subspace Sparse Attention for Efficient Long-Context Decoding

Abstract:Sparse attention reduces the KV-cache traffic that dominates long-context LLM decoding, but practical methods largely operate at coarse granularity. Token-level selection offers finer control, yet presents two challenges: low-dimensional scoring may lose query—key structure needed for accurate ranking, while gathering scattered selected tokens can offset the resulting latency benefit. We introduce Joint Subspace Sparse Attention (JSSA) to address both challenges. For selection quality, JSSA learns a shared low-dimensional query—key subspace by minimizing token-score preservation error. We upper-bound the resulting non-convex objective with a tractable marginal objective that admits a closed-form spectral solution; direct refinement of the exact objective provides no measurable improvement in the tested settings. For execution efficiency, we develop a gather-free token-level indexed-attention kernel that supports distinct per-KV-head selections under the standard grouped-query attention (GQA) KV-cache layout. Across standard GQA, hybrid sliding-window/global attention, and Multi-head Latent Attention (MLA) models, JSSA consistently remains close to full-KV accuracy while avoiding the substantial generation-length inflation and architecture-dependent failures observed with competing methods. On an NVIDIA B200 GPU, JSSA achieves up to $6.43\times$ the decoding throughput of full-KV attention at 128K context. JSSA achieves reasoning accuracy comparable to model-native DSA without model training and reaches 96—99\% of its decoding throughput even at 700B scale large models.

中文摘要

摘要：稀疏注意力减少了主导长上下文大语言模型（LLM）解码的KV缓存流量，但实际方法大多在粗粒度下操作。令牌级选择提供了更精细的控制，但带来两个挑战：低维评分可能丢失准确排序所需的查询-键结构，而收集分散的选中令牌可能抵消由此带来的延迟优势。我们提出联合子空间稀疏注意力（JSSA）以解决这两个挑战。在选择质量方面，JSSA通过最小化令牌评分保持误差来学习共享的低维查询-键子空间。我们对由此产生的非凸目标进行上界处理，使用可解的边际目标并获得闭式谱解；在测试设置下，对精确目标的直接优化未带来可测改进。在执行效率方面，我们开发了免收集的令牌级索引注意力内核，支持在标准分组查询注意力（GQA）KV缓存布局下进行每个KV头的独立选择。在标准GQA、混合滑动窗口/全局注意力以及多头潜在注意力（MLA）模型中，JSSA始终保持接近全KV精度，同时避免了竞争方法中观察到的生成长度显著膨胀和依赖架构的失败。在NVIDIA B200 GPU上，JSSA在128K上下文中解码吞吐量最高可达全KV注意力的$6.43 imes$。JSSA在推理准确性上可媲美模型原生的DSA，无需模型训练，即使在700B规模的大模型下也能达到其解码吞吐量的96—99%.

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19026 (HTTP 429)

Authors: Jiwon Song, Yoongon Kim, Jae-Joon Kim

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19026.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19026

Published: 2026-10-16T01:39:08.169Z

5. Crossing the Gender Binary: Modelling Gendered Reference in Science Fiction and General Fiction

Abstract:Science fiction is widely credited with unsettling gender. This article asks whether that holds at the linguistic level, in how readily gendered reference gives way to its counterpart, a neutral term, or other social category, and whether it separates the genre from general fiction. It does so by masking man/men, woman/women, and person/people across same-period science fiction and general fiction corpora and classifying RoBERTa’s five likeliest substitutes by gender. Retention and displacement are permutation-tested over five resampling rounds, and the underlying sentences are read closely. Both corpora cross the gender binary in both directions, but unequally: where woman is masked, the model predicts a masculine term about twice as often as it predicts a feminine term where man is masked. Each mismatch marks a statistical surprise, a word the corpus did not expect there. Against this shared pattern, science fiction shows greater gender permeability than the general fiction corpus, placing man in feminine contexts more often than general fiction while the reverse rate holds steady. Its degendering tendency is stronger too, drawing both gendered terms toward the neutral. Beyond the binary, a masked woman also draws substitutes naming a social category and no gender far more often than a masked man does. In measuring the gender permeability of man, woman, and person in linguistic distribution and testing whether genre changes it, the study gives science fiction studies linguistic evidence for thematic claims and computational literary studies a pipeline applicable to other category vocabularies, such as race and class.

中文摘要

摘要：科幻小说被广泛认为能动摇性别观念。本文探讨这种说法是否在语言层面上成立，即性别化的指称在多大程度上容易被其对应的中性词或其他社会类别取代，以及这种现象是否将该类型小说与一般小说区分开来。研究方法是对同期科幻小说和一般小说语料中的 man/men、woman/women 和 person/people 进行屏蔽，并将 RoBERTa 模型预测的五个最可能的替代词按性别分类。通过五轮重采样进行保留和替代的排列检验，并对基础句子进行仔细阅读。两个语料库在性别二元上都有跨越，但不均等：当 woman 被屏蔽时，模型预测男性词的频率大约是当 man 被屏蔽时预测女性词频率的两倍。每一个不匹配都标志着一个统计上的意外，是语料库未预期的词。基于这一共同模式，科幻小说比一般小说语料显示出更高的性别可渗透性，在女性语境中放置 man 的频率高于一般小说，而反向频率保持稳定。其去性别化倾向也更强，将两个性别词向中性词靠拢。在超越二元性别时，掩码 woman 还比掩码 man 更常引出命名社会类别和无性别的替代词。在测量 man、woman 和 person 在语言分布上的性别可渗透性并检验类型是否改变它时，这项研究为科幻小说的主题论断提供了语言学证据，同时也为计算文学研究提供了可应用于其他类别词汇（如种族和阶级）的研究流程。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19028 (HTTP 429)

Authors: Yuxuan Liu, Haim Dubossarsky, Ruth Ahnert

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19028.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19028

Published: 2026-10-16T01:39:08.169Z

Abstract:Record linkage and demographic inference from names are routine demands in social science. Most approaches to these problems require labeled training data that are difficult or costly to acquire. I show that pre-trained language models perform both tasks accurately without requiring task-specific training. For record linkage, I introduce EnsembleLink, an algorithm that uses pre-trained embeddings alongside string-based scorers to retrieve a short list of candidates and then identify the likeliest matches. EnsembleLink matches or exceeds the performance of most prior approaches while generalizing across scripts and formats that those methods are unable to handle. For ethnicity inference from names, I show that large language models (LLMs) achieve up to 83.8% accuracy on U.S. voter files, compared to 78.0% for name-supplemented BISG. Open-weight models offer reproducible implementations that approach these models in accuracy. Moreover, LLMs are able to infer group membership for religious, caste-based, and other ethnic group categories outside of the United States, and reduce bias in downstream estimates of geographic racial gaps. Where pre-training encodes the world knowledge required by each task, task-specific labels add less information.

中文摘要

摘要：姓名的记录链接和人口学推断是社会科学中的常规需求。这些问题的大多数方法都需要难以获得或成本高昂的标注训练数据。我展示了预训练语言模型可以在无需特定任务训练的情况下准确完成这两项任务。对于记录链接，我提出了EnsembleLink，这是一种结合预训练嵌入和基于字符串的评分器的算法，用于检索候选对象的短名单，然后识别最可能匹配的记录。EnsembleLink的表现与大多数以往方法相当或更优，同时能够在这些方法无法处理的不同文字和格式之间进行泛化。对于姓名的族群推断，我展示了大型语言模型（LLMs）在美国选民文件上的准确率可达83.8%，而使用姓名补充的BISG方法的准确率为78.0%。开放权重模型提供了可复现的实现，其准确率接近这些模型。此外，LLMs能够推断美国以外的宗教、种姓及其他族群类别的群体成员身份，并减少地理性种族差距下游估计中的偏差。当预训练编码了每个任务所需的世界知识时，特定任务的标注增加的信息较少。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19048 (HTTP 429)

Authors: Noah Dasanaike

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19048.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19048

Published: 2026-10-16T01:39:08.169Z

7. Similar Models, Broken Odds: Where Luce’s Axiom Fails in Chatbot Arena

Abstract:Chatbot Arena ranks language models with the Bradley-Terry model, which gives each model one score and so assumes Luce’s choice axiom. Choice theory predicts where this fails: near-copies are told apart more decisively than their scores imply. On 1.8 million Arena votes, votes between versions of one model are 1.37 times as decisive as Bradley-Terry predicts, between sizes of one generation 1.26 times and between models of one lab 1.12 times; a release a year later gives consistent multipliers, and the effect survives alternative tie treatments, prompt subsets and topics, and timing checks. How alike two models’ answers are worded and formatted tracks the effect. A correction with four to six parameters predicts held-out votes better than Bradley-Terry and than models with hundreds of parameters, mostly by modelling ties; frozen and carried to new releases in a test fixed in advance, its gains are concentrated in ties, and better head-to-head odds are not established. The leaderboard order barely moves. With language-model judges and three-answer menus, a near-copy takes votes mostly from its twin when choices are pooled over prompts (log-odds shift +0.10, 95% interval 0.05 to 0.16); within a prompt the shift is smaller and clear for two of four judges.

中文摘要

摘要：Chatbot Arena 使用布拉德利-特里（Bradley-Terry）模型对语言模型进行排名，该模型为每个模型赋予一个分数，因此假设了卢斯选择公理。选择理论预测了这种假设失败的情况：近似副本比其分数所表明的更容易被区分。在 180 万个 Arena 投票中，同一模型不同版本之间的投票比布拉德利-特里预测的更具决定性，比例为 1.37；同一代不同大小模型之间为 1.26；同一实验室不同模型之间为 1.12；一年后发布的版本给出了相似的倍数效果，而且这一效应在不同的平局处理、提示子集和主题以及时间检查中依然存在。两个模型的回答在措辞和格式上的相似程度与这一效应呈现相关。一种使用四到六个参数的修正方法比布拉德利-特里模型以及使用数百个参数的模型更好地预测了未见过的投票结果，这主要通过对平局进行建模实现；在预先固定的测试中冻结并用于新版本，其收益集中在平局结果中，更好的对决赔率并没有被确立。排行榜顺序几乎没有变化。在使用语言模型作为评审并提供三选项菜单的情况下，当选择在提示间合并时，一个近似副本的大部分票数来自其“孪生”模型（对数赔率变化 +0.10，95% 区间 0.05 至 0.16）；在单一提示内，变化较小，并且对于四名评审中的两名来说变化显著。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19095 (HTTP 429)

Authors: Soovadeep Bakshi, Apurba Bose, Aishwarya Bose, Sandip Nandi

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19095.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19095

Published: 2026-10-16T01:39:08.169Z

8. From Copying to Composition: How Language Models Develop In-Context Learning

Abstract:In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual information controls their predictions. We study the acquisition and trainability of ICL with a controlled diagnostic that separates copying a nearby demonstration from integrating information across the prompt. Across 17 models, about a third prefer the copied answer, a few are undecided, and the rest prefer the rule-consistent answer, which goes with higher accuracy and with control by the requested value rather than the closest conflicting example. Checkpoint analyses show that models begin prioritizing task-relevant prompt tokens over conflicting examples long before their accuracy improves beyond chance level. In matched training experiments, synthetic documents that define endings and then query them shift models toward compositional responses on the controlled diagnostic, whereas fine-tuning on definition exercises built from rewritten real documents improves accuracy when prompts reassign label meanings. The results suggest that ICL is not acquired as a single, uniform capability. Instead, prompt control, answer preference, construction without candidates, and transfer can develop along distinct trajectories.

中文摘要

摘要：情境学习（ICL）允许语言模型根据提示中的指令和示例调整其行为，但仅凭准确性无法揭示上下文信息如何控制它们的预测。我们通过一个受控诊断研究了ICL的获取和可训练性，该诊断将复制附近示例与跨提示整合信息分开。在17个模型中，大约三分之一的模型偏好复制的答案，少数模型犹豫不决，其余模型偏好规则一致的答案，这与更高的准确性以及对请求值而非最接近的冲突示例的控制相关。检查点分析显示，模型在准确率超过偶然水平之前很久，就开始优先考虑与任务相关的提示标记而非冲突示例。在匹配的训练实验中，定义结尾然后查询的合成文档会使模型在受控诊断中向组合式响应倾斜，而基于重写的真实文档构建的定义练习微调则在提示重新分配标签含义时提高准确性。结果表明，ICL并非作为单一、统一的能力获得。相反，提示控制、答案偏好、无候选构建和迁移可以沿着不同的轨迹发展。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19113 (HTTP 429)

Authors: Minglai Yang, Xinyu Guo, Renxiong Wang, Sunjie Hou, Daniel Yue Zhang, Yunzhong He

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19113.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19113

Published: 2026-10-16T01:39:08.169Z

9. DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision

Abstract:Emerging decision models such as JEV provide typed interfaces that map task instructions and observations directly to choices, judgments, and scores. As these models are increasingly used for automated decision-making, understanding their reliability under adversarial manipulation becomes critical. We introduce DecisionRobust, a benchmark for systematically evaluating the robustness of decision models across textual and visual inputs. Our benchmark covers practical decision-making scenarios and a diverse suite of attacks, including semantics-preserving text perturbations, image perturbations, and prompt injection in both modalities. We curate evaluation sets through external validity checks and cross-model candidate selection. We evaluate 30 model configurations on text tasks and 17 on vision tasks, including 24 and 12 held-out models, respectively, that are excluded from candidate selection. Our experiments reveal several key findings. First, attacks expose vulnerabilities beyond the models used for construction and selection, although optimized image perturbations transfer weakly to many targets even with multiple surrogates. Second, relative model robustness varies across tasks and attack types, and resistance to input perturbations does not necessarily imply resistance to prompt injection. Third, label-consistency checks substantially reduce measured attack success across text and image settings, highlighting the importance of validating apparent failures. The benchmark is available at this https URL.

中文摘要

摘要：新兴的决策模型如 JEV 提供了类型化接口，可以将任务指令和观察直接映射到选择、判断和分数。随着这些模型在自动化决策中的应用日益广泛，理解它们在对抗操作下的可靠性变得至关重要。我们提出了 DecisionRobust，这是一个用于系统评估决策模型在文本和视觉输入下稳健性的基准。我们的基准涵盖了实际决策场景和各种攻击方法，包括语义保持的文本扰动、图像扰动以及两种模态下的提示注入。我们通过外部有效性检查和跨模型候选选择来策划评估集。我们在文本任务上评估了 30 个模型配置，在视觉任务上评估了 17 个模型配置，其中分别包括 24 个和 12 个被排除在候选选择之外的保留模型。我们的实验揭示了若干关键发现。首先，攻击揭示的漏洞超出了参与构建和选择的模型范围，尽管优化的图像扰动即使用多个替代模型，其转移到许多目标模型上仍然较弱。其次，相对模型稳健性在不同任务和攻击类型中存在差异，对输入扰动的抵抗能力并不一定意味着对提示注入的抵抗能力。第三，标签一致性检查在文本和图像情境下显著降低了测得的攻击成功率，凸显了验证表面失败的重要性。该基准可通过此 https URL 获取。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19118 (HTTP 429)

Authors: Yifan Lan, Zhongye Liu, Yurui Chang, Hanyu Wang, Yongkang Du, Lu Lin, Jinghui Chen

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19118.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19118

Published: 2026-10-16T01:39:08.169Z

10. CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition

Abstract:Feature distillation is usually expected to add to logit distillation. We find that this expectation does not hold for two common feature losses in compact multilingual automatic speech recognition (ASR), and analyze why through their gradients. The gradient of a projection-based loss is orthogonal to the task gradient, while that of Centered Kernel Alignment (CKA), a similarity-based loss, is weakly aligned with it. CKA lowers the word error rate (WER) on non-English languages, but at the expense of data-rich English. Three loss-level remedies (one trained, two probed) all fall short, and the CKA gradient overlaps with the subspace English uses in the shared encoder by 97.5% to 99.5% (about 81% by chance). Guided by this diagnosis, we propose CAFE (Confined-Adapter Feature distillation), which confines the CKA gradient to adapters that English bypasses, addressing the trade-off in the parameters rather than in the loss. On the Open ASR Leaderboard, CAFE lowers the average WER from 8.55% to 8.02%, beats an adapter-only capacity control at 8.51%, and makes English more stable across checkpoints. At 254M parameters, it trails the 0.6B teacher by 2 WER points and is competitive with much larger models across 48 test sets in 24 languages. We use 564k hours of training data and state-of-the-art models to keep the findings relevant to practice.

中文摘要

摘要：特征蒸馏通常被期望对 logit 蒸馏有所补充。我们发现，对于紧凑型多语种自动语音识别（ASR）中的两种常见特征损失，这一期望并不成立，并通过它们的梯度分析了原因。基于投影的损失的梯度与任务梯度正交，而基于相似性的损失 Centered Kernel Alignment（CKA）的梯度则与其弱相关。CKA 降低了非英语语言的词错误率（WER），但以数据丰富的英语为代价。三种损失层面的补救方法（一种训练得到，两种通过探测得到）都未达到效果，并且 CKA 梯度与共享编码器中英语使用的子空间重叠率为 97.5% 至 99.5%（随机情况下约为 81%）。依据这一诊断，我们提出了 CAFE（Confined-Adapter 特征蒸馏），它将 CKA 梯度限制到英语绕开的适配器上，从而在参数而非损失上解决了权衡问题。在 Open ASR 排行榜上，CAFE 将平均 WER 从 8.55% 降至 8.02%，优于仅使用适配器的容量控制（8.51%），并使英语在各个检查点上更加稳定。在 2.54 亿参数情况下，它比 6 亿参数的教师模型落后 2 个 WER 点，并在 24 种语言的 48 个测试集上与更大模型具有竞争力。我们使用了 564k 小时的训练数据和最先进的模型，以确保研究结果对实际应用具有参考价值。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19134 (HTTP 429)

Authors: Tianchi Liu, Nicholas Moratelli, Khurram Azeem Hashmi, Mohammadreza Zolfaghari, Rishabh Jain, Louis Lu, Changdae Park, Amril Nazir

Categories: cs.CL

PDF URL: https://arxiv.org/pdf/2610.19134.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19134

Published: 2026-10-16T01:39:08.169Z

Agent Domain Papers

1. Why Static Skill Inconsistencies Do Not Equate to Runtime Risks: A Behavioral Analysis of LLM Agent Execution

Abstract:LLM agent frameworks heavily rely on external skill assets to standardize tool usage, encode operational workflows, and supply executable runtime resources. Despite their widespread adoption, multi-component skill packages frequently suffer from cross-artifact inconsistencies, where documentation, scripts, and configuration files specify conflicting behavioral constraints. Although static auditing techniques can uncover such contractual discrepancies, static defect volume cannot reliably reflect real-world agent performance degradation. This gap leaves practitioners without rigorous evidence to assess the practical risk of flawed agent skills. This paper presents a systematic empirical study to demystify the runtime implications of skill inconsistencies. We conduct large-scale auditing over 575 community and benchmark skills, yielding 1,182 manually validated contract inconsistencies. We further build a controlled evaluation pipeline with 49 executable agent tasks and three representative LLMs across NoSkill, OriginalSkill, and FixedSkill experimental settings. Through fine-grained behavioral analysis of 441 paired execution trajectories, we quantify the actual performance impacts of resolved defects.

中文摘要

摘要：大型语言模型（LLM）代理框架高度依赖外部技能资源来规范工具使用、编码操作工作流并提供可执行的运行时资源。尽管这些框架得到了广泛采用，多组件技能包仍常常存在跨文档不一致的问题，即文档、脚本和配置文件中规定了相互冲突的行为约束。尽管静态审计技术可以发现此类契约性差异，但静态缺陷数量无法可靠地反映真实世界中代理性能的下降。这一差距使得从业者缺乏严格的证据来评估有缺陷技能的实际风险。本文提出了一项系统的实证研究，以揭示技能不一致在运行时的影响。我们对 575 个社区和基准技能进行了大规模审计，产生了 1,182 个人工验证的契约不一致问题。我们进一步构建了一个受控评估管道，涵盖 49 个可执行代理任务和三种具有代表性的 LLM，在 NoSkill、OriginalSkill 和 FixedSkill 实验设置下进行测试。通过对 441 对执行轨迹的细粒度行为分析，我们量化了解决缺陷对实际性能的影响。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18927 (HTTP 429)

Authors: Fengrui Fan, Huahao Wang, Jingyi Fang, Yutao Hu, Yueming Wu, Deqing Zou

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18927.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18927

Published: 2026-10-16T01:39:28.871Z

2. The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

Abstract:Modern coding agents are fundamentally reshaping the role of human input for computational research: they write code, launch in silico experiments, analyze their results, and autonomously operate with ever-increasing time horizons. Despite these developments, fully autonomous research agents suffer from a limited ideation radius and myopic hill-climbing search dynamics. To bridge this gap, we introduce The Agentic Scientist, an end-to-end distributed pipeline for human researchers to conduct computational research alongside modern coding agents. Building on its predecessor, our system decomposes research into task drafting, idea generation, planning, sandboxed experimentation, result aggregation, baseline and ablation construction, post-hoc analysis, and report writing. Importantly, each step in the pipeline is powered by the next generation of coding agents and leverages agentic self-verification. The overall research process is driven by a meta-controller that manages parallel research agents and enforces rubric verification at each stage. This design enables the same orchestration layer to be reused across heterogeneous domains with different execution environments, agentic coding harnesses, and evaluation metrics. Our system can be used across a spectrum of autonomy: either as a co-scientist tool or to perform end-to-end scientific investigation. Within the co-scientist setting, we leveraged a development version of our system to submit a single research paper, which got accepted for publication at ALIFE 2026. We further apply The Agentic Scientist fully autonomously to open problem-solving tasks in machine learning, and demonstrate the system’s capability of scientific discovery across different domains of computational research. Our ablations suggest that reliable scientific autonomy depends not only on base model capabilities, but also on the cross-agent orchestration layer.

中文摘要

摘要：现代编码代理正在从根本上重塑人类在计算研究中的作用：它们编写代码、启动计算机实验、分析结果，并以越来越长的时间跨度自主运行。尽管取得了这些进展，完全自主的研究代理仍然存在创意半径有限和目光短浅的爬山式搜索动态问题。为填补这一空白，我们提出了“Agentic Scientist”（代理科学家），这是一个端到端的分布式流程，使人类研究人员能与现代编码代理一同开展计算研究。在其前身的基础上，我们的系统将研究分解为任务起草、创意生成、计划、沙箱实验、结果汇总、基线与消融构建、事后分析以及报告撰写。重要的是，流程中的每一步都由下一代编码代理驱动，并利用代理自我验证。总体研究过程由一个元控制器驱动，该控制器管理并行研究代理，并在每个阶段执行评分标准验证。这种设计使得同一个编排层可以在不同执行环境、代理编码能力和评估指标的异构领域中重复使用。我们的系统可以在多种自主性水平下使用：既可作为共同科学家工具，也可执行端到端科学研究。在共同科学家设置下，我们利用系统的开发版本提交了一篇研究论文，该论文被ALIFE 2026接收发表。我们还将“Agentic Scientist”完全自主地应用于机器学习中的开放问题解决任务，并展示了该系统在不同计算研究领域进行科学发现的能力。我们的消融实验表明，可靠的科学自主性不仅依赖于基础模型能力，还依赖于跨代理的编排层。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18933 (HTTP 429)

Authors: Robert Tjarko Lange, Luca Grillotti, Yutaro Yamada, Rujikorn Charakorn, David Ha

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18933.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18933

Published: 2026-10-16T01:39:28.871Z

3. Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench

Abstract:Most symbolic regression methods optimize candidate equations, but improving an equation’s fit does not establish whether its underlying scientific conjecture should be retained, revised, or rejected. A conjecture may admit multiple mathematical realizations, and similarly accurate equations may imply different explanations. This motivates a shift from refining individual expressions to developing and testing the conjectures that guide them. We introduce SCOPE, a framework that sustains multiple conjecture-driven research trajectories. Multiple scientist agents independently pursue conjectures through equation modeling, question formulation, and active experimentation, then exchange evidence to revise their conjectures and research directions. Their findings are consolidated into a scientific report linking models, conclusions, and evidence. Existing equation-discovery benchmarks primarily assess recovered expressions, leaving the evidence and scientific conclusions established through the investigation underexamined. We therefore introduce ClosureBench, an interactive benchmark of 60 closure-discovery tasks across five scientific domains. ClosureBench focuses on closure-discovery tasks to give equation discovery a concrete scientific purpose. Each task embeds an unknown algebraic relation in a known PDE system, where agents control experimental conditions in OpenFOAM to acquire new data for equation discovery. Beyond equation accuracy, ClosureBench proposes three metrics to evaluate the discovery process: evidence faithfulness, mechanism insight, and experimental effectiveness. Compared with diverse baselines, SCOPE achieves higher equation accuracy on both ClosureBench and a real-world scientific discovery case, while producing more accurate, reliable, and effective scientific discoveries.

中文摘要

摘要：大多数符号回归方法优化候选方程，但改善方程拟合度并不能确定其潜在科学假设是否应该被保留、修正或拒绝。一个假设可能有多种数学实现方式，同样精确的方程可能意味着不同的解释。这促使研究从改进单个表达式转向开发和检验引导它们的假设。我们引入了SCOPE，一个能够维持多条假设驱动研究轨迹的框架。多个科学家代理独立地通过方程建模、问题制定和主动实验来追求假设，然后交换证据以修正其假设和研究方向。他们的发现被整合成一份科学报告，将模型、结论和证据联系起来。现有方程发现基准主要评估恢复的表达式，而对通过调查确立的证据和科学结论缺乏检验。因此，我们引入了ClosureBench，这是一个涵盖五个科学领域的60个闭合发现任务的互动基准。ClosureBench聚焦于闭合发现任务，为方程发现提供具体的科学目标。每个任务在已知的PDE系统中嵌入未知代数关系，代理可以在OpenFOAM中控制实验条件以获取新的数据进行方程发现。除了方程精度外，ClosureBench提出了三个评价发现过程的指标：证据忠实度、机制洞察力和实验有效性。与多种基线方法相比，SCOPE在ClosureBench和一个真实科学发现案例中都达到了更高的方程精度，同时产生了更精确、可靠和有效的科学发现。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18938 (HTTP 429)

Authors: Yuxiao Hu, Qian Li, Ding Wang, Siheng Chen, Linfeng Zhang, Jinyue Yan, Yuntian Chen

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18938.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18938

Published: 2026-10-16T01:39:28.871Z

4. Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

Abstract:Low-Rank Adaptation (LoRA) has made diffusion model personalization affordable and widely shareable, but it also creates moderation risks when harmful or problematic content is not faithfully disclosed. Hence, it is desired to develop a mechanism that, given a LoRA adapter, discovers the concept it was trained to generate, so as to support downstream investigation. We characterize the target concept through the conditioning input that most clearly exposes what the LoRA has learned. Such an input should induce coherent LoRA outputs that are distinctive from those of the base model and strongly align with the learned LoRA subspace. We therefore formulate concept discovery as searching for the conditioning input that jointly maximizes output coherence, LoRA-induced distinctiveness, and subspace alignment. Even with a clear objective, however, optimizing it over an open-ended input space is prohibitively expensive, since reliable evaluation of coherence and distinctiveness requires repeated diffusion rendering. Crucially, we observe that an input need not be rendered to determine whether it is promising. Its alignment with the learned LoRA subspace provides a lightweight signal for localizing promising candidates before visual verification. Based on this insight, we develop a two-stage framework for text-to-image LoRAs. Subspace-Guided Evolutionary Search (SGES) uses render-free subspace alignment to explore candidate prompts and retain a compact shortlist, while Paired Visual Identification (PVI) renders only these candidates and compares the LoRA and base-model outputs to identify those exhibiting coherent and distinctive generation behavior. This design enables efficient and reliable discovery of previously unknown concepts encoded in LoRA adapters.

中文摘要

摘要：低秩适配（LoRA）使扩散模型的个性化变得经济且易于共享，但当有害或有问题的内容未被如实披露时，也会带来审查风险。因此，希望开发一种机制，给定一个LoRA适配器，能够发现其被训练生成的概念，从而支持下游调查。我们通过最清晰地揭示LoRA所学内容的条件输入来表征目标概念。这样的输入应能生成连贯的LoRA输出，这些输出与基础模型不同且与所学的LoRA子空间高度一致。因此，我们将概念发现制定为寻找能同时最大化输出连贯性、LoRA诱导的独特性以及子空间对齐的条件输入。即便目标明确，在开放式输入空间中进行优化仍然代价高昂，因为可靠评估连贯性和独特性需要重复进行扩散渲染。关键是，我们观察到不必渲染输入就能判断其是否有潜力。其与所学LoRA子空间的对齐提供了一种轻量级信号，用于在视觉验证前定位有潜力的候选项。基于这一洞察，我们开发了一个适用于文本到图像LoRA的两阶段框架。子空间引导的进化搜索（SGES）利用无需渲染的子空间对齐探索候选提示并保留紧凑的候选列表，而成对视觉识别（PVI）仅渲染这些候选并比较LoRA和基础模型的输出，以识别表现出连贯且独特生成行为的候选。该设计实现了对LoRA适配器中先前未知概念的高效且可靠的发现。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18990 (HTTP 429)

Authors: Yupeng Qiu, Han Fang, Dhananjai Sharma, Ee-Chien Chang

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18990.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18990

Published: 2026-10-16T01:39:28.871Z

Abstract:Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wrong certificate, for two reasons. First, the accuracy needed to act depends on what a wrong action costs, so the bar belongs to the deployment, not the module. Second, a module’s errors are rarely independent, and dependence has a sign: positively associated errors help where their damage overlaps and hurt where it compounds, so an accuracy threshold fails in a predictable direction, and once wrong beliefs interact and can do harm, no rule based on accuracy is right for every module. We test these predictions in task assignment, Level-Based Foraging and Hanabi, with injected errors, trained and Bayesian modules and five language models, and in human judgments. Accuracy fails in both directions, approving harmful language-model deployments in Hanabi, where shared errors compound, and rejecting beneficial ones in task assignment, where costly errors are rare. In our tasks it also buys far less redundancy than independence predicts: from GPT-5.4-mini to Claude Opus 5.5, error on teammates the models have evidence about falls from 32% to 11%, but the chance that all three observers misread one falls only from 16% to 9% (independence: 3.4% to 0.15%). We propose certifying the deployment, with a Blind-Spot Gate that prices error costs, pairwise interactions and measured dependence without deploying the module. On held-out data from the cells it was developed on, it makes no significant false approval in 20 splits (accuracy: 3.2 per split), and in preregistered tests on modules it was not developed on, it recovers what accuracy rejects (seven classifiers: regret 0.004 against 0.479) and catches what accuracy approves (a Bayesian module: significant false approvals in 17 of 20 splits for accuracy, 1 for the gate).

中文摘要

摘要：根据对队友的推断采取行动的协调者通常依赖这些推断的准确性来认证自身。我们认为准确性是错误的认证标准，原因有二。首先，采取行动所需的准确性取决于错误行动的成本，因此标准应针对部署而非模块。其次，模块的错误很少是独立的，而依赖性具有方向性：正相关的错误在损害重叠时有助，而在损害叠加时会造成不利，因此准确性阈值会朝可预测的方向失效，一旦错误信念相互作用并可能造成伤害，基于准确性的规则不可能适用于每个模块。我们在任务分配、基于级别的觅食和花火游戏中测试了这些预测，使用注入的错误、训练模块和贝叶斯模块以及五种语言模型，还包括人类判断。准确性在两个方向上都失败了：在花火游戏中，语言模型的共享错误会叠加从而批准了有害的部署；而在任务分配中，昂贵的错误很少，却拒绝了有益的部署。在我们的任务中，它购买的冗余也远低于独立性预测：从 GPT-5.4-mini 到 Claude Opus 5.5，对模型有证据的队友的错误率从 32% 降至 11%，但所有三名观察者误读同一对象的概率仅从 16% 降至 9%（独立性预测：从 3.4% 降至 0.15%）。我们提出对部署进行认证，使用“盲点门”（Blind-Spot Gate），在不部署模块的情况下评估错误成本、成对交互以及测得的依赖性。在其开发单元的保留数据上，它在 20 个拆分中没有显著的虚假批准（准确性：每拆分 3.2），在对未开发模块的预注册测试中，它恢复了准确性所拒绝的内容（七个分类器：后悔值 0.004 对比 0.479），并捕捉了准确性所批准的内容（一个贝叶斯模块：准确性在 20 个拆分中有 17 个显著虚假批准，而盲点门只出现 1 个）。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19051 (HTTP 429)

Authors: Will Beaumaster, Paul Schrater

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19051.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19051

Published: 2026-10-16T01:39:28.871Z

6. RLPE: Teaching LLMs to Prompt with Reinforcement Learning

Abstract:Prompting is a widely used method for adapting general-purpose LLMs to domain-specific tasks. Effective prompts can be tedious to create manually, so prompt writing is increasingly handed to LLMs, with recent work training LLMs to write prompts for other models. These techniques either write prompts without observing how the task model behaves, blind to the failure modes the prompt should address, or must run alongside the task LLM at inference time, adding overhead to every query. Neither reflects how prompt engineering is often practiced. To close this gap, we introduce Reinforcement Learned Prompt Engineering (RLPE), which trains a model to analyze traces of a separate task model and write a prompt targeting the failures it observes. RLPE rewards the trained model strictly on prompt performance, so it learns the prompting strategies that improve the task model. The prompt is generated once and reused across the task, so inference does not require the trained model. Since RLPE needs only query access to the model it prompts, it is also compatible with closed-source models. We apply RLPE to Enigmata, BIG-Bench Extra Hard, and PlanBench. We train a 9B-parameter model that writes better prompts than state-of-the-art frontier models (gpt-5.6-sol and claude-opus-5) and prompt optimizers (GEPA and TextGrad) on 2/3 benchmarks. Although the model is never explicitly trained to iterate on prompts, this capability emerges at a level that matches or beats frontier models. The learned prompting capability also generalizes, with the trained model improving task models 6x larger than the one it trained with, including ones from entirely different model families, and matching claude-opus-5 on a fourth benchmark held out from training.

中文摘要

摘要：提示（Prompting）是一种广泛使用的方法，用于将通用大型语言模型（LLM）适配到特定领域的任务。有效的提示手动创建可能非常繁琐，因此提示编写越来越多地交给LLM，最近的研究训练LLM为其他模型编写提示。这些技术要么在未观察任务模型行为的情况下编写提示，对提示应解决的失败模式视而不见，要么必须在推理时与任务LLM同时运行，从而增加每次查询的开销。这两者都未反映提示工程的常见实践。为弥补这一差距，我们提出了强化学习提示工程（Reinforcement Learned Prompt Engineering，RLPE），该方法训练一个模型分析独立任务模型的执行轨迹，并编写针对其观察到的失败的提示。RLPE严格根据提示的性能奖励训练模型，因此模型学会了可提升任务模型的提示策略。提示只生成一次并在整个任务中重复使用，因此推理时不需要训练模型。由于RLPE只需访问所提示模型的查询接口，因此也兼容封闭源代码模型。我们将RLPE应用于Enigmata、BIG-Bench Extra Hard和PlanBench。我们训练了一个9B参数的模型，在2/3个基准测试上生成的提示优于最先进的前沿模型（gpt-5.6-sol和claude-opus-5）以及提示优化器（GEPA和TextGrad）。尽管该模型从未被明确训练来迭代提示，这一能力仍然出现，其水平可与前沿模型相媲美或超越。所学的提示能力也具有泛化性，训练模型能提升比自身训练模型大6倍的任务模型，包括完全不同模型家族的模型，并在第四个训练中未涉及的基准测试上匹配claude-opus-5。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19062 (HTTP 429)

Authors: Rahul Singhal, Karime Maamari, Pradyumna Tambwekar

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19062.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19062

Published: 2026-10-16T01:39:28.871Z

7. Plan or Retrieve First? A Scale Study of Tool Discovery Architectures

Abstract:Tool-augmented LLM agents discover relevant APIs from large registries before planning execution, yet finite context windows prohibit injecting all tool definitions into a single prompt. The dominant approach, which we term Retrieve-then-Plan (RtP), embeds the user query and retrieves similar tools via approximate nearest-neighbor search. We study an alternative Plan-then-Retrieve (PtR) paradigm in which the LLM first generates Abstract Tool Stubs (hypothesized function signatures describing needed capabilities) which then serve as retrieval queries against the registry. Evaluating on ToolBench (46,984 unique API endpoints, 50 categories) across single-tool (G1), intra-category multi-tool (G2), and cross-category multi-tool (G3) query subsets, we find a clear complexity-dependent crossover: RtP outperforms PtR on single-tool queries while PtR outperforms RtP on multi-tool queries. This crossover holds at every retrieval budget (k=1 to 20), is robust across registry sizes from 1k to 47k APIs, and is consistent across model families (Gemini 2.5 Flash and gpt-oss-120b). We characterize a stub-hallucination failure mode (the LLM invents plausible but non-existent API names) that acts as a double-edged sword: on simple queries PtR’s stubs abstract away proper-noun signals that RtP matches directly, yet on complex queries the same mechanism decomposes multi-faceted intent into functional sub-needs. We further analyze token overhead and show that PtR’s extra discovery call roughly doubles inference cost, which pays off on multi-tool queries but hurts on single-tool ones, motivating complexity-adaptive routing as a practical direction, one that can be driven by a zero-cost signal already present in the pipeline (RtP’s top-1 retrieval score).

中文摘要

摘要：借助工具增强的 LLM 代理在计划执行之前会从大型注册表中发现相关的 API，但有限的上下文窗口无法将所有工具定义注入到单个提示中。主流方法，我们称之为“先检索再规划”（Retrieve-then-Plan, RtP），会对用户查询进行嵌入，并通过近似最近邻搜索检索相似工具。我们研究了一种替代性“先规划再检索”（Plan-then-Retrieve, PtR）范式，在该范式中，LLM 首先生成抽象工具存根（假设的函数签名，描述所需功能），然后将其作为检索查询对注册表进行查询。在 ToolBench（46,984 个独特 API 端点，50 个类别）上对单工具（G1）、同类多工具（G2）和跨类多工具（G3）查询子集进行评估，我们发现了一个明显的复杂性依赖的交叉点：RtP 在单工具查询上优于 PtR，而 PtR 在多工具查询上优于 RtP。这一交叉在每个检索预算（k=1 到 20）下都成立，在注册表规模从 1k 到 47k 个 API 时表现稳健，并且在不同模型系列（Gemini 2.5 Flash 和 gpt-oss-120b）中一致。我们描述了一种存根幻觉失败模式（LLM 发明了看似合理但实际上不存在的 API 名称），这种模式具有两面性：在简单查询中，PtR 的存根抽象掉了 RtP 可以直接匹配的专有名词信号，但在复杂查询中，同一机制将多方面意图分解为功能子需求。我们进一步分析了令牌开销，显示 PtR 的额外发现调用大约使推理成本翻倍，这在多工具查询中是值得的，但在单工具查询中会产生负面影响，因而推动了复杂性自适应路由作为一种实际方向，这可以由管道中已经存在的零成本信号驱动（即 RtP 的 top-1 检索分数）。

LLM Analysis

Q: 这篇论文试图解决什么问题？这篇论文试图解决\*\*工具增强型大语言模型（LLM）智能体在面对大规模API注册表时，如何高效且准确地发现相关工具\*\*的问题。具体而言，论文聚焦于以下核心矛盾与研究目标： ### 1. 核心问题：两种工具发现架构的系统性比较在上下文窗口受限的前提下，无法将所有API定义一次性注入提示。论文系统比较了两种截然不同的发现范式： - \*\*Retrieve-then-Plan (RtP)\*\*：主流方法，直接将用户查询嵌入向量空间，通过近似最近邻搜索检索相似API，再交由规划器生成执行计划。 - \*\*Plan-then-Retrieve (PtR)\*\*：替代方法，先由LLM根据用户查询生成\*\*抽象工具桩（Abstract Tool Stubs）\*\*——即假设的函数签名，描述完成任务所需的能力，再用这些桩作为检索查询去匹配真实API。 ### 2. 关键挑战：意图-注册表耦合（Intent-Registry Coupling）用户描述任务的方式与API文档的表述往往存在语义鸿沟。RtP假设这一鸿沟可通过单一嵌入向量直接弥合，但现有研究表明该假设在复杂查询下频繁失效。PtR则试图通过前置的规划步骤桥接这一鸿沟，但其本身引入了"桩幻觉"（stub hallucination）风险——即LLM可能生成看似合理但并不存在的API名称。 ### 3. 研究目标 - \*\*确定何时使用何种架构\*\*：论文发现存在\*\*复杂度依赖的交叉现象\*\*（complexity-dependent crossover）——RtP在单工具查询（G1）上更优，而PtR在多工具查询（G2/G3）上显著更优。 - \*\*刻画内在机制\*\*：分析"桩幻觉"作为双刃剑的作用机制——它在简单查询中可能抽象掉关键专有名词信号导致劣化，但在复杂查询中能将多面意图有效分解为功能子需求。 - \*\*实用化方向\*\*：基于token开销与检索质量的权衡，提出\*\*复杂度自适应路由\*\*（complexity-adaptive routing），即根据查询复杂度动态选择RtP或PtR，以在工业规模（5万级API）下降低约36%的发现调用开销而不损失检索质量。简言之，该工作并非提出一个需要训练的新系统，而是首次在受控的工业规模设置下，对"先检索"与"先规划"这两种基础范式进行了苹果对苹果的系统性刻画，阐明了其各自的优势边界与失效模式。 Q2: 有哪些相关研究？该论文的相关研究可归纳为以下几个方向： ### 1. 检索侧改进（Retrieval-Side Improvements）聚焦于优化工具描述或查询形式以提升向量检索质量，而非改变"先检索后规划"的基本流程： - \*\*Tool Retrieval Bridge\*\* (Chen et al., 2026)：通过桥接模型将模糊的用户指令对齐到检索器偏好的表示形式。 - \*\*Re-Invoke\*\* (Chen et al., 2024)：利用合成查询扩展工具描述，增强检索匹配度。 - \*\*TaskBench\*\* (Shen et al., 2024)：对"分解-再选择"（decompose-then-select）流程进行基准测试。 - \*\*Tool-MVR\*\* (Ma et al., 2025)：将工具桩幻觉（stub hallucination）视为需要修复的缺陷；本文则指出该现象在复杂查询中具有建设性作用。 - \*\*ToolFlood\*\* (Jawad and Brunel, 2026)：探讨如何通过语义覆盖隐藏有效工具以测试检索鲁棒性。 - \*\*工具学习综述\*\* (Qin et al., 2024a)：提供工具学习与检索的基础背景。 ### 2. LLM引导的工具选择（LLM-Guided Tool Selection）该路线依赖模型的参数化知识而非向量检索来进行工具筛选： - \*\*HuggingGPT\*\* (Shen et al., 2023)：利用ChatGPT从HuggingFace模型库中选择专用模型。 - \*\*AnyTool\*\* (Du et al., 2024)：使用GPT-4通过层次化结构逐级缩小API类别范围。 - \*\*LATM\*\* (Cai et al., 2024)：让大语言模型从零开始创建工具，而非从注册表中检索。 ### 3. 与Plan-then-Retrieve最接近的工作 - \*\*TOOLQP\*\* (Fang and Glass, 2026)：生成多轮查询改写（query reformulations）以提升API检索效果，采用强化学习训练。本文与之存在三点关键差异：(1) 工具桩通过单次零样本提示生成，无需训练；(2) 生成的是假设性函数签名而非自然语言查询改写；(3) 将PtR作为受控范式进行比较研究，而非构建一个需要训练的新系统。 ### 4. 复杂度自适应检索路由（Complexity-Adaptive Retrieval Routing）为支持"根据查询复杂度动态选择架构"这一实用方向，论文引用了以下 precedent： - \*\*Adaptive-RAG\*\* (Jeong et al., 2024)：根据问题复杂度在"无需检索"、"单步检索"与"迭代检索"之间进行路由决策，为本文提出的复杂度自适应路由（complexity-adaptive routing）提供了方法论先例。 ### 5. 基础背景与扩展工作 - \*\*Gorilla\*\* (Patil et al., 2024) 与 \*\*ToolLLM/ToolBench\*\* (Qin et al., 2024b)：奠定了大规模API检索与工具增强智能体的研究基础，本文的实验即基于ToolBench构建。 - \*\*ReAct\*\* (Yao et al., 2023) 与 \*\*Toolformer\*\* (Schick et al., 2023)：在讨论部分被引用，指出对于需要极多工具（如12个以上API）的查询，可能需要迭代式发现机制作为补充。 Q3: 论文如何解决这个问题？该论文通过\*\*受控架构比较\*\*与\*\*机制分析\*\*相结合的方式解决大规模API注册表下的工具发现问题，核心方法论与解决方案如下： --- ### 1. 形式化定义与受控比较两种发现范式论文将既有实践与替代方案抽象为两种范式，并在完全共享的基础设施下进行隔离比较： - \*\*Retrieve-then-Plan (RtP)\*\* 直接将用户查询 q 嵌入向量空间，基于余弦相似度从工具注册表 T 中检索Top- k API文档 d_t ：

R(RtP)(q) = TopK(t ∈ T) sim(q, dt)
随后规划器基于检索结果生成执行计划 π(RtP)(q, R(RtP)(q)) 。 - **Plan-then-Retrieve (PtR)** 先由LLM在无工具定义的情况下生成抽象工具桩（Abstract Tool Stubs） S(q) = s_1, …, s_m ，每个桩为包含假设工具名与推理的JSON对象。随后将每个桩嵌入并分别检索，取并集、去重、重排后得到Top- k 结果：
R(PtR)(q) = TopK(t ∈ T) max(s ∈ S(q)) sim(s, dt)
最终规划器基于 R(PtR)(q) 生成计划 π(PtR)(q, R(PtR)(q)) 。两种范式共享同一嵌入模型（GTE-base）、同一FAISS精确索引（`IndexFlatIP`，内积搜索等价于归一化向量的余弦相似度）、同一LLM（Gemini 2.5 Flash，温度=0）以及同一检索预算 k ，确保差异仅源于“先检索”与“先规划”的范式区别。 —- ### 2. 零样本生成抽象工具桩（PtR的核心机制） PtR不依赖训练或强化学习，而是通过**单次零样本提示调用**让LLM基于查询本身推测所需能力。生成的桩具有以下特性： - **假设性**：LLM在不访问真实注册表的情况下“幻觉”出工具名称（论文显示79–99%的桩名为查询间唯一，确认并非记忆真实API）。 - **功能性**：桩以函数签名形式描述所需能力，而非自然语言查询改写。该机制将用户的多面意图（multi-faceted intent）分解为多个功能子需求，每个子需求独立锚定向量空间中的不同区域。 —- ### 3. 识别复杂度依赖的交叉现象（Complexity-Dependent Crossover）通过在ToolBench（46,984个去重API端点）的三个子集上进行评估，论文发现架构优劣取决于查询复杂度： | 查询类型 | 代表子集 | 更优架构 | 关键指标（以 k=5 为例） | |————-|————-|————-|——————————-| | 单工具查询 | G1 ( n=200 ) | **RtP** | Recall: 0.372 vs. 0.289 ( p=0.002 ) | | 类别内多工具 | G2 ( n=200 ) | **PtR** | Recall: 0.188 vs. 0.143 ( p<0.001 ) | | 跨类别多工具 | G3 ( n=100 ) | **PtR** (方向性) | Recall: 0.172 vs. 0.160 ; Pass rate显著更优 | **该交叉现象具有稳健性**： - 在任意检索预算 k ∈ 1, 2, …, 20 下均成立； - 在注册表规模从1,000到47,000个API的范围内保持稳定； - 在不同模型家族（Gemini 2.5 Flash与gpt-oss-120b）上复现。 —- ### 4. 刻画Stub Hallucination的双重作用机制论文将“桩幻觉”重新界定为一种**双重机制**，而非单纯缺陷： - **在简单查询（G1）中劣化检索**：当查询包含API品牌名或专有名词时（高Intent-Registry Coupling），RtP可直接嵌入匹配；而PtR的桩将这些信号抽象为通用功能描述，导致错失精确匹配。 - **在复杂查询（G2/G3）中促进检索**：当查询与API文档的语义耦合较低时，单一嵌入难以捕获多面意图；桩幻觉将任务分解为多个独立的功能子查询，每个子查询分别检索，从而提升覆盖度。实验显示，正确与错误桩的Top-1余弦相似度几乎相同（如G1中 0.874 vs. 0.872 ），说明嵌入空间无法区分“好桩”与“坏桩”；但在多工具场景下，多桩覆盖的统计优势补偿了个体桩的失配。 —- ### 5. 提出复杂度自适应路由（Complexity-Adaptive Routing）基于上述发现，论文提出将查询按复杂度动态路由至不同架构，以解决PtR带来的**Token开销与延迟惩罚**（PtR的发现调用增加约75–104%的token，并引入一次额外的LLM串行往返）： - **路由策略**：单工具查询走RtP，多工具查询走PtR。 - **零成本路由信号**：无需额外模型或额外调用，可直接复用RtP流程中已产生的**Top-1余弦相似度分数**。分数高表明查询直接命名了API（高耦合，G1型），应继续使用RtP；分数低或弥散表明意图抽象（G2/G3型），应切换至PtR。 - **收益**：该路由策略可削减约**36%的PtR发现调用开销**（或总推理token的约16%），同时不损失检索质量。 —- ### 6. 验证检索作为行为瓶颈（Discovery as Binding Constraint）论文通过轨迹保真度（Trajectory Fidelity）分析验证：规划器在至少96%的情况下仅从检索到的候选集中选择工具。这意味着**检索阶段的架构选择几乎完全决定下游智能体行为**，从而确立了在检索层进行架构优化的必要性与充分性。 —- 综上，该论文的解决方案并非提出一个需要训练的新系统，而是通过**严格的受控实验**与**机制分析**，首次在工业规模下厘清了“先检索”与“先规划”的适用边界，并将这一理论认识转化为**低成本、可落地的复杂度自适应路由策略**。 Q4: 论文做了哪些实验？论文在工业规模（约5万API）下开展了一系列受控实验，系统比较 Retrieve-then-Plan (RtP) 与 Plan-then-Retrieve (PtR) 两种发现架构。实验可归纳为以下八个方面： —- ### 1. 实验设置与基准 - **注册表**：ToolBench 去重后的 46,984 个 API 端点，覆盖 50 个类别。 - **测试子集**： - **G1**：单工具查询（ n=200 ） - **G2**：类别内多工具查询（ n=200 ） - **G3**：跨类别多工具查询（ n=100 ） - **共享基础设施**：GTE-base 嵌入模型、FAISS `IndexFlatIP` 精确索引、Gemini 2.5 Flash（温度=0）作为规划器与 PtR 的发现模型。 - **评估指标**：检索召回率（Recall）、精确率（Precision）、通过率（Pass Rate）、轨迹保真度（Trajectory Fidelity）、工具选择分歧率（Tool Selection Divergence）及 Token 开销。 —- ### 2. 主实验：复杂度依赖的交叉现象在固定检索预算 k=5 下，对两种架构进行 head-to-head 比较： - **G1（单工具）**：RtP 全面领先。Recall 为 0.372 对 0.289 ± 0.010 （ p=0.002 ）；Precision 为 0.274 对 0.209 ± 0.008 （ p<0.001 ）；Pass Rate 为 0.884 ± 0.015 对 0.840 ± 0.024 （ p=0.022 ）。 - **G2（多工具-类别内）**：PtR 显著反超。Recall 为 0.188 ± 0.007 对 0.143 （ p<0.001 ）；Precision 为 0.204 ± 0.007 对 0.167 （ p=0.039 ）；Pass Rate 为 0.855 ± 0.020 对 0.747 ± 0.015 （ p<0.001 ）。 - **G3（多工具-跨类别）**：PtR 保持方向性优势，Recall 0.172 ± 0.008 对 0.160 （不显著），Pass Rate 显著更高（ 0.820 ± 0.014 对 0.732 ± 0.010 ， p<0.001 ）。逐查询胜负统计亦显示翻转：G1 上 RtP 赢 63 条 vs PtR 赢 33 条；G2 上 PtR 赢 69 条 vs RtP 赢 30 条。 —- ### 3. 检索预算消融实验（ k -ablation）验证交叉现象是否受检索预算 k 影响，实验遍历 k ∈ 1, 3, 5, 10, 20 ： | k | G1 RtP Recall | G1 PtR Recall | G2 RtP Recall | G2 PtR Recall | |:—-:|:—-:|:—-:|:—-:|:—-:| | 1 | 0.161 | 0.127 | 0.065 | 0.079 | | 3 | 0.307 | 0.223 | 0.110 | 0.148 | | 5 | 0.372 | 0.279 | 0.143 | 0.184 | | 10 | 0.453 | 0.356 | 0.186 | 0.234 | | 20 | 0.524 | 0.444 | 0.229 | 0.295 | 结果表明：**交叉在每个 k 下均成立**。PtR 的 G2 优势随 k 增大而扩大（从 +0.014 增至 +0.066 ），因为多桩各自贡献 Top- k 候选，提供更广的多面覆盖；RtP 的 G1 优势则保持稳定。 —- ### 4. 注册表规模缩放实验通过从 1k 到 47k 的按比例子注册表（5 个随机种子， distractor 采样）测试规模鲁棒性： - **Recall 曲线**：随注册表增大，双方 Recall 均下降，但 RtP 在 G1 上的领先与 PtR 在 G2 上的领先**在所有规模下保持稳定**（Recall 标准差 <0.012 ）。 - **工具选择分歧率**：从 1k 时的 88% 单调上升至 47k 时的 98% ，说明架构选择在更大规模下愈发关键。 —- ### 5. 智能体轨迹分歧分析将检索结果输入同一规划器后，从五个维度度量下游行为差异（结果基于 G1/G2/G3）： - **轨迹保真度（Trajectory Fidelity）**：规划器几乎总是从检索集合中选择工具， fidelity ≥ 96% ，证明**检索是行为瓶颈**。 - **选择率（Selection Rate）**：PtR 规划器使用了更高比例的已检索 API（ 27% – 29% vs 21% – 27% ）。 - **计划复杂度**：G3 上 PtR 产生更复杂的计划（平均 4.1 步、1.4 个 API），RtP 更稀疏（3.6 步、1.1 个 API）。 - **Jaccard 重叠度**：两架构选中的 API 集合相似度从 G1 的 0.40 降至 G3 的 0.28 。 - **检索失败后的恢复**：当检索完全脱靶时，双方均有约 70% 概率使用错误工具；但 RtP 在 G2 上更频繁地拒绝执行（ 47% vs 27% ）。 —- ### 6. 工具桩幻觉机制分析对 PtR 生成的抽象工具桩进行统计与定性分析： - **生成统计**：每查询平均生成 2.1 （G1）至 2.5 （G3）个桩；桩名唯一率高达 79% – 99% ，确认 LLM 在**从零幻觉**而非记忆真实 API。 - **匹配统计**：仅 40.6% （G1）和 24.8% （G3）的桩在 Top-1 匹配到相关 API。 - **嵌入空间不可区分性**：正确匹配与错误匹配的桩-API 余弦相似度几乎相同（G1: 0.874 vs 0.872 ），说明向量空间无法自行过滤幻觉桩。 - **案例研究**：展示了“RtP 赢”场景（查询含专有名词，PtR 桩过度泛化）与“PtR 赢”场景（查询描述功能需求，桩将多面意图分解为子查询）。 —- ### 7. 推理成本与开销分析量化 PtR 的额外代价： - **Token 开销**：PtR 的发现调用增加约 75% （G1）至 104% （G3）的 token，且引入一次额外的 LLM 串行往返（wall-clock 延迟惩罚）。 - **收益-成本权衡**：G1 上额外开销导致 Recall 下降 -0.072 ；G2 上则以 +80% token 换取 +0.042 Recall。 - **自适应路由收益**：若将 G1 查询路由至 RtP、G2/G3 路由至 PtR，可在不损失检索质量的前提下削减约 36% 的 PtR 发现调用开销（或总推理 token 的约 16% ）。 —- ### 8. 跨模型家族复现实验为验证结论是否模型特定，使用 **gpt-oss-120b** 复现主实验（保持 GTE-base、FAISS 索引与测试集不变）： - **G1**：RtP 仍全面优于 PtR（Recall 0.372 vs 0.259 ± 0.012 ， p<0.001 ）。 - **G2**：PtR 仍优于 RtR（Recall 0.194 ± 0.003 vs 0.143 ， p<0.001 ），Pass Rate 无显著差异。 - **G3**：与 Gemini 结果略有不同，gpt-oss-120b 上 G3 Recall 微弱反转至 RtP（ 0.160 vs 0.146 ），表明最小规模的 G3 跨模型稳定性最弱，但整体交叉方向保持一致。 —- 综上，实验设计覆盖了**检索预算、注册表规模、查询复杂度、下游规划行为、幻觉机制、推理成本及模型家族**七个维度，构成了对两种发现架构的系统性刻画。 Q5: 有什么可以进一步探索的点？基于论文的局限性与开放性问题，可进一步探索的方向包括： ### 1. 复杂度自适应路由的实证落地论文提出利用 **RtP 的 Top-1 余弦相似度** 作为零成本信号，将高耦合查询（G1 型）路由至 RtP、低耦合查询（G2/G3 型）路由至 PtR，并预期可削减约 36% 的发现调用开销。然而，该分类器尚未在真实流水线上进行端到端验证。未来工作可系统性地评估： - 该单一阈值的鲁棒性（如在不同的嵌入模型或注册表分布下是否依然有效）； - 更复杂的路由特征（如查询长度、从句数量、命名实体密度、并列子目标存在性）与 Top-1 分数的联合建模； - 路由决策本身引入的延迟与错误成本。 ### 2. Stub 校准机制（Stub Calibration） PtR 在 G1 上的失败源于桩对专有名词的过度抽象。未来可探索 **校准策略**，使 LLM 在生成桩时保留高耦合信号（如品牌名、产品名），同时在需要时进行功能抽象。例如： - 设计提示工程或轻量级后处理规则，识别查询中的专有名词并将其注入桩名； - 研究桩生成模型的微调方法，以在“抽象功能”与“保留专有名词”之间取得平衡，从而减少 G1 型查询的误路由。 ### 3. 超复杂查询的迭代式发现（Iterative Discovery）论文指出，对于需要 **12 个以上 API** 的复杂任务，单次静态检索（无论 RtP 还是 PtR）可能均不足够。未来可结合： - **ReAct**（Yao et al., 2023）或 **Toolformer**（Schick et al., 2023）式的迭代发现循环，让智能体在执行过程中动态发现新工具； - 将 PtR 作为迭代循环中的第一步，后续根据中间执行结果生成新的桩进行补充检索。 ### 4. 幻觉桩的识别与过滤机制实验发现，**正确与错误桩的嵌入相似度几乎不可区分**（如 G1 中 0.874 vs. 0.872 ），导致向量空间无法为桩质量提供监督信号。未来可探索： - 在注册表侧构建反向索引或验证层，对幻觉桩进行快速存在性校验； - 利用桩的语义描述与注册表元数据的对比学习，学习一个能区分“可匹配桩”与“不可匹配桩”的判别器； - 多桩间的共识机制（如多个桩指向同一真实工具时提高置信度）。 ### 5. 检索结果聚合与重排序的优化当前 PtR 采用简单的 **并集-去重-重排**（union, deduplicated, re-ranked）策略处理多桩检索结果。未来可研究更精细的融合策略： - 基于桩与文档的交互注意力机制进行重排序； - 考虑桩间功能依赖关系的图检索算法； - 动态分配各桩的检索预算（非均匀 k 分配），以替代固定 k 的均等检索。 ### 6. 跨注册表类型与领域的泛化论文实验仅基于 **ToolBench**（公开 API，英文文档，特定命名风格）。结论在以下场景的适用性尚待验证： - **企业内部注册表**：API 命名可能更加晦涩（如内部缩写、无自然语言描述的微服务）； - **多语言或跨语言注册表**：用户查询与 API 文档语言不一致时，意图-注册表耦合的形态可能改变； - **动态或私有注册表**：工具频繁增删时，桩生成的先验稳定性如何。 ### 7. 端到端执行评估与参数正确性当前研究以 **ToolBench 金轨迹** 作为检索代理指标，并假设规划器保真度 ≥ 96% 即足以反映最终性能。但： - **参数正确性**（如 API 参数名、类型、取值范围是否匹配）未加评估； - **最终答案质量**（工具调用后的实际返回是否满足用户需求）未在实验范围内； - 检索错误在端到端执行中的级联效应（cascading effect）尚不明确。 ### 8. 延迟优化与计算结构改进 PtR 的发现调用不仅在 Token 上增加 75–104% 开销，更关键的是引入了 **一次串行的 LLM 往返延迟**（处于关键路径上）。除复杂度自适应路由外，未来可探索： - **并行化架构**：如同时发起 RtP 与 PtR 检索，由轻量级裁决器选择结果（以计算换延迟）； - **桩生成与嵌入的流式处理**（streaming），减少首 token 等待时间； - 缓存高频查询的桩模板，避免重复生成。 ### 9. 模型规模与家族的系统化扩展虽然论文在 **Gemini 2.5 Flash** 与 **gpt-oss-120b** 上复现了交叉方向，但： - G3（最小测试集）在 gpt-oss-120b 上出现 Recall 微弱反转，表明跨模型稳定性在边界情况下存在波动； - 更小或更大的模型、不同训练范式（如推理时扩展 Thinking 模型）对桩质量与规划器行为的影响尚未系统研究。 ### 10. 混合架构（Hybrid Architecture）除“二选一”的路由外，可探索 **RtP 与 PtR 的深度融合**： - 以 RtP 的检索结果作为上下文，辅助 PtR 生成更准确的桩（减少幻觉）； - 以 PtR 的桩作为查询扩展（query expansion），增强 RtP 的原始查询表示，形成单一的联合检索向量。 Q6: 总结一下论文的主要内容该论文系统比较了工具增强型大语言模型（LLM）智能体在大规模API注册表（ sim 50k APIs）中的两种工具发现架构，揭示了查询复杂度对架构选择的决定性影响，并提出了实用的自适应路由策略。 —- ### 1. 研究背景与核心问题工具增强型LLM智能体需从包含数万API的注册表中检索相关工具，但有限的上下文窗口禁止一次性注入全部工具定义。现有主流范式 **Retrieve-then-Plan (RtP)** 直接嵌入用户查询并通过向量相似度检索Top- k API，假设用户意图与API文档之间的语义鸿沟可由单一嵌入向量弥合。论文研究一种替代范式 **Plan-then-Retrieve (PtR)**：LLM首先生成**抽象工具桩（Abstract Tool Stubs）**——即假设的函数签名，描述完成任务所需的能力，再用这些桩作为检索查询去匹配真实API。 —- ### 2. 核心方法论文在完全共享的基础设施下对两种范式进行了受控的“苹果对苹果”比较： - **共享组件**：GTE-base嵌入模型、FAISS精确内积索引（`IndexFlatIP`）、Gemini 2.5 Flash规划器（温度=0）、ToolBench注册表（46,984个去重API端点，50个类别）。 - **RtP流程**：将原始查询 q 嵌入，检索与文档 dt 余弦相似度最高的Top- k API： R(RtP)(q) = TopK(t ∈ T) sim(q, d_t) 。 - **PtR流程**：先由LLM在无工具定义的情况下生成桩集合 S(q) = s_1, …, s_m ，每个桩独立嵌入并检索，取并集、去重、重排后返回Top- k ： R(PtR)(q) = TopK(t ∈ T) max(s ∈ S(q)) sim(s, d_t) 。实验在ToolBench的三个子集上进行：**G1**（单工具， n=200 ）、**G2**（类别内多工具， n=200 ）、**G3**（跨类别多工具， n=100 ）。 —- ### 3. 主要发现：复杂度依赖的交叉现象论文发现两种架构的优劣存在**明确的复杂度依赖交叉**： - **G1（单工具查询）**：RtP显著优于PtR。Recall为 0.372 vs. 0.289 （ p=0.002 ），Precision为 0.274 vs. 0.209 （ p<0.001 ）。 - **G2（多工具-类别内）**：PtR显著优于RtP。Recall为 0.188 vs. 0.143 （ p<0.001 ），Pass Rate为 0.855 vs. 0.747 （ p<0.001 ）。 - **G3（跨类别多工具）**：PtR保持方向性优势，Pass Rate显著更高（ 0.820 vs. 0.732 ， p<0.001 ）。该交叉现象具有高度稳健性： - 在任意检索预算 k ∈ 1, 2, …, 20 下均成立； - 在注册表规模从1,000到47,000的范围内保持稳定； - 在gpt-oss-120b上复现了相同方向的结果，证明其是架构属性而非模型特有现象。 —- ### 4. 机制洞察：意图-注册表耦合与桩幻觉论文引入**意图-注册表耦合（Intent-Registry Coupling）**作为分析透镜，解释了交叉现象的根本原因： - **高耦合（G1）**：用户查询直接包含API名称或品牌名。RtP可通过单一嵌入精确匹配；而PtR的桩将专有名词抽象为通用功能描述，反而丢失了关键信号。 - **低耦合（G2/G3）**：用户意图多面且抽象，单一嵌入无法捕获。PtR生成的多个桩（平均每查询2.1–2.5个）将复杂任务分解为独立的功能子需求，各自锚定向量空间的不同区域，从而提升覆盖度。论文进一步指出，**桩幻觉（stub hallucination）**——LLM生成看似合理但实则不存在的API名称——是一种**双重机制**：在简单查询中破坏检索，在复杂查询中促进检索。实验显示，正确与错误桩的Top-1余弦相似度几乎相同（如G1中 0.874 vs. 0.872 ），意味着嵌入空间无法区分幻觉桩的质量，但多桩覆盖的统计优势在多工具场景下补偿了个体失配。 —- ### 5. 检索作为行为瓶颈通过**轨迹保真度（Trajectory Fidelity）**分析，论文证明规划器在至少96%的情况下仅从检索集合中选择API。这意味着**检索阶段的架构选择几乎完全决定了下游智能体的行为轨迹**，使得检索层成为整个流水线的 binding constraint。 —- ### 6. 实际意义：复杂度自适应路由 PtR的先规划步骤带来显著开销：每查询增加约75–104%的token消耗，并引入一次额外的LLM串行往返延迟。基于上述发现，论文提出**复杂度自适应路由**： - 将G1型（单工具）查询路由至RtP，G2/G3型（多工具）查询路由至PtR； - 使用**RtP的Top-1余弦相似度**作为零成本路由信号：高分表示高耦合（应走RtP），低分表示多面意图（应走PtR）； - 该策略可在不损失检索质量的前提下，削减约**36%的PtR发现调用开销**（或总推理token的约16%），并完全消除G1型查询的额外延迟。 —- ### 7. 局限性与展望论文明确指出现有局限：评估以ToolBench金轨迹为检索代理，未测试参数正确性与最终答案质量；实验限于英文公开API注册表，对企业内部API的泛化性尚待验证；PtR与RtP的串行结构存在延迟瓶颈，未来可探索更细粒度的路由特征与迭代式发现机制。 Q7: 想要进一步了解论文以上只是了解一篇论文的几个基本FAQ。如果你还想与Kimi进一步讨论该论文，请点击这里为你跳转Kimi AI网页版，并启动一个与该论文相关的新会话。

Authors: Mingfei Zhang, Amit Surana, Ziauddin Majid, Chenxi Wang, Michael Xu

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19068.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19068

Published: 2026-10-16T01:39:28.871Z

8. GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning

Abstract:Constructing robust GUI agents relies on multimodal visual grounding: the ability to precisely map natural language intents to actionable on-screen components. Despite abundant desktop, web, and mobile benchmarks, high-fidelity resources tailored for diverse environments are surprisingly scarce. To bridge this gap, we introduce the Screen Referring Grounding Dataset (ScreenRef), a comprehensive GUI grounding dataset consisting of GUI screenshots paired with natural language referring expressions and box or point annotations. Spanning desktop, web, and mobile domains, ScreenRef comprises 586K screenshots and over 633K referring expression instances, covering tasks such as text recognition, visual matching, spatial understanding, and refusal scenarios. To boost the model’s grounding performance, we propose the Distillation-to-Reinforcement Learning (D2RL) method to address the limited generalization of supervised fine-tuning and the sparse reward problem in reinforcement learning. Utilizing ScreenRef, we train Qwen3-VL-8B-Instruct via sequential SFT and D2RL training, yielding GroundGUI-8B. GroundGUI-8B achieves 95.2% on ScreenSpot-V2, 63.9% on ScreenSpot-Pro, 65.4% on OSWorld-G, 86.8% on MMBench-GUI L2, and 38.3% on UI-Vision. Among 8B-scale models, it produces substantial gains over models trained on open-source datasets, with performance comparable or even superior to models trained on closed-source data. Code, models, and datasets are available at this https URL.

中文摘要

摘要：构建稳健的 GUI 代理依赖于多模态视觉定位：即将自然语言意图精确映射到可操作的屏幕组件的能力。尽管桌面、网页和移动端基准测试资源丰富，但针对多样化环境的高精度资源却异常稀缺。为填补这一空白，我们推出了屏幕引用定位数据集（ScreenRef），这是一个综合性的 GUI 定位数据集，由 GUI 截图、自然语言引用表达以及盒状或点状标注组成。ScreenRef 涵盖桌面、网页和移动端领域，包括 586K 张截图和超过 633K 条引用表达实例，涵盖文本识别、视觉匹配、空间理解以及拒绝场景等任务。为了提升模型的定位性能，我们提出了蒸馏到强化学习（D2RL）方法，以解决监督微调泛化性有限和强化学习中奖励稀疏的问题。利用 ScreenRef，我们通过顺序 SFT 与 D2RL 训练，训练了 Qwen3-VL-8B-Instruct，生成了 GroundGUI-8B。GroundGUI-8B 在 ScreenSpot-V2 上达到了 95.2%，在 ScreenSpot-Pro 上为 63.9%，OSWorld-G 为 65.4%，MMBench-GUI L2 为 86.8%，UI-Vision 为 38.3%。在 8B 规模模型中，相较于使用开源数据集训练的模型，它取得了显著提升，其性能与使用闭源数据训练的模型相当甚至更优。代码、模型和数据集可通过此 https 链接获取。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19100 (HTTP 429)

Authors: Qingsong Xie, Jiahao Tian, Jiacheng Tang, Zhifeng Li, Madalina Ciobanu, Navan Preet Singh, Qingqing Mao, Ritankar Das

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19100.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19100

Published: 2026-10-16T01:39:28.871Z

9. Plant Memory: Technical Language Processing and a Verified Maintenance Ontology for Agent-Accessible Failure Knowledge from SAP PM Work Orders

Abstract:Maintenance work orders are the largest written record of how industrial equipment fails, yet in most plants they are close to unusable for analysis. Short texts are abbreviated and misspelt, asset identity is buried in free text, work-type labels contradict the work described, and the fields that matter most for reliability (symptom, root cause, action taken) are usually empty. As a result, diagnostic knowledge stays with senior technicians and leaves when they do. We present Plant Memory, a system that applies technical language processing (TLP) to SAP Plant Maintenance (PM) exports and builds an ISO 14224-aligned maintenance ontology in which every extracted statement is a reified fact carrying a confidence score, the evidence it was read from, and a verification state. Extraction is rule-first, with an optional large language model (LLM) acting as a second reader whose disagreements raise a fact’s review priority rather than overwrite it. Technicians verify facts through a review queue, answer targeted knowledge-gap questions where failures repeat without a recorded cause, and approve or reject ontology drift proposals when new terms appear. The resulting graph is served to people through a web application and to AI agents through a Model Context Protocol (MCP) server, so both query the same verified knowledge and every answer cites the work orders behind it. On a synthetic 12-month export of 400 work orders from a parboiled-rice plant, rule-based extraction reaches 96.2% component accuracy, 88.9% failure-mode accuracy and 100% root-cause precision at 62% coverage; on 28 diagnostic questions, ranked root causes from the ontology are correct at top-1 in 82.1% of cases against 53.6% for the raw SAP root-cause column.

中文摘要

摘要：维护工单是工业设备故障情况的最大书面记录，但在大多数工厂中，它们几乎无法用于分析。文本简短且存在缩写和拼写错误，资产身份埋藏在自由文本中，工作类型标签与所描述的工作相矛盾，而对于可靠性最重要的字段（症状、根本原因、采取的措施）通常为空。因此，诊断知识停留在资深技术员手中，随着他们的离开而流失。我们提出了Plant Memory系统，它将技术语言处理（TLP）应用于SAP工厂维护（PM）导出数据，并构建了一个符合ISO 14224标准的维护本体，其中每个提取的陈述都是一个具化事实，携带置信度分数、来源证据和验证状态。提取以规则为主，可选的大型语言模型（LLM）作为第二阅读者，其分歧会提高事实的审核优先级，而不是覆盖已有事实。技术员通过审查队列验证事实，针对重复发生但未记录原因的故障回答有针对性的知识空缺问题，并在出现新术语时批准或拒绝本体漂移建议。生成的知识图谱通过网页应用提供给人类用户，通过模型上下文协议（MCP）服务器提供给AI代理，从而使两者查询相同的已验证知识，每个答案都引用背后的工单。在一份来自一个预煮米厂的400份工单为期12个月的合成导出数据上，基于规则的提取实现了96.2%的组件准确率，88.9%的故障模式准确率，以及62%覆盖范围下100%的根本原因精度；在28个诊断问题上，本体中的排名根本原因在首位的正确率为82.1%，而SAP原始根本原因列的正确率为53.6%。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19131 (HTTP 429)

Authors: Akash Raj

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19131.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19131

Published: 2026-10-16T01:39:28.871Z

10. Graph-Based Orchestration of Heterogeneous AI Models: A Control Node Approach to Composite Intelligence

Abstract:We present a graph-based framework for Composite AI systems that integrate multiple AI techniques to address complex problems. We describe an architecture using specialized control nodes to orchestrate diverse models including planning, machine learning, constraint programming, and large language models. The system employs directed acyclic graphs and behavior trees to manage model relationships and execution sequencing. Our methodology encompasses data connectors, computational services, and a containerized deployment framework supporting hierarchical, sequential, and concurrent model composition patterns. Through case studies, we demonstrate how this architecture enables the creation of composite systems that leverage the strengths of multiple AI techniques within a unified operational schema, overcoming the limitations of single-model approaches.

中文摘要

摘要：我们提出了一个基于图的复合人工智能系统框架，该系统整合了多种人工智能技术以解决复杂问题。我们描述了一种使用专用控制节点来协调多样模型的架构，这些模型包括规划、机器学习、约束编程和大型语言模型。该系统采用有向无环图和行为树来管理模型关系和执行顺序。我们的方法包括数据连接器、计算服务以及支持分层、顺序和并发模型组合模式的容器化部署框架。通过案例研究，我们展示了该架构如何实现复合系统的创建，使其在统一的操作模式下利用多种人工智能技术的优势，从而克服单模型方法的局限性。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19136 (HTTP 429)

Authors: G. Michael Youngblood, Filip Dvořák, Slavo Švancár, Tomáš Balyo, Michal Ficek, Martin Dousek

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19136.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19136

Published: 2026-10-16T01:39:28.871Z

Evaluation Domain Papers

1. Why Static Skill Inconsistencies Do Not Equate to Runtime Risks: A Behavioral Analysis of LLM Agent Execution

中文摘要

摘要：大型语言模型（LLM）代理框架高度依赖外部技能资源来规范工具使用、编码操作工作流并提供可执行的运行时资源。尽管这些框架被广泛采用，多组件技能包经常存在跨工件的不一致问题，即文档、脚本和配置文件指定了相互冲突的行为约束。尽管静态审计技术可以发现此类合同差异，但静态缺陷数量无法可靠反映现实世界中代理性能的下降。这一差距使得从业者无法获得严格证据来评估技能缺陷代理的实际风险。本文提出了一项系统的实证研究，以揭示技能不一致的运行时影响。我们对575个社区和基准技能进行了大规模审计，产生了1,182个经过人工验证的合同不一致问题。我们进一步构建了一个受控评估管道，在无技能（NoSkill）、原始技能（OriginalSkill）和修复技能（FixedSkill）三种实验设置下，使用49个可执行代理任务和三种代表性LLM进行评估。通过对441对执行轨迹的细粒度行为分析，我们量化了解决缺陷对实际性能的影响。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18927 (HTTP 429)

Authors: Fengrui Fan, Huahao Wang, Jingyi Fang, Yutao Hu, Yueming Wu, Deqing Zou

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18927.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18927

Published: 2026-10-16T01:39:49.113Z

2. The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

中文摘要

摘要：现代编码代理正在从根本上重塑人类在计算研究中的作用：它们编写代码、启动计算机实验、分析结果，并以越来越长的时间跨度自主运行。尽管取得了这些进展，完全自主的研究代理仍然存在创意半径有限和目光短浅的爬山式搜索动态问题。为填补这一空白，我们提出了“自主科学家”（The Agentic Scientist），这是一个端到端的分布式管道，旨在让人类研究者与现代编码代理共同开展计算研究。在其前身的基础上，我们的系统将研究分解为任务草拟、创意生成、规划、沙箱实验、结果汇总、基线与消融构建、事后分析以及报告撰写。重要的是，管道中的每一步都由新一代编码代理提供支持，并利用代理自我验证。整体研究过程由元控制器驱动，该控制器管理并行研究代理，并在每个阶段执行规范验证。这一设计使得相同的编排层能够在具有不同执行环境、代理编码工具和评估指标的异构领域中重复使用。我们的系统可以在不同的自主性水平下使用：既可作为协同科学家工具，也可执行端到端的科学研究。在协同科学家模式中，我们利用系统的开发版本提交了一篇研究论文，该论文被ALIFE 2026接收发表。我们进一步将“自主科学家”完全自主地应用于机器学习中的开放问题解决任务，并展示了系统在计算研究不同领域进行科学发现的能力。我们的消融实验表明，可靠的科学自主性不仅依赖于基础模型的能力，还依赖于跨代理的编排层。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18933 (HTTP 429)

Authors: Robert Tjarko Lange, Luca Grillotti, Yutaro Yamada, Rujikorn Charakorn, David Ha

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18933.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18933

Published: 2026-10-16T01:39:49.113Z

3. Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench

中文摘要

摘要：大多数符号回归方法优化候选方程，但改善方程拟合度并不能确定其潜在科学假设是否应该被保留、修正或拒绝。一个假设可能有多种数学实现方式，同样精确的方程可能意味着不同的解释。这促使研究从改进单个表达式转向开发和检验引导它们的假设。我们引入了SCOPE，一个支持多条假设驱动研究轨迹的框架。多个科学家代理通过方程建模、问题提出和主动实验独立追求假设，然后交换证据以修正他们的假设和研究方向。他们的发现被整合成一份科学报告，将模型、结论和证据联系起来。现有的方程发现基准主要评估恢复的表达式，而对通过研究建立的证据和科学结论关注不足。因此，我们引入了ClosureBench，一个跨五个科学领域的60个闭环发现任务的交互式基准。ClosureBench关注闭环发现任务，以赋予方程发现明确的科学目标。每个任务在已知的PDE系统中嵌入未知代数关系，代理在OpenFOAM中控制实验条件以获取新的数据用于方程发现。除了方程精度之外，ClosureBench提出了三个评估发现过程的指标：证据可信度、机制洞察力和实验有效性。与多种基线方法相比，SCOPE在ClosureBench和一个真实科学发现案例中都实现了更高的方程精度，同时产生更准确、可靠和有效的科学发现。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18938 (HTTP 429)

Authors: Yuxiao Hu, Qian Li, Ding Wang, Siheng Chen, Linfeng Zhang, Jinyue Yan, Yuntian Chen

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18938.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18938

Published: 2026-10-16T01:39:49.113Z

4. Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

中文摘要

摘要：低秩适配（LoRA）使扩散模型个性化变得经济且易于共享，但当有害或问题内容未被如实披露时，也带来了审核风险。因此，有必要开发一种机制，能够在给定 LoRA 适配器的情况下，发现其训练生成的概念，从而支持后续调查。我们通过能够最清晰地揭示 LoRA 学习内容的条件输入来表征目标概念。这样的输入应能生成连贯的 LoRA 输出，这些输出与基础模型的输出区别明显，并且与学到的 LoRA 子空间高度一致。因此，我们将概念发现表述为寻找条件输入，以联合最大化输出连贯性、LoRA 诱导的区别性和子空间对齐。尽管目标明确，但在开放式输入空间上优化代价高昂，因为对连贯性和区别性进行可靠评估需要重复的扩散渲染。关键的是，我们观察到不必渲染输入即可判断其是否有潜力。其与学得 LoRA 子空间的对齐提供了一种轻量信号，用于在视觉验证前定位有潜力的候选项。基于这一洞察，我们开发了一个适用于文本到图像 LoRA 的两阶段框架。子空间引导的进化搜索（SGES）利用无需渲染的子空间对齐探索候选提示并保留紧凑的候选列表，而配对视觉识别（PVI）仅渲染这些候选并比较 LoRA 与基础模型的输出，以识别表现出连贯且有区别的生成行为的项目。该设计实现了对 LoRA 适配器中先前未知概念的高效且可靠的发现。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.18990 (HTTP 429)

Authors: Yupeng Qiu, Han Fang, Dhananjai Sharma, Ee-Chien Chang

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.18990.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.18990

Published: 2026-10-16T01:39:49.113Z

中文摘要

摘要：根据对队友的推断采取行动的协调者通常依赖这些推断的准确性来认证自身。我们认为准确性是错误的认证标准，原因有二。首先，采取行动所需的准确性取决于错误行动的成本，因此标准应针对部署而非模块。其次，模块的错误很少是独立的，而依赖性具有方向性：正相关的错误在损害重叠时有助，而在损害叠加时会造成不利，因此准确性阈值会朝可预测的方向失效，一旦错误信念相互作用并可能造成伤害，基于准确性的规则不可能适用于每个模块。我们在任务分配、基于级别的觅食和花火游戏中测试了这些预测，使用注入的错误、训练模块和贝叶斯模块以及五种语言模型，还包括人类判断。准确性在两个方向上都失败了：在花火游戏中，语言模型的共享错误会叠加从而批准了有害的部署；而在任务分配中，昂贵错误很少，却拒绝了有益的部署。在我们的任务中，它带来的冗余也远低于独立性预测：从GPT-5.4-mini到Claude Opus 5.5，模型对有证据的队友的错误率从32%降至11%，但三位观察者全部误读的概率仅从16%降至9%（独立性预测应为3.4%到0.15%）。我们提出认证部署的方法，使用盲点门（Blind-Spot Gate）在不部署模块的情况下评估错误成本、成对交互和测量的依赖性。在从其开发的单元中抽出的留出数据上，它在20个拆分中没有显著的错误批准（准确性：每个拆分3.2），在对未开发模块的事先注册测试中，它恢复了准确性拒绝的内容（七个分类器：后悔值0.004对比0.479），并捕捉了准确性批准的内容（一个贝叶斯模块：准确性在20个拆分中有17次显著的错误批准，而盲点门仅有1次）。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19051 (HTTP 429)

Authors: Will Beaumaster, Paul Schrater

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19051.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19051

Published: 2026-10-16T01:39:49.113Z

6. RLPE: Teaching LLMs to Prompt with Reinforcement Learning

中文摘要

摘要：提示（Prompting）是一种广泛使用的方法，用于将通用大型语言模型（LLM）适配到特定领域的任务。有效的提示手动创建可能非常繁琐，因此提示编写越来越多地交给LLM完成，最近的研究则训练LLM为其他模型编写提示。这些技术要么在不观察任务模型行为的情况下编写提示，对提示应解决的失败模式一无所知，要么必须在推理时与任务LLM同时运行，从而为每次查询增加开销。这两种方式都不反映提示工程的常见实践。为缩小这一差距，我们提出了强化学习提示工程（RLPE），该方法训练模型分析独立任务模型的追踪数据，并编写针对它观察到的失败的提示。RLPE严格根据提示性能奖励训练模型，因此它学习能够改进任务模型的提示策略。提示生成一次即可在整个任务中重复使用，因此推理时不需要训练模型。由于RLPE仅需要对其提示的模型进行查询访问，因此也兼容闭源模型。我们将RLPE应用于Enigmata、BIG-Bench Extra Hard和PlanBench。我们训练了一个拥有90亿参数的模型，其在2/3的基准测试上比现有最先进的前沿模型（gpt-5.6-sol和claude-opus-5）及提示优化器（GEPA和TextGrad）生成更好的提示。尽管该模型从未被明确训练用于迭代优化提示，但这种能力仍然以匹配或超越前沿模型的水平出现。所学习的提示能力也具有泛化性，训练好的模型可以改善比其训练模型大6倍的任务模型，包括来自完全不同模型体系的模型，并且在第四个训练时未使用的基准上达到与claude-opus-5相当的效果。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19062 (HTTP 429)

Authors: Rahul Singhal, Karime Maamari, Pradyumna Tambwekar

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19062.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19062

Published: 2026-10-16T01:39:49.113Z

7. Plan or Retrieve First? A Scale Study of Tool Discovery Architectures

中文摘要

摘要：借助工具的LLM代理能够在规划执行之前从大型注册表中发现相关的API，但有限的上下文窗口禁止将所有工具定义注入到单一的提示中。主流的方法，我们称之为“先检索再规划”（Retrieve-then-Plan, RtP），会将用户查询嵌入并通过近似最近邻搜索检索相似工具。我们研究了一种替代的“先规划再检索”（Plan-then-Retrieve, PtR）范式，其中LLM首先生成抽象工具存根（即描述所需能力的假设函数签名），然后这些存根作为检索查询在注册表中使用。在ToolBench（46,984个独特API端点，50个类别）上评估单工具（G1）、类内多工具（G2）和跨类多工具（G3）查询子集时，我们发现一个明显的复杂度依赖交叉：RtP在单工具查询上表现优于PtR，而PtR在多工具查询上优于RtP。这种交叉在每个检索预算（k=1到20）下都成立，且在注册表大小从1k到47k API之间表现出强鲁棒性，并在模型家族（Gemini 2.5 Flash和gpt-oss-120b）中一致。我们描述了一种存根幻觉失败模式（LLM发明了看似合理但不存在的API名称），它是一把双刃剑：在简单查询上，PtR的存根抽象掉了RtP直接匹配的专有名词信号，但在复杂查询上，同一机制将多面向意图分解为功能子需求。我们进一步分析了Token开销，并显示PtR额外的发现调用大约使推理成本翻倍，这在多工具查询上是有益的，但在单工具查询上则不利，从而激发了复杂度自适应路由作为一种实际方向的动机，而这种路由可以通过管道中已有的零成本信号驱动（即RtP的top-1检索得分）。

LLM Analysis

Authors: Mingfei Zhang, Amit Surana, Ziauddin Majid, Chenxi Wang, Michael Xu

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19068.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19068

Published: 2026-10-16T01:39:49.113Z

8. GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning

中文摘要

摘要：构建稳健的 GUI 代理依赖于多模态视觉定位：即将自然语言意图精确映射到可操作的屏幕组件的能力。尽管桌面、网页和移动端基准测试资源丰富，但针对多样化环境的高精度资源却异常稀缺。为弥补这一空白，我们引入了屏幕指向定位数据集（ScreenRef），这是一个全面的 GUI 定位数据集，由 GUI 截图与自然语言指向表达及框或点标注配对组成。ScreenRef 覆盖桌面、网页和移动端领域，包含 586K 截图和超过 633K 个指向表达实例，涵盖文本识别、视觉匹配、空间理解及拒绝场景等任务。为了提升模型的定位性能，我们提出了蒸馏-强化学习（Distillation-to-Reinforcement Learning, D2RL）方法，以解决监督微调泛化能力有限及强化学习中稀疏奖励的问题。利用 ScreenRef，我们通过顺序 SFT 和 D2RL 训练 Qwen3-VL-8B-Instruct，得到 GroundGUI-8B。GroundGUI-8B 在 ScreenSpot-V2 上取得 95.2%，ScreenSpot-Pro 上 63.9%，OSWorld-G 上 65.4%，MMBench-GUI L2 上 86.8%，UI-Vision 上 38.3%的成绩。在 8B 规模模型中，其性能相比在开源数据集训练的模型有显著提升，并且与闭源数据训练的模型相当甚至更优。代码、模型和数据集可在此 https URL 获得。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19100 (HTTP 429)

Authors: Qingsong Xie, Jiahao Tian, Jiacheng Tang, Zhifeng Li, Madalina Ciobanu, Navan Preet Singh, Qingqing Mao, Ritankar Das

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19100.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19100

Published: 2026-10-16T01:39:49.113Z

9. Plant Memory: Technical Language Processing and a Verified Maintenance Ontology for Agent-Accessible Failure Knowledge from SAP PM Work Orders

中文摘要

摘要：维护工单是工业设备故障情况的最大书面记录，但在大多数工厂中，它们几乎无法用于分析。文本简短，存在缩写和拼写错误，资产身份埋藏在自由文本中，工作类型标签与所描述的工作相矛盾，而可靠性最重要的字段（症状、根本原因、采取的措施）通常为空。因此，诊断知识停留在高级技术人员手中，并随着他们的离开而消失。我们提出了Plant Memory系统，该系统将技术语言处理（TLP）应用于SAP Plant Maintenance（PM）导出的数据，并构建一个符合ISO 14224标准的维护本体，其中每个提取的陈述都是一个具化的事实，包含置信度、读取的证据以及验证状态。提取过程以规则为主，同时可选的大型语言模型（LLM）作为第二读者，其分歧会提高事实的审查优先级，而不是覆盖原有事实。技术人员通过审查队列验证事实，针对重复发生但未记录原因的故障回答有针对性的知识空缺问题，并在出现新术语时批准或拒绝本体漂移建议。生成的知识图谱通过网页应用提供给人类用户，通过模型上下文协议（MCP）服务器提供给AI代理，从而使两者查询相同的已验证知识，每个答案都引用背后的工单。在一份来自一个预煮米厂的400份工单为期12个月的合成导出数据上，基于规则的提取实现了96.2%的组件准确率，88.9%的故障模式准确率，以及62%覆盖范围下100%的根本原因精度；在28个诊断问题上，本体中的排名根本原因在首位的正确率为82.1%，而SAP原始根本原因列的正确率为53.6%。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19131 (HTTP 429)

Authors: Akash Raj

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19131.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19131

Published: 2026-10-16T01:39:49.113Z

10. Graph-Based Orchestration of Heterogeneous AI Models: A Control Node Approach to Composite Intelligence

中文摘要

摘要：我们提出了一个基于图的复合人工智能系统框架，该系统整合了多种人工智能技术以解决复杂问题。我们描述了一种使用专用控制节点来协调多样化模型的架构，包括规划、机器学习、约束编程和大型语言模型。该系统使用有向无环图和行为树来管理模型关系和执行顺序。我们的方法涵盖数据连接器、计算服务以及支持层级、顺序和并行模型组合模式的容器化部署框架。通过案例研究，我们展示了该架构如何实现复合系统的创建，在统一的操作模式下利用多种人工智能技术的优势，从而克服单一模型方法的局限性。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19136 (HTTP 429)

Authors: G. Michael Youngblood, Filip Dvořák, Slavo Švancár, Tomáš Balyo, Michal Ficek, Martin Dousek

Categories: cs.AI

PDF URL: https://arxiv.org/pdf/2610.19136.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19136

Published: 2026-10-16T01:39:49.113Z

VLM Domain Papers

1. Action Jets: Learning the Local Shape of the Action-conditioned Data Manifold

Abstract:World models are a powerful means of reasoning and planning, but many of these action-conditioned predictive models are high dimensional and computationally expensive. These challenges limit the number of action candidates that can be evaluated by a planner. We show that this cost is avoidable for planners over short horizons, which only require local information with respect to the current state. For a fixed state, the frames reachable by varying a low-dimensional action (a camera movement or an end-effector command) lie on a low-dimensional manifold. We estimate the local shape of that manifold directly via a network trained to estimate how each image region moves per unit of each action axis supervised by data collected from samples by the generator. A correction head fit to trajectory data estimates the shape of this manifold region up to a third-order Taylor polynomial. We call this network estimating the Taylor polynomial describing the manifold shape an Action Jet and compute its radius of validity in closed form. By traversing this approximation of the local manifold shape, we only have to perform inference once per state for any number of candidate actions. Scoring sixteen candidates from a single extraction is over 300x faster than with either source generator, and this gap widens with the number of candidates a planner evaluates. We measure against state-of-the-art action-conditioned generators, IRASim and WorldGym on BridgeData V2, a real 7-DoF manipulation benchmark, and FrameCrafter on DL3DV scenes, where we chart from the novel-view generators DepthSplat and LagerNVS. Because only the tangent comes from the generator and every other term from the data, the Action Jet also predicts held-out frames more accurately than IRASim, the generator that supplied its tangent, at short horizons and at over 100x less compute.

中文摘要

摘要：世界模型是一种强大的推理和规划工具，但许多基于动作的预测模型具有高维度和计算成本高的特点。这些挑战限制了规划者可以评估的动作候选数量。我们展示了对于短期规划，这种成本是可以避免的，因为短期规划只需要相对于当前状态的局部信息。在固定状态下，通过改变低维动作（例如摄像机移动或末端执行器指令）可到达的帧位于低维流形上。我们通过训练网络直接估计该流形的局部形状，该网络用于估计数据生成器样本监督下每个图像区域在每个动作轴每单位动作下的移动。一个拟合轨迹数据的修正头估计该流形区域的形状，直到三阶泰勒多项式。我们称这个估计描述流形形状的泰勒多项式的网络为动作喷射（Action Jet），并以闭式形式计算其有效半径。通过遍历这个局部流形形状近似，我们只需要对每个状态执行一次推理，即可评估任意数量的动作候选。从单次提取中对十六个候选进行评分，比使用任一源生成器快300倍以上，而且随着规划者评估的候选数量增加，这一差距还会扩大。我们在BridgeData V2（一个真实的7自由度操作基准）上，以及在DL3DV场景上的FrameCrafter上，对最先进的动作条件生成器IRASim和WorldGym进行了测量，同时我们还从新视角生成器DepthSplat和LagerNVS进行了分析。由于只有切线来自生成器而其他所有项来自数据，动作喷射在短期预测和计算量减少超过100倍的情况下，对未观察帧的预测精度也高于其切线来源的生成器IRASim。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19007 (HTTP 429)

Authors: Anton Arapin, Chahyon Ku, Bernadette Bucher

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19007.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19007

Published: 2026-10-16T01:40:09.817Z

2. How Far Can Coding Agents Go in Visual Generation?

Abstract:Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: general-purpose coding agents that construct images through executable programs. Our study of zero-shot drawing on AgentDrawBench, a 200-prompt benchmark, reveals a distinctive capability profile. Agents excel at preserving structure and satisfying precise visual constraints, achieving competitive results in domains where compositional accuracy matters more than photorealistic appearance, while remaining limited in perceptual richness. Their visual expressiveness varies with the drawing interface, and a qualitative study on reconstruction shows how procedural skills extracted from human demonstrations support different artistic media. We then examine code as an external visual language of thought: a structured, executable representation that agents can construct and inspect. We test it in visual question answering task and find reconstructed visual code retains substantial information for question answering and, in some domains, supports higher accuracy than the original image. These abilities raise a further question: do agents draw on a shared internal representation of scene content beyond a particular drawing program? We explore an internal visual language-of-thought hypothesis, in which a shared, compositionally structured representation supports visual operations across drawing languages. Cross-harness drawings and images assembled from independent single-pixel queries exhibit related layouts and tones. Agents also maintain self-generated room arrangements across viewpoints after visible drawings are removed. These results demonstrate that the coding agent would maintain a consistent and coherent world inside its imagine even before the harness gets involved and the generation starts.

中文摘要

摘要：现代图像生成在很大程度上依赖于专门的生成模型，这些模型将图像构建过程隐式地保存在学习到的参数中。我们探索一种替代方法：通用编码代理，它们通过可执行程序来构建图像。我们在AgentDrawBench（一个包含200个提示的基准测试）上对零样本绘图的研究揭示了其独特的能力特征。代理在保持结构和满足精确视觉约束方面表现出色，在组合精度比照片写实效果更重要的领域取得了有竞争力的结果，但在感知丰富性方面仍然有限。它们的视觉表现力随绘图界面而变化，对重建的定性研究显示，从人类示范中提取的程序化技能如何支持不同的艺术媒介。随后，我们将代码视为一种外部视觉思维语言：一种结构化的、可执行的表示，代理可以构建和检查。在视觉问答任务中进行测试发现，重建的视觉代码保留了大量用于问答的信息，并且在某些领域可以支持比原始图像更高的准确率。这些能力提出了一个进一步的问题：代理是否在特定绘图程序之外利用共享的场景内容内部表示？我们探索了内部视觉思维语言假说，其中共享的、组合结构化的表示支持跨绘图语言的视觉操作。通过独立单像素查询组装的交叉操作绘图和图像显示了相关的布局和色调。代理在可见绘图被移除后，也能保持自生成空间布置在不同视角下的一致性。这些结果表明，编码代理甚至在挂载和生成开始之前，就能在其想象中保持一个一致且连贯的世界。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19025 (HTTP 429)

Authors: Hengguang Zhou, Yuanhao Ban, Jiaqi Feng, Xiaohuan Pei, Justin Cui, Cho-Jui Hsieh

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19025.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19025

Published: 2026-10-16T01:40:09.817Z

3. Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment

Abstract:Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems. However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the multimodal understanding and interaction capabilities of VLMs. We revisit RPM with VLMs and provide empirical evidence for two major bottlenecks: generic visual semantic representations fail to preserve fine-grained physiological cues; while discrete token prediction in autoregressive generation is inherently misaligned with continuous numerical measurement. To address these challenges, we propose VITAL (Visual-physiological Token Alignment for Language Models), a visual-physiological token alignment framework for RPM and preserving general vision tasks. VITAL retains the original visual pathway for semantic understanding and introduces an independent physiological pathway to jointly model semantic and physiological information. Moreover, we propose a structured numerical alignment approach to improve the reliability of generative heart rate estimation by enhancing numerical relationship modeling. Extensive experiments show that VITAL improves cross-domain estimation performance on multiple rPPG datasets while preserving the general visual understanding capability of VLMs, demonstrating its effectiveness as a unified visual-physiological model. Code will be released.

中文摘要

摘要：基于视频的远程生理测量（RPM）具有广泛的应用，而视觉-语言模型（VLMs）为未来具备人类感知能力的智能系统实现端到端生理监测提供了有前景的机会。然而，现有的RPM方法依赖于特定任务的模型，无法与VLMs的多模态理解和交互能力无缝集成。我们重新审视了基于VLMs的RPM，并提供了两个主要瓶颈的实证证据：通用视觉语义表示无法保留细粒度的生理线索；而自回归生成中的离散令牌预测本质上与连续数值测量不完全对齐。为应对这些挑战，我们提出了VITAL（视觉-生理令牌对齐语言模型，Visual-physiological Token Alignment for Language Models），这是一个用于RPM并能保留通用视觉任务的视觉-生理令牌对齐框架。VITAL保留了原有的视觉通路用于语义理解，并引入了独立的生理通路以联合建模语义和生理信息。此外，我们提出了一种结构化数值对齐方法，通过增强数值关系建模来提高生成心率估计的可靠性。大量实验表明，VITAL在多个rPPG数据集上提升了跨域估计性能，同时保留了VLMs的通用视觉理解能力，证明了其作为统一视觉-生理模型的有效性。代码将公开发布。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19052 (HTTP 429)

Authors: Xiao Yang, Zekun Zhang, Yuxuan Fan, Duoxun Tang, Ziyang Liu, Dengbo He, Kaishun Wu, Jiyao Wang

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19052.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19052

Published: 2026-10-16T01:40:09.817Z

4. WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

Abstract:As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requires both preserving previously observed content and ensuring coherent world evolution over time. World memory is a critical capability to evaluate alongside visual fidelity and video duration. We introduce WorldMemBench, a benchmark for systematically evaluating world memory in video world models. Across 260 test cases, it assesses static memory through the retention of human identity and appearance, object appearance and geometry, and environmental content and geometry, and dynamic memory through event progression and state persistence. Using controlled camera departures and multi-revisit trajectories, the benchmark evaluates static content against fixed initial references and dynamic events against expected state transitions. It combines appearance and geometry metrics with event-specific assessments, and reports video quality and camera-following accuracy as complementary diagnostics. Evaluation of nine representative models shows that strong static retention does not imply reliable dynamic state evolution, while repeated revisits reveal memory degradation beyond the first return. Human studies support the alignment between our automatic evaluations and human judgments. WorldMemBench provides a systematic testbed for diagnosing memory failures and guiding the development of persistent, temporally coherent video worlds.

中文摘要

摘要：随着视频模型能够提供更长时间、更真实的互动体验，保持世界与其历史一致变得至关重要。这既需要保留先前观察到的内容，也需要确保世界随时间的演变保持连贯。世界记忆是评估视觉逼真度和视频时长之外的关键能力。我们提出了 WorldMemBench，一个用于系统评估视频世界模型中世界记忆的基准。在 260 个测试用例中，它通过保留人类身份和外观、物体外观和几何形状、环境内容和几何形状来评估静态记忆，通过事件进展和状态延续来评估动态记忆。通过受控相机移动和多次重访轨迹，该基准将静态内容与固定初始参考进行评估，并将动态事件与预期状态转变进行对比。它结合了外观和几何度量指标与特定事件评估，并报告视频质量和跟随相机的准确性作为补充诊断。对九个代表性模型的评估表明，强大的静态保持性并不意味着动态状态演变可靠，而重复重访揭示了首次返回之后的记忆退化。人类研究支持了我们的自动评估与人类判断之间的一致性。WorldMemBench 提供了一个系统化的测试平台，用于诊断记忆失败并指导持久、时间上连贯的视频世界的开发。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19063 (HTTP 429)

Authors: Nan Zhou, Jianxiong Gao, Qiuyu Wang, Wen Wang, Jiahao Wang, Yufeng Yuan, Shuaiting Li, Hao Ouyang, Yujun Shen, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19063.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19063

Published: 2026-10-16T01:40:09.817Z

5. MonoMIP: Multi-Height Inverse Perspective Mapping for Monocular 3D Object Detection

Abstract:Monocular 3D object detection methods increasingly rely on depth-guided transformers, but they operate entirely in the perspective image plane where spatial relationships are distorted by foreshortening. We propose MonoMIP, a dual-decoder framework that adds complementary 3D geometric reasoning through Bird’s-Eye View (BEV) features constructed via Multi-Height Inverse Perspective Mapping. We perform IPM at K elevation planes spanning ground to rooftop, producing an ordered stack of height-conditioned metric BEV features that reduces above-ground misprojection and exposes depth-dependent appearance variation across planes. A BEV decoder with multi-scale deformable attention aggregates this multi-height evidence per query, while a depth-guided 3D decoder provides complementary perspective-space predictions. Without extra training data, MonoMIP attains 25.84% Moderate AP3D|R40 on KITTI val and 21.66% on the official test set for Car, outperforming recent published monocular methods on both the KITTI and Waymo benchmarks.

中文摘要

摘要：单目三维物体检测方法越来越依赖深度引导的变换器，但它们完全在透视图像平面上操作，在该平面上空间关系会因缩短效应而失真。我们提出了 MonoMIP，一种双解码器框架，通过通过多高度逆透视映射构建的鸟瞰图（BEV）特征增加了互补的三维几何推理。我们在从地面到屋顶的 K 个高度平面上执行 IPM，生成有序堆叠的高度条件化度量 BEV 特征，这减少了地面以上的投影误差，并揭示了跨平面的深度依赖外观变化。具有多尺度可变形注意力的 BEV 解码器为每个查询聚合这些多高度证据，同时深度引导的三维解码器提供互补的透视空间预测。在无需额外训练数据的情况下，MonoMIP 在 KITTI 验证集上对车辆实现了 25.84% 的 Moderate AP3D|R40，在官方测试集上为 21.66%，在 KITTI 和 Waymo 基准上均优于最近发表的单目方法。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19071 (HTTP 429)

Authors: Emir Ulurak, Furkan Kıraç

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19071.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19071

Published: 2026-10-16T01:40:09.817Z

Abstract:Aerial and remote sensing video is increasingly evaluated like web video, by posing multiple-choice questions to multimodal large language models (MLLMs) over sampled frames and reporting accuracy. This paper argues that this recipe measures the wrong aspect, and supports the claim with a convergent meta-analysis of a broad set of published results, organized in four evidence tiers, in which a claim becomes a position only when collected evidence agrees. Parameter scale exhibits a median of +3.4 accuracy points per tenfold increase on aerial benchmarks, far below a human gap of at least 19 points, whereas acquisition conditions control perception, as 315 of 318 evaluations in 19 within-paper contrasts lose accuracy when conditions such as altitude, view, object size or time horizon become harder. No aerial benchmark in the corpus combines video input with per-task human and random baselines, competence collapses from answering to acting in closed-loop navigation, and the same MLLMs score higher on the aerial benchmark than on ground spatial ones, but lower on studies that test cross-view geometry. From these findings the current study derives five falsifiable principles and proposes the AVU-Card, a reporting card for aerial video benchmarks

中文摘要

摘要：航拍和遥感视频越来越像网络视频一样被评估，通过在采样帧上向多模态大型语言模型（MLLMs）提出多项选择题并报告准确率。本文认为这种方法测量的是错误的方面，并通过对一组广泛发表的结果进行汇聚元分析来支持这一观点，这些结果按四个证据层次组织，只有在收集的证据一致时，一个主张才成为立场。参数规模在航拍基准测试上的中位数表现为每增加十倍提高3.4点准确率，远低于至少19点的人类差距，而采集条件则控制感知，因为在19个同文对比中有318次评估中有315次在海拔、视角、物体大小或时间范围变得更困难时准确率下降。语料库中没有任何航拍基准将视频输入与每项任务的人类和随机基线结合，能力从回答问题到闭环导航中的行动会坍塌，同一MLLM在航拍基准上的得分高于地面空间基准，但在测试跨视角几何的研究中得分较低。基于这些发现，本研究提出了五个可证伪原则，并提出了AVU-Card，一种针对航拍视频基准的报告卡。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19102 (HTTP 429)

Authors: Georgios Angelidis, Aggelos Psiris, Maria Vakalopoulou, Georgios Th. Papadopoulos, Giorgos Sfikas, Iraklis Varlamis

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19102.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19102

Published: 2026-10-16T01:40:09.817Z

7. Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

Abstract:Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such approaches. In this survey, we consider approaches to fine-tuning pretrained MLLMs for spatial reasoning tasks, ranging from supervised to reinforcement learning, with an emphasis on the datasets used. We investigate themes of dataset construction, data modality and annotation, and evaluation of the resulting models. Based on these themes, we categorize research into three groups: dataset design, dataset construction and evaluation. We find four main methods for constructing fixed spatial annotations: rules and templates, LLM prompting, procedural generation, and human-in-the-loop. We also observe substantial variation in evaluation approaches, including different scoring methods, tolerance metrics, and benchmark types. Finally, we identify opportunities for future research in unified dynamic annotation, efficient spatial reasoning, standardization of evaluation, expanded spatial range, and human-centered spatial interactions.

中文摘要

摘要：多模态大型语言模型（MLLMs）已广泛应用于具身智能体和机器人领域的视觉任务。然而，MLLMs在空间推理和感知方面存在局限性。后训练技术在提升这些能力方面显示出潜力；然而，目前针对这类方法的数据驱动型综述非常有限。在本综述中，我们考虑了针对空间推理任务对预训练MLLMs进行微调的方法，涵盖从监督学习到强化学习的多种策略，并重点关注所使用的数据集。我们研究了数据集构建、数据模态与标注，以及生成模型评价等主题。基于这些主题，我们将研究分类为三类：数据集设计、数据集构建和评价。我们发现构建固定空间标注的主要方法有四种：规则与模板、LLM提示、程序化生成以及人类参与。此外，我们还观察到评价方法存在显著差异，包括不同的评分方法、容差度量和基准类型。最后，我们识别出未来研究的机会，包括统一的动态标注、高效的空间推理、评价标准化、扩展空间范围以及以人为中心的空间交互等方向。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19108 (HTTP 429)

Authors: Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh, Sheila Schoepp, Osmar R. Zaiane

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19108.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19108

Published: 2026-10-16T01:40:09.817Z

8. Agronomic Characterization and Time-Series UAV Phenotyping to Support Field Pea Breeding in Iowa

Abstract:Field pea has gained interest in the Midwest U.S. and has the potential to benefit Iowa cropping systems. This study evaluated 17 accessions and 7 Canadian varieties for agronomic and seed yield performance. This study found that PI accession trials exhibit genetic diversity in phenology and canopy traits, whereas the Canadian elite varieties demonstrated great yield potential under Iowa conditions. The accession genotypes matured between 75 DAP (late June) and 95 DAP (late July) across six site-years. The top-yielding Canadian varieties averaged 1961 kg/ha across two site-years. To comprehensively assess field pea performance, this study extracted time-series Unmanned Aerial Vehicle(UAV)-derived traits, assessed UAV-derived traits, and evaluated their correlations with multiple agronomic traits and yield. The results found that UAV-derived time-series canopy area was the most yield-relevant trait (r = 0.63) among all UAV-derived canopy features, vegetation indices, and manually collected measurements. Compared with yield, UAV-derived canopy area showed higher selection reliability (H2 = 0.87, CV = 13.46%), making it suitable for phenotypic selection and accelerating Midwest field pea breeding programs. This study identifies useful UAV-derived features for pea breeding and demonstrates the value of integrating diverse germplasm, elite commercial varieties, and time-series UAV phenotyping. Combining genetic variation from the accession trial with the yield potential of high-performing commercial varieties provides valuable breeding materials and phenotyping strategies for developing locally adapted, high-yielding field pea varieties for future production in the Midwest.

中文摘要

摘要：铁豌豆在美国中西部引起了关注，并有潜力惠及爱荷华州的种植系统。本研究评价了17个来源材料和7个加拿大品种的农艺性状和种子产量表现。研究发现，PI来源材料试验在物候和冠层性状上表现出遗传多样性，而加拿大优秀品种在爱荷华州条件下表现出较高的产量潜力。来源材料基因型的成熟期在六个地点年份中为75天后开花（6月底）至95天后开花（7月底）。产量最高的加拿大品种在两个地点年份的平均产量为1961千克/公顷。为了全面评估铁豌豆的表现，本研究提取了无人机（UAV）时序衍生性状，评估了UAV衍生性状，并分析了它们与多种农艺性状及产量的相关性。结果发现，UAV衍生的时序冠层面积在所有UAV衍生的冠层特征、植被指数和人工收集测量值中，与产量的相关性最高（r = 0.63）。与产量相比，UAV衍生的冠层面积表现出更高的选择可靠性（H2 = 0.87，CV = 13.46%），使其适合用于表型选择，并加快中西部铁豌豆育种计划。研究确定了对豌豆育种有用的UAV衍生特征，并展示了整合多样化种质、优良商业品种和时序无人机表型的价值。将来源材料试验中的遗传变异与高产商业品种的产量潜力结合，为未来在中西部生产地区开发本地适应性高产铁豌豆品种提供了宝贵的育种材料和表型策略。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19154 (HTTP 429)

Authors: Yanben Shen, Joscif Raigne, Asheesh Singh, Arti Singh

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19154.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19154

Published: 2026-10-16T01:40:09.817Z

9. Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors

Abstract:We present LidarVerse, a real-time unified LiDAR world model for sensor simulation across tasks and sensors. Existing LiDAR simulators largely follow one of two complementary paradigms: reconstruction-based methods preserve the geometry of a captured scene but require lengthy per-scene optimization, whereas generative methods amortize learning across data but lack the consistency needed for long-horizon rollouts. LidarVerse brings both paradigms into a single, unified conditional generative framework. A universal tokenizer maps LiDAR range sequences into a shared latent space, where a bidirectional teacher learns a common interface for conditioning on a LiDAR prefix, an HD-Map, a persistent spatial cache, and optionally, text. Varying the available scene evidence turns the same model into a generator of new worlds, a point-cloud forecaster, or a generalizable resimulator of observed scenes without per-scene optimization. We distill the teacher into a four-step autoregressive causal student and maintain both a rolling temporal K/V cache and a world-aligned spatial cache, enabling persistent long-horizon rollouts with real-time inference at 11 FPS on a single H100 GPU. LidarVerse achieves state-of-the-art performance on most reported metrics across all three evaluation protocols, while its tokenizer supports 32-, 64-, and 128-beam LiDAR and generalizes to an unseen scanning pattern. These results demonstrate that generative and reconstruction-grounded LiDAR simulation can be realized within one model, providing the core capabilities for future closed-loop sensor simulation.

中文摘要

摘要：我们提出了 LidarVerse，一种用于跨任务和传感器的实时统一 LiDAR 世界模型传感器仿真。现有的 LiDAR 仿真器大多遵循两种互补的范式之一：基于重建的方法保留了捕获场景的几何信息，但每个场景都需要耗时的优化；而生成方法则通过数据学习实现了成本分摊，但缺乏长时间滚动所需的一致性。LidarVerse 将这两种范式整合到一个统一的条件生成框架中。一个通用分词器将 LiDAR 距离序列映射到共享的潜在空间，其中双向教师模型学习了一个通用接口，以便在 LiDAR 前缀、HD 地图、持久空间缓存以及可选的文本条件下进行生成。通过改变可用的场景证据，同一模型可以作为新世界生成器、点云预测器，或在无需每个场景优化的情况下对观察到的场景进行可泛化的重新仿真。我们将教师模型蒸馏为四步自回归因果学生模型，并同时维护滚动时间 K/V 缓存和世界对齐空间缓存，实现持久的长时间滚动，并在单个 H100 GPU 上以 11 FPS 的实时推理速度运行。LidarVerse 在三个评估协议下的大多数已报告指标上都实现了最先进的性能，且其分词器支持 32、64 和 128 光束 LiDAR，并可泛化至未见过的扫描模式。这些结果表明，生成式与基于重建的 LiDAR 仿真可以在一个模型中实现，为未来闭环传感器仿真提供核心能力。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19201 (HTTP 429)

Authors: Henry Che, Xuanchi Ren, Sanja Fidler, Shenlong Wang, Laura Leal-Taixé, Shengyu Huang

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19201.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19201

Published: 2026-10-16T01:40:09.817Z

10. Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction

Abstract:World models of visual information predict future image frames conditioned on given action sequences. This predicted scene evolution is driven by deterministic changes such as controlled motion on a robot platform, as well as stochastic changes such as rendering of manipulated objects or previously occluded regions. Our key insight is that the spatial location of these stochastic changes is itself learnable. We leverage this insight by jointly regressing both the deterministic next-state prediction and the spatial uncertainty map, then using the uncertainty map to guide a conditional diffusion process. We propose Decoupled Drift-Diffusion World Model (D3WM), which uses a transformer trained with a Gaussian negative-log-likelihood objective to produce both the deterministic next-state prediction and a spatial map of heteroscedastic aleatoric uncertainty. The learned uncertainty map then provides a structured spatial gate for a conditional diffusion model: deterministic regions are anchored to the predicted state, while uncertain regions, such as disocclusions or dynamic textures, are sampled by the diffusion process. We frame this decomposition under a variance-gated stochastic differential equation that separates deterministic drift from stochastic diffusion. D3WM shows improved results over competitive baselines in two distinct types of action spaces: robotic tabletop manipulation and camera motion.

中文摘要

摘要：视觉信息的世界模型在给定动作序列的条件下预测未来的图像帧。这种预测的场景演变由确定性变化（如机器人平台上的受控运动）以及随机变化（如被操作物体的渲染或之前被遮挡的区域）驱动。我们的关键见解是，这些随机变化的空间位置本身是可学习的。我们利用这一见解，通过联合回归确定性下一状态预测和空间不确定性图，然后使用不确定性图来指导条件扩散过程。我们提出了分离漂移扩散世界模型（D3WM），该模型使用经过高斯负对数似然目标训练的变压器，同时生成确定性下一状态预测和异方差偶然性不确定性的空间图。然后，学习到的不确定性图为条件扩散模型提供了结构化的空间门控：确定性区域被固定在预测状态，而不确定区域（例如消遮挡区域或动态纹理）由扩散过程进行采样。我们将这种分解框架化在方差门控的随机微分方程下，将确定性漂移与随机扩散分开。D3WM在两种不同类型的动作空间中——机器人桌面操作和相机运动——相较于竞争基线显示出改进的结果。

LLM Analysis

LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.19212 (HTTP 429)

Authors: Anton Arapin, Aditya Singh, Yixin Wang, Jun Gao, Bernadette Bucher

Categories: cs.CV

PDF URL: https://arxiv.org/pdf/2610.19212.pdf

CoolPaper URL: https://papers.cool/arxiv/2610.19212

Published: 2026-10-16T01:40:09.817Z

ArXiv Domain 2026-10-16

LLM Domain Papers

1. Following the Flow the Law Requires: Training Voice Agents to Transfer to Unseen Regulated Procedures in Debt Collection

2. Evaluating Large Language Models for Multilingual Abstractive Summarization: A Comprehensive Study of English and Bengali

3. Can LLMs Access their World Knowledge for Event Prediction?

4. Joint Subspace Sparse Attention for Efficient Long-Context Decoding

5. Crossing the Gender Binary: Modelling Gendered Reference in Science Fiction and General Fiction

6. Pre-Trained Language Models as Zero-Shot Tools for Social Science Research

7. Similar Models, Broken Odds: Where Luce’s Axiom Fails in Chatbot Arena

8. From Copying to Composition: How Language Models Develop In-Context Learning

9. DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision

10. CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition

Agent Domain Papers

1. Why Static Skill Inconsistencies Do Not Equate to Runtime Risks: A Behavioral Analysis of LLM Agent Execution

2. The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

3. Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench

4. Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

5. Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

6. RLPE: Teaching LLMs to Prompt with Reinforcement Learning

7. Plan or Retrieve First? A Scale Study of Tool Discovery Architectures

8. GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning

9. Plant Memory: Technical Language Processing and a Verified Maintenance Ontology for Agent-Accessible Failure Knowledge from SAP PM Work Orders

10. Graph-Based Orchestration of Heterogeneous AI Models: A Control Node Approach to Composite Intelligence

Evaluation Domain Papers

1. Why Static Skill Inconsistencies Do Not Equate to Runtime Risks: A Behavioral Analysis of LLM Agent Execution

2. The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

3. Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench

4. Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

5. Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

6. RLPE: Teaching LLMs to Prompt with Reinforcement Learning

7. Plan or Retrieve First? A Scale Study of Tool Discovery Architectures

8. GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning

9. Plant Memory: Technical Language Processing and a Verified Maintenance Ontology for Agent-Accessible Failure Knowledge from SAP PM Work Orders

10. Graph-Based Orchestration of Heterogeneous AI Models: A Control Node Approach to Composite Intelligence

VLM Domain Papers

1. Action Jets: Learning the Local Shape of the Action-conditioned Data Manifold

2. How Far Can Coding Agents Go in Visual Generation?

3. Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment

4. WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

5. MonoMIP: Multi-Height Inverse Perspective Mapping for Monocular 3D Object Detection

6. The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understanding

7. Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models

8. Agronomic Characterization and Time-Series UAV Phenotyping to Support Field Pea Breeding in Iowa

9. Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors

10. Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction