ArXiv Domain 2026-10-03
数据来源:ArXiv Domain
LLM Domain Papers
1. On-Device Named-Entity Recognition: A Deployability Study of Accuracy, Cost, Reliability, and Confidence
Abstract:Named-entity recognition (NER) is increasingly wanted on-device (no API, low latency, data kept local). The practitioner’s question is not the leaderboard but which model is deployable, how to evaluate it without human annotation, and whether its confidence can be trusted. We answer these jointly. We place nine systems across three paradigms and 13 M to 8 B parameters: a classical tagger (spaCy), bidirectional-encoder specialists (GLiNER, 166 to 460 M), and generative LLMs run locally (Qwen3-0.6B/1.7B/4B-Instruct, DeepSeek-R1-1.5B/8B), on three datasets of differing character, and report accuracy plus two axes the literature omits: latency and output validity. Because our corpus (RSS-News) had no gold, we built silver gold from a cross-family LLM judge panel, then measured its fidelity against benchmark gold and a full human re-validation of the corpus (strict F1 0.95, an upper bound since the human gold was silver-seeded); gold provenance flips the paradigm ranking, moving from LLM-authored silver to human gold raises every encoder and lowers every generative model. On accuracy alone a 4 B instruct LLM is competitive (it leads on clean newswire), so the encoder’s case is deployability: it matches or slightly trails at one-ninth to one-twenty-fourth the size, at millisecond-to-second latency, with zero malformed output, while the smallest generative models emit up to 27% invalid output on long inputs, a failure fixed by scale, not output budget. We then characterize GLiNER’s per-span confidence: it ranks correctness well (AUROC 0.76 to 0.86) but is overconfident (ECE 0.24 to 0.47, halved by temperature scaling); thresholding gives a small honest out-of-sample F1 gain; an all-local small-to-large cascade gives a modest, corpus-dependent gain over cost-matched random routing; and confidence tracks correctness but not novelty. Every number recomputes offline from per-span records.
中文摘要
摘要:命名实体识别(NER)对设备端的需求日益增加(无需 API、低延迟、数据保留在本地)。实践者关注的问题不是排行榜,而是哪个模型可部署、如何在无需人工标注的情况下评估它,以及其置信度是否可靠。我们将这些问题综合回答。我们在三种范式和 1,300 万到 80 亿参数之间测试了九个系统:传统标注器(spaCy)、双向编码器专用模型(GLiNER,1.66 亿到 4.6 亿)以及本地运行的生成式大模型(Qwen3-0.6B/1.7B/4B-Instruct,DeepSeek-R1-1.5B/8B),在三种不同特性的 数据集 上进行测试,并报告准确率,以及文献中通常忽略的两个维度:延迟和输出有效性。由于我们的语料库(RSS-News)没有金标,我们从跨家族大模型评审小组构建了银标金,然后将其与基准金标和语料库的完整人工复验证(严格 F1 0.95,人类金标是以银标为种子的上限)进行忠实度测量;金标来源会改变范式排名,从大模型生成的银标换成人工金标会提升每个编码器的表现,并降低所有生成模型的表现。仅在准确率上看,一个 40 亿参数的指令型大模型具有竞争力(在干净新闻语料上领先),所以编码器的优势在于可部署性:它在模型大小仅为四分之一到二十四分之一时表现相当或略低,延迟为毫秒至秒,并且输出完全无畸形,而最小的生成模型在处理长输入时最多会产生 27% 无效输出,这一问题通过增加模型规模,而非输出预算得以解决。然后,我们描述了 GLiNER 的每个跨度置信度:它对正确性排序良好(AUROC 0.76 到 0.86),但自信过高(ECE 0.24 到 0.47,通过温度缩放可减半);阈值设定可以带来小幅、诚实的样本外 F1 提升;全本地小到大的级联相比成本匹配的随机路由可带来适度、依赖语料库的增益;且置信度追踪正确性,但不追踪新颖性。所有数字均从每个跨度的记录离线重新计算。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00007 (HTTP 429)
Authors: Vinay Kumar Chaganti
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00007.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00007
Published: 2026-10-03T02:19:11.047Z
2. SCM-based Fairness and Faithful Explainability for Legal Document Classification
Abstract:Transformer models such as LegalBERT are increasingly used in legal decision support, raising concerns about both fairness and the transparency of model explanations. These properties are usually evaluated separately, leaving open whether a debiasing intervention that changes fairness also changes how faithfully explanations reflect model reasoning. This study investigates that relationship on the ECtHR alleged-violations corpus from LexGLUE. It compares a LegalBERT baseline with a fairness-regularized variant that penalizes stereotypical warmth and competence representations during fine-tuning. The evaluation covers predictive performance, demographic fairness, and SHAP explanation faithfulness across five random seeds. At the performance-optimal regularization strength, the intervention does not reduce demographic disparity. This null result holds across two fairness definitions and a conventional word-pair control on the gender axis. Classification performance is largely unchanged. However, the intervention consistently degrades explanation sufficiency across all five seeds and three thresholds. A shuffled-pair control reproduces this degradation while leaving performance and fairness unchanged, indicating that the effect arises from contrastive representational regularization rather than specifically from the warmth and competence structure. The results demonstrate a dissociation between fairness and explanation faithfulness: changes in explanation behavior do not necessarily indicate changes in fairness, and fairness must therefore be evaluated directly.
中文摘要
摘要:诸如 LegalBERT 的 Transformer 模型在法律决策支持中越来越多地被使用,这引发了关于公平性和模型解释透明度的担忧。这些属性通常是分别评估的,因此尚不清楚改变公平性的去偏干预是否也会改变解释对模型推理的忠实反映。本研究在 LexGLUE 的 ECtHR 违反指控语料库上调查了这种关系。研究比较了 LegalBERT 基线模型与在微调过程中对刻板的温暖和能力表示施加惩罚的公平性正则化变体。评估内容包括预测性能、人群公平性及 SHAP 解释的忠实性,并跨五个随机种子进行。在性能最优的正则化强度下,该干预并未减少人口差异。这一零结果在两种公平性定义下以及性别轴上的常规词对控制中均成立。分类性能基本保持不变。然而,该干预在所有五个随机种子和三个阈值下都持续降低了解释充分性。打乱对照组复现了这种降低,同时性能和公平性保持不变,表明该效应源于对比表示正则化,而不是特定的温暖和能力结构。结果显示了公平性与解释忠实性之间的分离:解释行为的变化不一定意味着公平性的变化,因此必须直接评估公平性。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00045 (HTTP 429)
Authors: Yasmina El Kacemi, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansoor Alsahag
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00045.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00045
Published: 2026-10-03T02:19:11.047Z
3. The First Token Is Not the Verdict: Hidden Costs of Reading LLM Judges Without Generating
Abstract:Reading an LLM judge’s verdict from the logits of its first generated token is cheap, requires no generation, and is exactly what constrained decoding and likelihood-scoring evaluation harnesses produce. We show that this readout distorts position bias in one direction: it overstates it in every condition we test, so figures obtained this way behave as upper bounds. The mechanism is that judges do not always lead with a verdict token, on 12% to 49% of pairs for three Qwen3 judges and under 3% for Llama-3.1-8B and Phi-3.5-mini, and forcing a read on those pairs returns whichever response was shown first rather than a judgment. Pooled over the 924 pairs where a judge did not commit, the forced read flips on 89.7% of them when the responses are swapped, against 47.5% read after generation (paired difference +0.422, 95% CI [+0.365, +0.467]). The distortion is specific to what is measured: it moves position bias by 42 points while moving judge accuracy by under one point in seven of ten conditions, so it misleads whoever audits a judge rather than whoever uses one. A second, smaller failure occurs even when the judge does lead with a verdict token, since it sometimes opens with one letter and reasons its way to the other, on 0 to 5.5% of pairs at a rate uncorrelated with compliance. We recommend reporting the rate at which a judge leads with a verdict token, which costs one forward pass and no labels, alongside any position-bias figure.
中文摘要
摘要:从大型语言模型(LLM)法官生成的第一个标记的 logits 中读取其裁决是廉价的,不需要生成,并且正是受限解码和似然评分评估所利用的产物。我们展示了这种读取会在一个方向上扭曲位置偏差:在我们测试的每种条件下它都被高估,所以以这种方式获得的数据表现为上限。其机制在于法官并不总是以裁决标记开头,在三个 Qwen3 法官的 12% 到 49% 的案例对中如此,而在 Llama-3.1-8B 和 Phi-3.5-mini 中这一比例低于 3%,强制在这些案例对上读取会返回最先显示的响应而不是裁决。在 924 个法官未承诺的案例对中,当交换响应时,强制读取在 89.7% 的情形下会翻转,而生成后读取为 47.5%(配对差异 +0.422,95% 置信区间 [+0.365, +0.467])。这种扭曲是特定于所测量内容的:它将位置偏差移动了 42 点,而在十种条件中有七种中法官准确率的变化不足一点,因此它误导的是审计法官的人,而不是使用法官的人。即便在法官确实以裁决标记开头时,也可能出现第二种较小的失败,因为它有时会以一个字母开头,之后推理得出另一个字母,在 0 到 5.5% 的案例对中发生,其发生率与合规性无关。我们建议报告法官以裁决标记开头的比率,这只需一次前向传播而无需标签,同时报告任何位置偏差数据。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00054 (HTTP 429)
Authors: Gnaneswar Villuri, Hashmath Shaik, Alex Doboli
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00054.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00054
Published: 2026-10-03T02:19:11.047Z
4. Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluation
Abstract:Researchers increasingly use human-derived bias constructs to study Large Language Models (LLMs), including social-cognitive constructs such as implicit bias and stereotype activation, and cognitive biases such as anchoring, framing effects, and confirmation bias. Such approaches offer alternatives to overt bias probes, particularly when direct questioning may obscure bias or when model behaviour appears normatively acceptable. However, adapting human bias constructs to LLMs introduces an inferential gap. Psychological instruments were developed to study human cognition and social behaviour, whereas LLM evaluations rely on probabilities, text completions, rankings, or simulated decisions. This paper critiques human-centered bias evaluation in LLMs. We show how this gap arises from mismatches pertaining to human-derived constructs, human-model differences, and evaluation contexts, which can blur distinct interpretations of model bias. We then introduce a framework providing an analytical lens for relating these elements to warranted interpretations, with attention to target constructs, operationalizations, scope of inference, and limits of human analogy.
中文摘要
摘要:研究人员越来越多地使用源自人类的偏见构念来研究大型语言模型(LLMs),包括社会认知构念如隐性偏见和刻板印象激活,以及认知偏差如锚定效应、框架效应和确认偏差。这类方法为显性偏见探测提供了替代方案,特别是在直接提问可能掩盖偏见或模型行为在规范上似乎可接受时。然而,将人类偏见构念适配到LLMs中引入了推理上的差距。心理学工具是为了研究人类认知和社会行为而开发的,而LLM评估依赖的是概率、文本补全、排序或模拟决策。本文批判了以人为中心的LLM偏见评估。我们展示了这种差距如何源于人类源构念不匹配、人类与模型的差异以及评估情境,这可能模糊对模型偏见的不同解释。然后,我们引入了一个框架,为将这些元素与合理解释联系起来提供分析视角,重点关注目标构念、操作化、推断范围以及人类类比的局限性。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00070 (HTTP 429)
Authors: Antonela Tommasel, Markus Schedl
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00070.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00070
Published: 2026-10-03T02:19:11.047Z
5. Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights
Abstract:The advancement of natural language processing (NLP) has expanded AI-based text classification in the legal domain. However, accurately classifying legal documents remains challenging due to the complexity of legal texts and subtle differences between legal categories. This study evaluates legal text classification models ranging from traditional machine learning techniques to large language models (LLMs) using ten categories of Korean sexual offense precedents. The results show that fine-tuning small-scale models such as KLUE-BERT on legal data outperforms general-purpose models such as GPT-3.5 and GPT-4.0, as well as traditional machine learning models. KLUE-BERT achieved the highest accuracy of 99.3%, indicating that domain adaptation and fine-tuning can be more important than model size for legal document classification. We further employ explainable AI (XAI) techniques to analyze model predictions and misclassification cases. XAI analysis identifies linguistic features influencing model decisions and limitations in capturing subtle textual cues. Using KICS data, which closely resembles real-world legal case records, we further evaluate the model’s generalization capabilities and find that it struggles to interpret implicit contextual cues. These findings highlight the importance of both performance and interpretability in legal AI and demonstrate how XAI can improve transparency in legal text classification. AI-assisted tools can support legal professionals in tasks including document classification, legal information retrieval, and case assessment.
中文摘要
摘要:自然语言处理(NLP)的进步推动了基于人工智能的法律文本分类的发展。然而,由于法律文本的复杂性以及法律类别之间的细微差异,准确分类法律文件仍然具有挑战性。本研究评估了从传统机器学习技术到大型语言模型(LLMs)的法律文本分类模型,使用了十类韩国性犯罪判例。结果显示,在法律数据上微调小规模模型(如 KLUE-BERT)的效果优于通用模型(如 GPT-3.5 和 GPT-4.0)以及传统机器学习模型。KLUE-BERT 达到了 99.3% 的最高准确率,表明在法律文件分类中,领域适应和微调可能比模型规模更为重要。我们进一步采用可解释人工智能(XAI)技术分析模型的预测和误分类案例。XAI 分析识别出影响模型决策的语言特征以及在捕捉细微文本线索方面的局限性。利用与真实法律案例记录高度相似的 KICS 数据,我们进一步评估了模型的泛化能力,发现模型在解读隐含的上下文线索方面存在困难。这些发现强调了法律人工智能中性能与可解释性的重要性,并展示了 XAI 如何提升法律文本分类的透明度。人工智能辅助工具可以支持法律专业人士完成包括文件分类、法律信息检索和案件评估在内的任务。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00087 (HTTP 429)
Authors: Jeongmin Lee
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00087.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00087
Published: 2026-10-03T02:19:11.047Z
6. BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL
Abstract:Data agents over structured sources must fit database schema into the model’s context window. Large catalogs can span many databases and thousands of columns, so cost constraints may require choosing between table coverage and serialization detail well before the context window is full. We introduce BudgetSchemaBench, an execution-grounded diagnostic for this setting. Its construction derives relevance labels mechanically from gold SQL, without human- or LLM-authored ground truth. Using a pooled 80-database catalog, we sweep four schema-context budgets and compare three representations while keeping each retriever’s table ranking fixed. A source-namespace check rejects queries that obtain the correct result from the wrong database. The evaluation covers three conditions: end-to-end retrieval; frozen-gold, in which the required tables are guaranteed; and a probe that removes those tables. For the primary solver with raw serialization, raising the budget from 2.5% to 50% of the catalog improves execution accuracy on 1,279 held-out questions by 18 percentage points under lexical retrieval but only 3 under dense retrieval; the dense retriever already finds most required tables at the smallest budget. When the required tables are removed, 94.6% of correct predictions name one of them exactly, consistent with reconstruction of absent schema from parametric knowledge. For the two main solvers in the frozen-gold condition, the three representations differ by at most 2 percentage points, and the widest paired 95% confidence interval bounds the difference within +/-4 points. We observe the same qualitative patterns with one reasoning model from a different family. When retrieval is coverage-limited, execution accuracy is more sensitive to the schema budget than to the tested serializations. The diagnostic and the code used to construct and evaluate it are publicly available.
中文摘要
摘要:在结构化数据源上工作的数据代理必须将数据库模式适配到模型的上下文窗口中。大型目录可能跨越多个数据库和数千列,因此成本限制可能要求在上下文窗口还未填满之前,就在表覆盖率和序列化细节之间进行选择。我们提出了 BudgetSchemaBench,这是一个针对这一情境的基于执行的诊断工具。其构建过程从黄金 SQL 机械地导出相关性标签,无需人工或大语言模型撰写的真实标签。使用一个包含 80 个数据库的汇总目录,我们扫描了四种模式上下文预算,并在保持每个检索器表排名固定的情况下比较三种表示。一个源命名空间检查会拒绝从错误数据库中获得正确结果的查询。评估涵盖三种条件:端到端检索;冻结黄金(frozen-gold),其中保证所需表存在;以及一个移除这些表的探测条件。对于使用原始序列化的主要解算器,将预算从目录的 2.5% 提高到 50%,在 1,279 个保留问题上的执行准确率在词汇检索下提升了 18 个百分点,而在密集检索下仅提升 3 个百分点;密集检索器在最小预算下已经找到了大多数所需表。当所需表被移除时,94.6% 的正确预测准确命名其中一个表,这与基于参数化知识重建缺失模式的结果一致。在冻结黄金条件下的两个主要解算器中,三种表示之间的差异最多为 2 个百分点,最宽的成对 95% 置信区间将差异限制在 +/-4 个百分点以内。我们从另一个不同家族的推理模型中观察到相同的定性模式。当检索受到覆盖率限制时,执行准确率对模式预算比对测试的序列化方式更加敏感。该诊断工具及其构建和评估所用的代码已公开提供。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00092 (HTTP 429)
Authors: Chen Shen
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00092.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00092
Published: 2026-10-03T02:19:11.047Z
7. FACET at WMT 2026 Automated Translation Quality Evaluation Task
Abstract:Different error types in machine translation require different evidence. Whether meaning is preserved can be judged only against the source, while whether the target is well-formed, or whether it names one entity consistently, can be judged from the target alone. We present FACET, our reference-free submission to the WMT26 Automated Translation Quality Evaluation Task, which decomposes evaluation into Fluency, Accuracy, and Consistency passes and gives each pass only the context its error type requires. A single fixed model is prompted three times, and the merged error spans yield the three task outputs, error spans, quality scores, and error-free labels, with no trained components. We also submit FACET-C, which omits the Consistency pass. Without gold labels, we characterize the predictions of FACET. Its system rankings place post-edited human translation first, and the Consistency pass changes about a tenth of segment scores while leaving the ranking nearly unchanged.
中文摘要
摘要:机器翻译中的不同错误类型需要不同的证据。是否保留了原意只能通过对照源文本来判断,而目标文本是否结构良好,或者是否持续命名同一实体,则可以仅通过目标文本来判断。我们提出了FACET,这是我们在WMT26自动翻译质量评估任务中的无参考提交方法,它将评估分解为流畅性、准确性和一致性三个阶段,并且在每个阶段只提供其错误类型所需的上下文。一个固定的模型被提示三次,合并的错误范围产生三个任务输出:错误范围、质量评分和无错误标签,而不包含任何训练组件。我们还提交了FACET-C,它省略了一致性阶段。在没有金标准标签的情况下,我们描述了FACET的预测。其系统排名将后编辑的人类翻译排在第一位,一致性阶段更改了大约十分之一的片段评分,同时几乎没有改变排名。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00096 (HTTP 429)
Authors: Ahrii Kim, Chanjun Park, Seong-heum Kim
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00096.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00096
Published: 2026-10-03T02:19:11.047Z
8. Metric-Construction Coupling Inflates Measured Synthetic Dialect Recovery
Abstract:Korean dialect corpora are available but not redistributable: weights may be released, while reproducing training and evaluation from the underlying data cannot be. We ask how much of that supervision synthetic data recovers, and whether that recovery can be measured independently of the synthesis pipeline. We contribute KoDialectBench, 1,000 items across five regions on three axes, released as identifier hashes and scoring code so users reconstruct the items from their own licensed copy. Recovery is strongly axis-dependent: our best synthetic arm reaches 91.2% of the real-data gain on region identification but 63.7% on comprehension. On generation the answer depends on the metric: the deployed marker lexicon reports 92.3% on dialectness and 119.3% on region match, the latter exceeding the real-data reference, whereas reference-based generation reaches 72.4%. We find the marker metrics’ scoring inventory is entirely contained in the inventory our transformation rules can emit. We test the effect of construction access directly with an exact-form construction-disjoint arm that withholds 20% of marker types from the rules. At exactly matched training size (8,600 examples) it reduces dialectness recovery from 91.8% to 8.1% and region-match recovery from 101.9% to 25.6% on the held-out marker inventory, while the three pipeline-independent measurements do not fall at all. A complementary evaluator sweep defines metric-construction coverage (MCC) and finds measured dialectness recovery increasing monotonically as overlap rises from MCC=0 to MCC=1. Shared construction and evaluation inventories can therefore substantially inflate estimates of synthetic-data recovery.
中文摘要
摘要:韩语方言语料库可用,但不可重新分发:模型权重可能会发布,而基于底层数据再现训练和评估则不能。我们探讨了合成数据可以恢复多少监督信息,以及这种恢复是否可以独立于合成流程进行测量。我们贡献了 KoDialectBench,该基准在三个维度上涵盖五个地区的 1,000 个项目,以标识符哈希和评分代码形式发布,使用户可以从其授权副本中重构项目。恢复效果高度依赖维度:我们最好的合成模型在地区识别上达到真实数据增益的 91.2%,而在理解能力上为 63.7%。在生成任务中,结果取决于评价指标:使用部署的标记词典显示方言识别度为 92.3%,地区匹配度为 119.3%,后者甚至超过了真实数据参考,而基于参考的生成仅达到 72.4%。我们发现标记词典的评分清单完全包含在我们的转换规则可生成的清单中。我们通过直接测试构造访问的影响,使用了一个精确形式构造不重叠的模型,该模型在规则中保留了 20% 的标记类型。在训练规模完全匹配(8,600 个样本)时,它使持出标记清单上的方言识别度恢复从 91.8% 降至 8.1%,地区匹配度恢复从 101.9% 降至 25.6%,而三个独立于流程的测量指标则完全没有下降。一个补充的评估器测试定义了指标-构造覆盖率(MCC),并发现随着重叠从 MCC=0 增加到 MCC=1,测得的方言识别度恢复呈单调增加。因此,共享的构造和评估清单可以显著高估合成数据恢复的效果。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00164 (HTTP 429)
Authors: Hyojung Han
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00164.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00164
Published: 2026-10-03T02:19:11.047Z
9. When a Data Artifact Isn’t a Shortcut: Causal Auditing of Synthetic RLVR Corpora
Abstract:Several recent pipelines build RLVR training data by masking a span of real corpus text and asking a language model to invent plausible wrong answers around it. The correct option is therefore genuine human prose; every distractor is synthetic. Correctness and provenance become entangled, and a policy could in principle learn the second instead of the first. We audit that possibility in GooseReason-0.7M. First we ask whether the asymmetry is visible at all: a classifier reading only five surface statistics (never the meaning) reaches AUROC 0.562 over 315,499 options, barely above chance. The aggregate hides something, though. Code sits at 0.416, below chance, and manual inspection explains why: code distractors turn out to be single-operator mutations of the gold answer rather than freely written alternatives, so the two classes are nearly identical by construction. Detecting a signal is not the same as showing a model uses it, so we then run an intervention. We build a paraphrase-matched control corpus, hold training-set size identical across arms, and train two policies under one fixed budget. The exploitation gap does not favour the unmodified-data arm: 0.021 against 0.027 for the control. Under our budget, in other words, a detectable artifact went unexploited. We think that dissociation, along with the domain-specific construction finding, is worth knowing for anyone curating corpora of this kind, and we release the audit as a mostly CPU-only protocol.
中文摘要
摘要:最近的几条流水线通过屏蔽真实语料文本的一段内容,并让语言模型围绕其编造合理的错误答案,从而构建RLVR训练数据。因此,正确选项是真正的人类散文;每个干扰项都是合成的。正确性和来源变得纠缠不清,从理论上讲,策略可能学习第二个而不是第一个。我们在GooseReason-0.7M中审计了这种可能性。首先,我们要问这种不对称是否可见:一个仅读取五个表面统计数据(从未读取含义)的分类器,在315,499个选项上的AUROC为0.562,仅略高于随机水平。然而,整体数据隐藏了一些信息。代码部分为0.416,低于随机水平,手动检查解释了原因:代码干扰项实际上是黄金答案的单运算符变体,而不是自由编写的替代选项,因此根据构造方法,两个类别几乎相同。检测到信号并不等同于显示模型使用它,因此我们接着进行了干预。我们构建了一个与释义匹配的控制语料库,在所有实验组中保持训练集规模相同,并在固定的预算下训练两个策略。利用差距并不偏向未修改数据的实验组:0.021,而控制组为0.027。换句话说,在我们的预算下,可检测的假象未被利用。我们认为,这种解离现象以及特定领域构建发现,对于任何策划此类语料库的人来说都值得了解,我们将此次审计作为一个主要基于CPU的协议公开发布。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00202 (HTTP 429)
Authors: Esther Xin
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00202.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00202
Published: 2026-10-03T02:19:11.047Z
10. A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model
Abstract:As ever larger language models grow more ubiquitous, it is crucial to consider their environmental impact. Characterised by extreme size and resource use, recent generations of models have been criticised for their voracious appetite for compute, and thus significant carbon footprint. Although reporting of carbon impact has grown more common in machine learning papers, this reporting is usually limited to compute resources used strictly for training. In this work, we propose a holistic assessment of the footprint of an extreme-scale language model, Noor. Noor is an ongoing project aiming to develop the largest multi-task Arabic language models — with up to 13B parameters — leveraging zero-shot generalisation to enable a wide range of downstream tasks via natural language instructions. We assess the total carbon bill of the entire project: starting with data collection and storage costs, including research and development budgets, pretraining costs, future serving estimates, and other exogenous costs necessary for this international cooperation. Notably, we find that inference costs and exogenous factors can have a significant impact on total budget. Finally, we discuss pathways to reduce the carbon footprint of extreme-scale models.
中文摘要
摘要:随着越来越大型的语言模型变得普及,考虑其环境影响变得至关重要。由于其极大的规模和资源使用,近期几代模型因其对计算资源的贪婪需求而受到批评,因此产生了显著的碳足迹。尽管在机器学习论文中对碳影响的报告变得越来越常见,这类报告通常仅限于严格用于训练的计算资源。在本研究中,我们提出对极大规模语言模型 Noor 的碳足迹进行整体评估。Noor 是一个正在进行的项目,旨在开发最大的多任务阿拉伯语语言模型——参数规模高达 130 亿——利用零样本泛化,通过自然语言指令支持广泛的下游任务。我们评估整个项目的总碳账单:从数据收集和存储成本开始,包括研发预算、预训练成本、未来服务估算以及其他国际合作所必需的外生成本。值得注意的是,我们发现推理成本和外生因素对总预算可能有显著影响。最后,我们讨论了减少极大规模模型碳足迹的路径。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00223 (HTTP 429)
Authors: Imad Lakim, Ebtesam Almazrouei, Ibrahim Abu Alhaol, Merouane Debbah, Julien Launay
Categories: cs.CL
PDF URL: https://arxiv.org/pdf/2610.00223.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00223
Published: 2026-10-03T02:19:11.047Z
Agent Domain Papers
1. Heavy-Tailed Memory Traces in Long-Horizon Language Agents
Abstract:Long-horizon language agents increasingly rely on external memory as a frozen world model, yet current memory systems are usually judged only by task success or token cost. We argue that the missing object is the shape of memory use: under finite context and repeated retrieval, agent memory can concentrate on a small core while leaving rare states in a long tail where prediction errors accumulate. We study this effect through a conservative tail audit and find that concentration is reproducible but policy-dependent. Random-walk agents produce log-normal-compatible retrieval artifacts, whereas semantic LLM policies yield the strongest truncated-power-law-compatible core—tail traces. Motivated by this audit, we propose Core—Tail World Model (CTWM), a rank-based memory controller that allocates prompt budget with a single exponent $\tau$ while retaining a summarized tail. On Synthetic Graph World, CTWM preserves full state and transition coverage, reduces prompt tokens by 5.9%, and lowers bottom-half tail prediction error by 13.6% relative to a graph-memory baseline. The same paired comparison gives consistent token savings on ALFWorld and a 24.48% token reduction on LongMemEval with aggregate accuracy parity. These results suggest that heavy-tailed memory traces are not only a diagnostic of finite retrieval, but also a practical control signal for token-efficient agent world models.
中文摘要
摘要:长期语言智能体越来越依赖外部记忆作为冻结的世界模型,但目前的记忆系统通常只通过任务成功率或令牌成本来评判。我们认为缺失的要素是记忆使用的形状:在有限上下文和重复检索下,智能体的记忆可能集中于一个小的核心,同时将罕见状态留在长尾中,在长尾中预测误差会累积。我们通过保守的尾部审计研究这一效应,并发现集中性是可重复的,但依赖于策略。随机游走智能体会产生与对数正态兼容的检索特征,而语义LLM策略则产生最强的与截断幂律兼容的核心-尾痕迹。受此审计启发,我们提出核心-尾世界模型(CTWM),一种基于排序的记忆控制器,它通过单个指数$ au$分配提示预算,同时保留已总结的尾部。在合成图世界中,CTWM保持了完整的状态和转移覆盖,相比图记忆基线,提示令牌减少了5.9%,底部半尾的预测误差降低了13.6%。同样的配对比较在ALFWorld上带来了一致的令牌节省,并在LongMemEval上实现了24.48%的令牌减少,同时保持整体准确率相当。这些结果表明,重尾记忆痕迹不仅是有限检索的诊断工具,也是令牌高效智能体世界模型的实用控制信号。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00010 (HTTP 429)
Authors: Xinyuan Song, Zekun Cai
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00010.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00010
Published: 2026-10-03T02:19:32.106Z
2. When Do Causal World Models Help Modular LLM Agents
Abstract:LLM agents increasingly act through modular systems, such as order, payment, inventory, and shipment services, where actions in one module change which transitions are valid in another. Standard world models usually fit observational traces, but this is not the quantity needed for intervention-time planning: a trace may show that payment precedes shipment without identifying whether payment authorizes shipment, inventory mediates the effect, or a hidden trigger explains both. We study this gap through FedCausalCompose, a causal world-model framework for modular LLM agents in which local actions provide intervention-response evidence for cross-module interfaces. We first show that observational world models incur an irreducible interventional error under unblocked back-door paths, that interface recovery improves with intervention-response coverage, and that an oracle causal composition can beat the non-causal lower bound when coverage and local mechanism errors are controlled. We then test the resulting prediction in diagnostic agent settings. Causal interfaces help most in structured tool environments, where API signatures expose preconditions and downstream effects. In contrast, dialogue and narrative environments often ignore raw edge lists unless a short attention anchor makes the causal information decision-relevant. These results identify a concrete condition for causal world models in LLM agents: causal structure helps when cross-module interfaces are both statistically identifiable and presented in a form the agent can use at action time.
中文摘要
摘要:大型语言模型(LLM)代理越来越多地通过模块化系统执行操作,例如订单、支付、库存和发货服务,其中一个模块中的操作会改变另一个模块中哪些转换是有效的。标准的世界模型通常拟合观测轨迹,但这并不是干预时规划所需的量:轨迹可能显示支付发生在发货之前,但无法确定支付是否授权发货,库存是否在其中起中介作用,或隐藏触发因素是否解释了两者。我们通过 FedCausalCompose 探讨这一差距,它是一个针对模块化 LLM 代理的因果世界模型框架,其中局部操作为跨模块接口提供干预-响应证据。我们首先表明,在未阻塞的后门路径下,观测世界模型会产生不可约的干预误差,接口恢复会随着干预-响应覆盖的增加而改善,并且当覆盖和局部机制误差得到控制时,理想的因果组合能够超过非因果下界。随后,我们在诊断代理环境中测试了这一预测。因果接口在结构化工具环境中最有帮助,这些环境中 API 签名揭示了前置条件和下游效应。相比之下,对话和叙事环境通常会忽略原始边列表,除非短注意锚使因果信息与决策相关。这些结果为 LLM 代理中的因果世界模型确定了一个具体条件:当跨模块接口既可以在统计上识别,又以代理在执行操作时可利用的形式呈现时,因果结构会起到帮助作用。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00012 (HTTP 429)
Authors: Xinyuan Song, Zekun Cai
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00012.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00012
Published: 2026-10-03T02:19:32.106Z
3. From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution
Abstract:Large-language-model agents can propose and execute actions, but proposal, authority, dispatch, verified external effect, and serving promotion are different claims. We present Praxa, an agent harness that represents these states explicitly through deterministic admission, brokered execution, external read-back, reconciliation, and reviewed promotion. We report four evidence lanes. First, an author-run repository-local audit at a pinned revision passed 1,027/1,027 unit tests and 89/89 Workerd tests, instrumented all 363 expected source files, and met four coverage floors; raw per-test transcripts and independent reproduction are unavailable. Second, in a provider-backed Terminal-Bench Core 0.1.1 pilot across 12 curated tasks, baseline and reliability-layer arms each passed 17/36 strict trials. The reliability layer used 37.49% more input and 50.73% more output tokens, so the pilot does not support superiority. Third, in a post-debug, two-order coordination-proxy development comparison, baseline and a source-authored candidate each completed 180/180 trials with equal measured accuracy, full hermetic crash recovery, and zero protected violations. The candidate used 37.11% fewer tokens, 33.84% lower estimated endpoint cost, and 11.63% fewer steps; this does not establish improved quality, latency, or production behavior. Fourth, deployed source/configuration evidence shows bounded reflection, recall accounting, memory compilation, and tool-health paths, but no production outcome lift. Praxa’s supported contribution is an evidence-bound architecture that makes authority-to-effect transitions explicit and testable. Current evidence does not establish adversarial security, production safety, general specialist superiority, autonomous recursive optimization, or user benefit.
中文摘要
摘要:大型语言模型代理可以提出和执行操作,但提出、权限、调度、经验证的外部效果和服务提拔是不同的主张。我们提出了 Praxa,这是一种代理框架,通过确定性准入、中介执行、外部回读、对账和审查提拔明确地表示这些状态。我们报告了四条证据线。首先,在作者运行的、固定修订版本的本地仓库审计中,1,027/1,027 单元测试和 89/89 Workerd 测试通过,对所有 363 个预期源文件进行了监控,并满足了四个覆盖率下限;原始每个测试的成绩单和独立复现不可用。其次,在提供方支持的 Terminal-Bench Core 0.1.1 试点中,涵盖 12 个精选任务,基线和可靠性层各自通过了 17/36 严格试验。可靠性层使用了 37.49% 更多输入和 50.73% 更多输出标记,因此试点结果不支持其优越性。第三,在调试后两阶协调代理开发比较中,基线和源作者候选各自完成了 180/180 次试验,具有相等的测量准确性、完整的密封崩溃恢复和零保护违规。候选使用了 37.11% 更少的标记,估计端点成本降低 33.84%,步骤数减少 11.63%;这并不说明质量、延迟或生产行为有所改善。第四,部署的源/配置证据显示了有界反思、回忆核算、记忆编译和工具健康路径,但没有生产结果提升。Praxa 支持的贡献是一个证据约束的架构,使权限到效果的转换明确且可测试。现有证据未能证明对抗性安全性、生产安全性、一般专家优越性、自主递归优化或用户收益。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00015 (HTTP 429)
Authors: Stefan G. Creadore
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00015.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00015
Published: 2026-10-03T02:19:32.106Z
4. What Do Rationales Communicate? A Message-Intervention Study in Role-Specialized QA
Abstract:Role-specialized QA pipelines increasingly pass rationales from a reasoner to a verifier, but it is unclear what this message actually buys: better answers, stronger support assessment, or a new failure surface. We introduce a message-intervention diagnostic that fixes the evidence and candidate answer while varying only the rationale passed across the reasoner-to-verifier boundary. On 400 MuSiQue, HotpotQA, and 2WikiMultiHopQA examples with DeepSeek as generator and verifier, faithful rationales add almost no answer accuracy over no rationale, while corrupted rationales strongly alter support judgments. Under a blind verifier prompt, harmless paraphrases shift support by only 0—2.5%, whereas corrupted rationales shift support by 10—22%; an explicit rationale-checking prompt amplifies the same pattern to 34—55%. Final answers move less (2—30%), and only 2.9—35.3% of corrupted support flips co-occur with answer changes. Human audits show why this matters: 16/42 valid corruptions are corruption-overtrust cases, and blind humans reject or mark unclear 9/10 audited corrupted rationales that the model accepts. Cross-model and task-boundary checks show when the channel is active, amplified, inert, or folded into the task label. Rationale sharing should be evaluated as a verification-message mechanism, not merely as a route to higher answer accuracy.
中文摘要
摘要:专门角色的问答(QA)流程越来越多地将推理者的理由传递给验证者,但目前尚不清楚这一信息到底带来了什么:更好的答案、更强的支持评估,还是新的失败面。我们提出了一种信息干预诊断方法,它固定证据和候选答案,仅改变在推理者与验证者之间传递的理由。在使用 DeepSeek 作为生成器和验证者的 400 个 MuSiQue、HotpotQA 和 2WikiMultiHopQA 示例中,可信理由几乎没有比没有理由带来额外的答案准确率,而被破坏的理由则会强烈改变支持判断。在盲验证者提示下,无害的改写仅会使支持变化 0—2.5%,而被破坏的理由会使支持变化 10—22%;使用明确的理由检查提示会将相同模式放大到 34—55%。最终答案的变化较小(2—30%),且只有 2.9—35.3% 的被破坏支持翻转会伴随答案变化。人工审查显示了其重要性:42 个有效破坏中有 16 个属于过度信任破坏案例,盲目的人类会拒绝或标记不清的 9/10 被模型接受的被破坏理由。跨模型和任务边界的检查表明该通道在何时处于活跃、增强、无效或被融入任务标签。理由共享应被评估为一种验证信息机制,而不仅仅是提高答案准确率的途径。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00018 (HTTP 429)
Authors: Jiameng Zhang, Hongqiu Wu
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00018.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00018
Published: 2026-10-03T02:19:32.106Z
5. Measuring the Microtask Eligibility Gap: When Is an Off-the-Shelf SLM Enough for an Agent Harness?
Abstract:Agent harnesses increasingly want to run small language models (SLMs) on the microtasks around a frontier large language model (LLM) planner: auto-approving shell commands, writing memory, selecting tools, ranking past turns. We ask whether off-the-shelf SLMs meet practitioner-defined thresholds and, when they fail, why, and whether quantization changes the answer. We build a benchmark of 4 such microtasks with fixed prompts and automatic metrics, each with a pre-specified threshold $\tau$ anchored to a cheap non-LLM baseline and a CI-aware eligibility rule (a configuration passes only if its confidence bound clears $\tau$). Sweeping Qwen3 0.6/1.7/4/8B at their best (FP16, greedy, one frozen prompt, no tuning), we find an eligibility gap: 0 of 16 (4 tasks $\times$ 4 models) configurations pass (verified by checking the raw outputs and parser behavior). A logprob decision-threshold diagnostic (T1/T3/T4; T2 via a context-length/cascade probe) separates the failures into capability deficits and failures that can be addressed by changing the decoding threshold (4 regimes). Quantization to 4-bit (RTN/GPTQ/AWQ) does damage that depends on model size and moves no configuration into eligibility (certified on the reconstructable hard-label tasks T1/T3, diagnostic/windowed robustness on T2/T4), so the gap tracks model size more than precision; it replicates on Llama-3.x (12/12 ineligible) and is robust to the anchor choice (a $\tau$-sweep) and to prompt wording (0/112 eligible across the original plus 3 neutral paraphrases per cell). The practical implication: place SLMs behind a baseline that meets the CI-backed threshold, and use the SLM only where the baseline fails to meet the threshold; e.g. a 4B re-ranker over a BM25 shortlist beats BM25 ($+0.047$ [0.020, 0.073], without itself certifying eligibility).
中文摘要
摘要:代理利用越来越多,希望在边界大语言模型(LLM)规划器周围的微任务上运行小型语言模型(SLMs):自动批准 shell 命令、写入内存、选择工具、排序过去的回合。我们探讨现成的 SLM 是否达到从业者定义的门槛,当它们未达到时,原因是什么,以及量化是否会改变结果。我们构建了一个包含 4 个此类微任务的基准,使用固定提示和自动评估指标,每个任务都有一个预先设定的门槛 $ au$,该门槛参考一个廉价的非 LLM 基线,并配有置信区间(CI)感知的资格规则(只有当其置信区间超过 $ au$ 时,配置才算通过)。对 Qwen3 0.6/1.7/4/8B 进行最佳状态(FP16、贪婪、一条固定提示、无调优)扫描时,我们发现资格缺口:16 个配置(4 个任务 × 4 个模型)中没有一个通过(通过检查原始输出和解析器行为验证)。使用 logprob 决策阈值诊断(T1/T3/T4;通过上下文长度/级联探测诊断 T2)将失败分为能力不足和可通过改变解码阈值解决的失败(4 种情况)。量化到 4 位(RTN/GPTQ/AWQ)会造成与模型大小相关的损害,并且没有配置进入资格(在可重建硬标签任务 T1/T3 上进行验证,在 T2/T4 上做诊断/窗口稳定性测试),因此缺口更多地与模型大小相关而非精度;在 Llama-3.x 上复现(12/12 不具备资格),并且对基准选择($ au$ 扫描)和提示措辞(原始提示及每个单元格的 3 个中性释义,共 0/112 合格)都具有稳健性。实践意义:将 SLM 安置在符合 CI 支持门槛的基线之后,仅在基线未达到门槛时使用 SLM;例如,对 BM25 候选集使用 4B 重排序器超过 BM25($+0.047$ [0.020, 0.073],但本身未认证合格)。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00025 (HTTP 429)
Authors: Jundong Hu, Shekar Ramachandran
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00025.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00025
Published: 2026-10-03T02:19:32.106Z
6. Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs
Abstract:Diversity collapse in parallel chain-of-thought has motivated inference-time interventions built on a natural design: when a process reward model (PRM) prunes a chain, its high-PRM prefix is extracted and grafted verbatim as an in-context demonstration into a still-decoding sibling. We isolate this mechanism, PRM-Pruned Fragment Grafting (PPFG), as the most cost-minimal operationalization of cross-trajectory step-level transfer, and test it at the operating point where prior fragment-grafting work reports gains only under additional compensating ingredients. On Qwen2.5-7B-Instruct with Math-Shepherd on full MATH500 (n=500, three seeds), PPFG in both stagnation- and random-targeting variants is statistically indistinguishable from an independent parallel-CoT baseline on every measured axis. We characterize why: a four-bucket classification of 322 stagnation-rule injection events shows only 14% targeted a genuinely struggling chain; the rest landed on chains that had already succeeded, were near completion, or sat on a flat PRM plateau, states a rescue graft cannot change. No compound-gate refinement jointly achieves well-targeted firing and adequate density, and a random control matches the same parity at 2.4x the firing rate, so the inertness is not heuristic-specific. The finding replicates across three base LMs, six benchmarks, a second PRM, and a compatibility-gate sweep; two-one-sided-tests analysis promotes the parity to positive equivalence on all twelve Qwen/LLaMA cells. A per-event spot-check finds injected chains prune at 2.75x the matched-step rate, but a surviving-sibling counterfactual finds no population-level compensation. A hindsight oracle bounds any per-problem gain from choosing PPFG over independent at +0.13 pp. We contribute an equivalence-testing template for establishing inference-time mechanism nulls, with every claim scoped to its tested operating point.
中文摘要
摘要:在并行思维链中多样性崩溃的问题促使研究者开发了基于自然设计的推理时干预方法:当一个过程奖励模型(PRM)修剪一条思维链时,它的高PRM前缀会被提取出来,并原样嫁接为仍在解码的兄弟链中的上下文示例。我们将这一机制隔离出来,称为PRM修剪片段嫁接(PPFG),它是跨轨迹步骤级转移的成本最低的操作化实现,并在以往片段嫁接工作仅在额外补偿因素下报告收益的操作点上进行测试。在使用Qwen2.5-7B-Instruct和Math-Shepherd对完整MATH500(n=500,三次随机种子)进行测试时,PPFG在停滞目标和随机目标的两种变体上,在每个测量维度上与独立并行思维链基线的统计差异不可区分。我们分析了原因:对322次停滞规则注入事件进行四类归类显示,只有14%真正针对了陷入困境的思维链;其余的落在已经成功、接近完成或处于平坦PRM平台的链上,而救援式嫁接无法改变这些状态。没有复合门限的优化能够同时实现良好的目标触发和足够的密度,而随机控制以2.4倍的触发率匹配了相同的效果,因此这种惰性并非启发式特有。该发现可在三种基础语言模型、六个基准、另一PRM以及兼容性门限扫描中复现;两单侧检验分析将平价提升为在所有十二个Qwen/LLaMA单元上的正等效性。每次事件的抽查显示,被注入的思维链的修剪率是匹配步骤的2.75倍,但幸存兄弟链的反事实分析显示未产生群体级补偿。事后预言者将选择PPFG而非独立方法的每题潜在增益限定在+0.13个百分点。我们贡献了一个用于建立推理时机制空效应的等效性测试模板,每一项结论均限定在其测试的操作点上。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00047 (HTTP 429)
Authors: Khawaja Murad ul Hassan, Mehran Ebrahimi
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00047.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00047
Published: 2026-10-03T02:19:32.106Z
7. Gradient-Aligned Pair Selection for Personalized Preference Optimization
Abstract:Personalizing large language models (LLMs) requires aligning generation behavior with user-specific preferences rather than aggregate quality. While Direct Preference Optimization (DPO) provides a stable framework for preference learning, its effectiveness in personalized settings critically depends on how preference pairs are selected. Existing approaches typically rely on heuristic criteria, such as likelihood-based extremes, which decouple optimization from explicit user utility and can lead to degraded personalization. We formalize personalized preference learning as a geometry-aligned optimization problem by analyzing the first-order interaction between gradients of expected user utility and DPO update directions. Our analysis reveals that, under off-policy sampling, the DPO update transitions from a purely error-corrective signal to a reinforcement-like update when preference margins are directionally aligned with utility gradients. This perspective exposes pair selection as a geometric decision that governs whether preference optimization advances or hinders personalization. Motivated by this insight, we propose GAP-DPO (Geometry-Aligned Preference DPO), an iterative algorithm that performs utility-aware, geometry-aligned pair selection while controlling distribution shift via epoch-wise regeneration. Experiments on personalized text generation benchmarks show that GAP-DPO consistently improves stylistic fidelity, preference alignment, and generation quality compared to standard DPO variants. Together, our results establish gradient alignment as a unifying principle for personalized preference optimization and demonstrate that pair selection is an intrinsic component of the optimization geometry rather than a heuristic preprocessing step.
中文摘要
摘要:个性化大型语言模型(LLM)需要将生成行为与用户特定偏好对齐,而不是仅关注总体质量。虽然直接偏好优化(DPO)为偏好学习提供了一个稳定的框架,但其在个性化场景中的有效性关键取决于偏好对的选择方式。现有方法通常依赖启发式标准,例如基于可能性的极值,这会将优化与明确的用户效用脱钩,并可能导致个性化效果下降。我们通过分析期望用户效用梯度与DPO更新方向的一阶交互,将个性化偏好学习形式化为一个几何对齐的优化问题。我们的分析显示,在离策略采样下,当偏好边际方向与效用梯度对齐时,DPO更新会从纯粹的误差校正信号转变为类似强化学习的更新。这一视角揭示了偏好对选择是一个几何决策,它决定了偏好优化是推进还是阻碍个性化。受到这一见解的启发,我们提出了GAP-DPO(几何对齐偏好DPO),这是一种迭代算法,能够在控制分布迁移的同时,通过按周期再生成执行效用感知、几何对齐的偏好对选择。个性化文本生成基准实验表明,与标准DPO变体相比,GAP-DPO在风格一致性、偏好对齐和生成质量方面均稳定提升。综合来看,我们的结果确立了梯度对齐作为个性化偏好优化的统一原则,并证明偏好对选择是优化几何的内在组成部分,而非启发式预处理步骤。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00061 (HTTP 429)
Authors: Ruoming Jin, Xinyu Li, Hao Zhou, Jianfeng Zhu, Ruixin Guo, Feodor Dragan, Lei Xu, Haixun Wang, Yang Zhou
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00061.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00061
Published: 2026-10-03T02:19:32.106Z
8. K-Dense BYOK: An Open-Source AI Research Assistant That Runs Locally and Keeps a Hash-Chained Lab Notebook
Abstract:K-Dense BYOK (bring your own keys) is a free, open-source AI research assistant for scientists in any field that runs on the researcher’s own computer. The researcher supplies access to a model of their choice, hosted or running locally, and the application supplies everything else: a place for the work to run, a layer of scientific scaffolding, and a complete record. Each project is an ordinary folder, so the data, the code, the results, and the record stay on a machine the researcher administers and can be read years later without the application. Three things separate it from a chat assistant or a general-purpose coding agent. It ships a library of written scientific procedures, guided workflow templates, catalogs of where research data can be found, and reviewer and writer roles the agent can hand work to. It keeps a Living Lab Notebook whose entries link into an argument and are added to but never erased. And it records what happened by watching what the agent does rather than by taking the agent’s word for it, in a log the agent has no tool that can write to. That choice targets the most common failure, model overclaiming, in our earlier benchmark of nine frontier models, by making claims checkable rather than preventing them. On twenty interdisciplinary research prompts, scored under a rubric fixed in advance, K-Dense BYOK led two managed platforms on both scientific quality and research execution. Its deliverables were the only ones that recorded the software they ran in, and the only ones that usually arrived with a command that regenerates the results. One of the managed platforms ran the same frontier model and supplied neither. Those environment records were files the agent wrote, not part of the observed log, which does not yet capture the software environment itself. The code is available under the MIT license at this https URL.
中文摘要
摘要:K-Dense BYOK(自带密钥)是一个免费的开源人工智能研究助手,适用于任何领域的科学家,在研究者自己的计算机上运行。研究者提供对他们选择的模型的访问权限,无论是托管的还是本地运行的,而应用程序提供其他所有功能:一个用于运行工作的环境、一层科学支架以及完整的记录。每个项目都是一个普通文件夹,因此数据、代码、结果和记录都保存在研究者管理的机器上,即使几年后也可以在没有应用程序的情况下读取。三个方面将其与聊天助手或通用编码代理区分开来。它提供了一个书面的科学流程库、指导工作流程的模板、研究数据所在位置的目录,以及代理可以交付工作的审稿员和撰写者角色。它维护一个活实验笔记本,其中的条目与论证相连,并且新增内容从不删除。它通过观察代理的行为来记录发生的事情,而不是仅凭代理的陈述记录日志,日志中代理没有任何工具可以写入。这个选择针对我们之前对九个前沿模型的基准测试中最常见的失败——模型过度声明,通过使声明可验证而非阻止这些声明。在二十个跨学科研究提示下,根据事先固定的评分标准,K-Dense BYOK在科学质量和研究执行方面都领先于两个管理平台。其交付物是唯一记录运行软件的,并且通常是唯一附带可重新生成结果命令的。一个管理平台运行了相同的前沿模型,但两者都未提供这些信息。这些环境记录是代理写的文件,而非观察日志的一部分,观察日志尚未捕捉软件环境本身。该代码可在此 HTTPS URL 下获得,并遵循 MIT 许可协议。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00074 (HTTP 429)
Authors: Aubrey M. Brueckner, Darshil Patel, Yuhuan He, Timothy Kassis
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00074.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00074
Published: 2026-10-03T02:19:32.106Z
9. Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks
Abstract:Detailed profession-specific system prompts raise token use and estimated cost per response without a consistent accuracy gain. We evaluate Scientific Agents, an open-source corpus of 503 profession-specific this http URL profiles, with Gemini 3.8 Flash via OpenRouter in the Pi agent harness. We compare matched profiles with four controls: a minimal baseline (“You are a helpful assistant”), the profile’s opening role sentence, a generic scientific rigor guide, and a profile from an unrelated domain. Across nine text-based science benchmarks (4,531 sampled questions, 100 matched profiles), 4,488 items completed all five conditions after API-error retries, scored with automated, rule-based grading. The average profile-baseline accuracy difference is -0.6 percentage points (95% bootstrap interval [-1.5, +0.2] across fixed tasks), and no benchmark shows a statistically clear improvement. Matched profiles produced 1.5-2.3 times as many output tokens and cost 2.2-4.5 times more per successful call. On 60 tool-using BioMysteryBench bioinformatics problems (three runs each for baseline and profile), mean solve rates were 46.7% with the profile and 56.7% at baseline, a difference of -10.0 percentage points (95% interval [-16.7, -3.3]) driven by more frequent token- and time-limit stops under the profile. Longer prompts had one unexpected operational advantage: on SuperGPQA, frequent provider API drops left the short baseline with a correct first-pass answer on only 54.0% of items, against 71.6% with the profile. Generic and mismatched prompts were about as reliable, so this gain comes from prompt length or formatting rather than domain expertise. For the tested model and tasks, loading full profession profiles by default does not improve accuracy and costs considerably more; whether selective retrieval of profile sections or open-ended scientific tasks would change this remains to be tested.
中文摘要
摘要:详细的特定职业系统提示会增加令牌使用量和每次响应的预计成本,但不会带来一致的准确性提升。我们使用 Pi 代理框架,通过 OpenRouter 调用 Gemini 3.8 Flash,评估了 Scientific Agents,这是一个包含 503 个特定职业简介的开源语料库(此 http URL)。我们将匹配的简介与四个对照条件进行比较:最小基线(“你是一个乐于助人的助手”)、简介的开场角色句子、通用科学严谨指导,以及来自不相关领域的简介。在九个基于文本的科学基准测试(4,531 个抽样问题,100 个匹配简介)中,4,488 个条目在 API 错误重试后完成了所有五种条件的测试,并通过自动规则评分。平均简介与基线的准确率差异为 -0.6 个百分点(95% 自助法区间 [-1.5, +0.2],固定任务范围内),且没有任何基准显示出统计上明显的提升。匹配简介生成的输出令牌数量是基线的 1.5-2.3 倍,每次成功调用的成本是基线的 2.2-4.5 倍。在 60 个使用工具的 BioMysteryBench 生物信息学问题上(基线和简介各运行三次),平均解题率为:使用简介为 46.7%,基线为 56.7%,差异为 -10.0 个百分点(95% 区间 [-16.7, -3.3]),主要原因是简介下的令牌和时间限制导致的停止更频繁。较长的提示有一个意外的操作优势:在 SuperGPQA 上,频繁的提供商 API 中断导致短基线提示仅在 54.0% 条目中首次尝试正确回答,而使用简介时为 71.6%。通用和不匹配的提示的可靠性大致相当,因此该提升来自提示长度或格式,而非领域专业知识。对于测试的模型和任务,默认加载完整职业简介并不会提高准确性,却明显增加成本;是否通过选择性检索简介部分或开放式科学任务会改变这一情况,仍有待测试。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00084 (HTTP 429)
Authors: Timothy Kassis
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00084.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00084
Published: 2026-10-03T02:19:32.106Z
10. Comedic Fool’s Gold: Reward Exploits and Countermeasures in Conversational Humor
Abstract:We investigate automated rewards for training language models in conversational humor, focusing on reward exploits and countermeasures. Two approaches aim to capture understandable surprise and predicted audience amusement. Controlled tests show that an embedding-based surprise reward accepts word-shuffled replies as readily as witty ones. A fluency filter detects the shuffles, but the combined reward also rejects some witty replies and fails further validation. An audience model’s predicted laughter is instead vulnerable to laughter cues in either speaker’s messages. Normalizing these cues across speakers blocks the covered attacks, although unmatched expressions remain exploitable. Three reinforcement-learning runs evaluate training with successive reward revisions. The final run improves the combined evaluation score by 0.0903 and reduces zero-score sessions by 40%, but its humor-specific improvement remains below our preregistered target. These findings illustrate a broader challenge for automated reward design: countermeasures must block exploitable shortcuts while preserving the behavior the reward was intended to encourage.
中文摘要
摘要:我们研究了用于训练会话幽默语言模型的自动奖励,重点关注奖励漏洞及其应对措施。两种方法旨在捕捉可理解的惊喜感和预测的观众愉悦感。受控测试显示,基于嵌入的惊喜奖励对打乱单词顺序的回复和机智回复的接受度相当。流畅性过滤器可以检测到这些打乱,但组合奖励同时也拒绝了一些机智回复,并且未能通过进一步验证。观众模型预测的笑声容易受到任意一方发言中的笑声提示影响。跨说话者规范化这些提示可以阻止已知攻击,尽管未匹配表达仍然可能被利用。三个强化学习实验评估了连续奖励修正的训练效果。最终一次实验使组合评估分数提高了0.0903,并将零分会话减少了40%,但其幽默特定的改进仍低于我们预先注册的目标。这些发现说明了自动奖励设计的更广泛挑战:应对措施必须阻止可利用的捷径,同时保留奖励旨在鼓励的行为。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00197 (HTTP 429)
Authors: Sam Larson
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00197.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00197
Published: 2026-10-03T02:19:32.106Z
Evaluation Domain Papers
1. Heavy-Tailed Memory Traces in Long-Horizon Language Agents
Abstract:Long-horizon language agents increasingly rely on external memory as a frozen world model, yet current memory systems are usually judged only by task success or token cost. We argue that the missing object is the shape of memory use: under finite context and repeated retrieval, agent memory can concentrate on a small core while leaving rare states in a long tail where prediction errors accumulate. We study this effect through a conservative tail audit and find that concentration is reproducible but policy-dependent. Random-walk agents produce log-normal-compatible retrieval artifacts, whereas semantic LLM policies yield the strongest truncated-power-law-compatible core—tail traces. Motivated by this audit, we propose Core—Tail World Model (CTWM), a rank-based memory controller that allocates prompt budget with a single exponent $\tau$ while retaining a summarized tail. On Synthetic Graph World, CTWM preserves full state and transition coverage, reduces prompt tokens by 5.9%, and lowers bottom-half tail prediction error by 13.6% relative to a graph-memory baseline. The same paired comparison gives consistent token savings on ALFWorld and a 24.48% token reduction on LongMemEval with aggregate accuracy parity. These results suggest that heavy-tailed memory traces are not only a diagnostic of finite retrieval, but also a practical control signal for token-efficient agent world models.
中文摘要
摘要:长期语言智能体越来越依赖外部记忆作为冻结的世界模型,但目前的记忆系统通常只通过任务成功率或令牌成本来评判。我们认为缺失的要素是记忆使用的形状:在有限上下文和重复检索下,智能体的记忆可能集中于一个小的核心,同时将罕见状态留在长尾中,在长尾中预测误差会累积。我们通过保守的尾部审计研究这一效应,并发现集中性是可重复的,但依赖于策略。随机游走智能体会产生与对数正态兼容的检索特征,而语义大语言模型策略则产生最强的与截断幂律兼容的核心——尾部轨迹。受到该审计的启发,我们提出了核心——尾部世界模型(CTWM),一种基于等级的记忆控制器,它通过单个指数 $ au$ 分配提示预算,同时保留总结的尾部。在合成图世界中,CTWM保持了完整的状态和转移覆盖,相比图记忆基线,提示令牌减少了 5.9%,底半尾部预测误差降低了 13.6%。同样的配对比较在 ALFWorld 上也显示出一致的令牌节省,并在 LongMemEval 上实现了 24.48% 的令牌减少,同时总体准确率保持一致。这些结果表明,重尾记忆轨迹不仅是有限检索的诊断指标,也是实现令牌高效智能体世界模型的实用控制信号。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00010 (HTTP 429)
Authors: Xinyuan Song, Zekun Cai
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00010.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00010
Published: 2026-10-03T02:19:52.438Z
2. When Do Causal World Models Help Modular LLM Agents
Abstract:LLM agents increasingly act through modular systems, such as order, payment, inventory, and shipment services, where actions in one module change which transitions are valid in another. Standard world models usually fit observational traces, but this is not the quantity needed for intervention-time planning: a trace may show that payment precedes shipment without identifying whether payment authorizes shipment, inventory mediates the effect, or a hidden trigger explains both. We study this gap through FedCausalCompose, a causal world-model framework for modular LLM agents in which local actions provide intervention-response evidence for cross-module interfaces. We first show that observational world models incur an irreducible interventional error under unblocked back-door paths, that interface recovery improves with intervention-response coverage, and that an oracle causal composition can beat the non-causal lower bound when coverage and local mechanism errors are controlled. We then test the resulting prediction in diagnostic agent settings. Causal interfaces help most in structured tool environments, where API signatures expose preconditions and downstream effects. In contrast, dialogue and narrative environments often ignore raw edge lists unless a short attention anchor makes the causal information decision-relevant. These results identify a concrete condition for causal world models in LLM agents: causal structure helps when cross-module interfaces are both statistically identifiable and presented in a form the agent can use at action time.
中文摘要
摘要:大型语言模型(LLM)代理越来越多地通过模块化系统执行操作,例如订单、支付、库存和发货服务,其中一个模块中的操作会改变另一个模块中哪些转换是有效的。标准的世界模型通常拟合观测轨迹,但这并不是干预时规划所需的量:轨迹可能显示支付发生在发货之前,但无法确定支付是否授权发货,库存是否中介了该效果,或是否存在隐藏触发因素解释两者。我们通过FedCausalCompose研究这一差距,这是一种针对模块化LLM代理的因果世界模型框架,其中局部操作提供跨模块接口的干预-响应证据。我们首先表明,在未阻塞的后门路径下,观测世界模型会产生不可消除的干预误差;随着干预-响应覆盖的增加,接口恢复会得到改善;当覆盖率和局部机制误差得到控制时,理论上的因果组合可以超过非因果下界。然后,我们在诊断代理设置中测试得到的预测。因果接口在结构化工具环境中帮助最大,因为API签名揭示了前置条件和下游影响。相比之下,对话和叙事环境通常会忽略原始边列表,除非有短期注意锚点使得因果信息与决策相关。这些结果为LLM代理中的因果世界模型确定了一个具体条件:当跨模块接口既可以在统计上识别,又以代理在行动时可以使用的形式呈现时,因果结构会产生帮助。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00012 (HTTP 429)
Authors: Xinyuan Song, Zekun Cai
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00012.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00012
Published: 2026-10-03T02:19:52.438Z
3. From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution
Abstract:Large-language-model agents can propose and execute actions, but proposal, authority, dispatch, verified external effect, and serving promotion are different claims. We present Praxa, an agent harness that represents these states explicitly through deterministic admission, brokered execution, external read-back, reconciliation, and reviewed promotion. We report four evidence lanes. First, an author-run repository-local audit at a pinned revision passed 1,027/1,027 unit tests and 89/89 Workerd tests, instrumented all 363 expected source files, and met four coverage floors; raw per-test transcripts and independent reproduction are unavailable. Second, in a provider-backed Terminal-Bench Core 0.1.1 pilot across 12 curated tasks, baseline and reliability-layer arms each passed 17/36 strict trials. The reliability layer used 37.49% more input and 50.73% more output tokens, so the pilot does not support superiority. Third, in a post-debug, two-order coordination-proxy development comparison, baseline and a source-authored candidate each completed 180/180 trials with equal measured accuracy, full hermetic crash recovery, and zero protected violations. The candidate used 37.11% fewer tokens, 33.84% lower estimated endpoint cost, and 11.63% fewer steps; this does not establish improved quality, latency, or production behavior. Fourth, deployed source/configuration evidence shows bounded reflection, recall accounting, memory compilation, and tool-health paths, but no production outcome lift. Praxa’s supported contribution is an evidence-bound architecture that makes authority-to-effect transitions explicit and testable. Current evidence does not establish adversarial security, production safety, general specialist superiority, autonomous recursive optimization, or user benefit.
中文摘要
摘要:大型语言模型代理可以提出和执行操作,但提出、权限、调度、经验证的外部效果和服务提拔是不同的主张。我们提出了 Praxa,这是一种代理框架,通过确定性准入、中介执行、外部回读、对账和审查提拔明确表示这些状态。我们报告了四条证据路径。首先,在作者运行的存储库本地审核中,在固定修订版本下,1,027/1,027 个单元测试和 89/89 个 Workerd 测试通过,所有 363 个预期源文件都进行了检测,并满足四个覆盖率下限;原始每个测试的成绩单和独立复现不可用。其次,在提供方支持的 Terminal-Bench Core 0.1.1 试点中,涵盖 12 个精选任务,基线和可靠性层各自通过 36 次严格试验中的 17 次。可靠性层使用了 37.49% 更多输入和 50.73% 更多输出标记,因此试点结果不支持其优越性。第三,在调试后,两阶协调代理开发比较中,基线和源作者候选各自完成 180/180 次试验,测得精度相同,完全包裹的崩溃恢复且零受保护违规。候选使用的标记减少了 37.11%,估算终端成本降低了 33.84%,步骤减少了 11.63%;但这并不能证明质量、延迟或生产行为的改进。第四,部署的源/配置证据显示了有界反思、回忆核算、记忆编译和工具健康路径,但没有生产结果提升。Praxa 支持的贡献是一个证据约束的架构,使权限到效果的转换明确且可测试。现有证据未能证明对抗性安全性、生产安全性、一般专家优越性、自主递归优化或用户收益。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00015 (HTTP 429)
Authors: Stefan G. Creadore
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00015.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00015
Published: 2026-10-03T02:19:52.438Z
4. What Do Rationales Communicate? A Message-Intervention Study in Role-Specialized QA
Abstract:Role-specialized QA pipelines increasingly pass rationales from a reasoner to a verifier, but it is unclear what this message actually buys: better answers, stronger support assessment, or a new failure surface. We introduce a message-intervention diagnostic that fixes the evidence and candidate answer while varying only the rationale passed across the reasoner-to-verifier boundary. On 400 MuSiQue, HotpotQA, and 2WikiMultiHopQA examples with DeepSeek as generator and verifier, faithful rationales add almost no answer accuracy over no rationale, while corrupted rationales strongly alter support judgments. Under a blind verifier prompt, harmless paraphrases shift support by only 0—2.5%, whereas corrupted rationales shift support by 10—22%; an explicit rationale-checking prompt amplifies the same pattern to 34—55%. Final answers move less (2—30%), and only 2.9—35.3% of corrupted support flips co-occur with answer changes. Human audits show why this matters: 16/42 valid corruptions are corruption-overtrust cases, and blind humans reject or mark unclear 9/10 audited corrupted rationales that the model accepts. Cross-model and task-boundary checks show when the channel is active, amplified, inert, or folded into the task label. Rationale sharing should be evaluated as a verification-message mechanism, not merely as a route to higher answer accuracy.
中文摘要
摘要:专门角色的问答(QA)流程越来越多地将推理者的理由传递给验证者,但目前尚不清楚这种信息到底带来了什么:更好的答案、更强的支持评估,还是新的失败面。我们提出了一种信息干预诊断方法,它固定证据和候选答案,仅改变在推理者与验证者之间传递的理由。在使用 DeepSeek 作为生成器和验证者的 400 个 MuSiQue、HotpotQA 和 2WikiMultiHopQA 示例中,可信理由几乎没有比没有理由带来额外的答案准确率,而被破坏的理由则会强烈改变支持判断。在盲验证者提示下,无害的改写仅会使支持变化 0—2.5%,而被破坏的理由会使支持变化 10—22%;使用明确的理由检查提示会将相同模式放大到 34—55%。最终答案的变化较小(2—30%),且只有 2.9—35.3% 的被破坏支持翻转会伴随答案变化。人工审查显示了其重要性:42 个有效破坏中有 16 个属于过度信任破坏案例,盲目人类会拒绝或标记不清的 9/10 被模型接受的被破坏理由。跨模型和任务边界的检查表明该通道在何时处于活跃、增强、无效或被融入任务标签。理由共享应作为验证信息机制来评估,而不仅仅作为提高答案准确率的途径。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00018 (HTTP 429)
Authors: Jiameng Zhang, Hongqiu Wu
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00018.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00018
Published: 2026-10-03T02:19:52.438Z
5. Measuring the Microtask Eligibility Gap: When Is an Off-the-Shelf SLM Enough for an Agent Harness?
Abstract:Agent harnesses increasingly want to run small language models (SLMs) on the microtasks around a frontier large language model (LLM) planner: auto-approving shell commands, writing memory, selecting tools, ranking past turns. We ask whether off-the-shelf SLMs meet practitioner-defined thresholds and, when they fail, why, and whether quantization changes the answer. We build a benchmark of 4 such microtasks with fixed prompts and automatic metrics, each with a pre-specified threshold $\tau$ anchored to a cheap non-LLM baseline and a CI-aware eligibility rule (a configuration passes only if its confidence bound clears $\tau$). Sweeping Qwen3 0.6/1.7/4/8B at their best (FP16, greedy, one frozen prompt, no tuning), we find an eligibility gap: 0 of 16 (4 tasks $\times$ 4 models) configurations pass (verified by checking the raw outputs and parser behavior). A logprob decision-threshold diagnostic (T1/T3/T4; T2 via a context-length/cascade probe) separates the failures into capability deficits and failures that can be addressed by changing the decoding threshold (4 regimes). Quantization to 4-bit (RTN/GPTQ/AWQ) does damage that depends on model size and moves no configuration into eligibility (certified on the reconstructable hard-label tasks T1/T3, diagnostic/windowed robustness on T2/T4), so the gap tracks model size more than precision; it replicates on Llama-3.x (12/12 ineligible) and is robust to the anchor choice (a $\tau$-sweep) and to prompt wording (0/112 eligible across the original plus 3 neutral paraphrases per cell). The practical implication: place SLMs behind a baseline that meets the CI-backed threshold, and use the SLM only where the baseline fails to meet the threshold; e.g. a 4B re-ranker over a BM25 shortlist beats BM25 ($+0.047$ [0.020, 0.073], without itself certifying eligibility).
中文摘要
摘要:代理利用越来越多,希望在边界大语言模型(LLM)规划器周围的微任务上运行小型语言模型(SLMs):自动批准 shell 命令、写入记忆、选择工具、排序过去的回合。我们探讨现成的 SLM 是否达到从业者定义的门槛,当它们失败时原因何在,以及量化是否会改变结果。我们建立了一个包含 4 个此类微任务的基准,使用固定提示和自动指标,每个任务都有预先设定的门槛 $ au$,以廉价非 LLM 基线为参考,并设有置信区间(CI)感知的合格规则(配置仅在其置信区间超过 $ au$ 时才算通过)。在最佳条件下(FP16、贪心算法、一个固定提示、无调优)对 Qwen3 0.6/1.7/4/8B 进行测试,我们发现存在合格差距:16 个配置(4 个任务 × 4 个模型)中无一通过(通过检查原始输出和解析器行为验证)。一个对数概率决策阈值诊断(T1/T3/T4;T2 通过上下文长度/级联探针)将失败分为能力不足和可以通过调整解码阈值解决的失败(4 种情形)。量化至 4 位(RTN/GPTQ/AWQ)造成的损害取决于模型大小,并且没有任何配置进入合格(在可重建硬标签任务 T1/T3 上得到验证,在 T2/T4 上通过诊断/窗口化稳健性验证),因此差距更多地与模型大小相关,而非精度;在 Llama-3.x 上得到重复验证(12/12 不合格),并且对锚点选择($ au$ 扫描)和提示表达方式稳健(不同细胞的原版加 3 个中性改写,总共 0/112 合格)。实际意义:将 SLM 放置在满足 CI 支持门槛的基线之后,仅在基线未达到门槛时使用 SLM;例如,对 BM25 候选列表使用 4B 重排序器优于 BM25(提升 +0.047 [0.020, 0.073],而本身并不保证合格)。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00025 (HTTP 429)
Authors: Jundong Hu, Shekar Ramachandran
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00025.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00025
Published: 2026-10-03T02:19:52.438Z
6. Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs
Abstract:Diversity collapse in parallel chain-of-thought has motivated inference-time interventions built on a natural design: when a process reward model (PRM) prunes a chain, its high-PRM prefix is extracted and grafted verbatim as an in-context demonstration into a still-decoding sibling. We isolate this mechanism, PRM-Pruned Fragment Grafting (PPFG), as the most cost-minimal operationalization of cross-trajectory step-level transfer, and test it at the operating point where prior fragment-grafting work reports gains only under additional compensating ingredients. On Qwen2.5-7B-Instruct with Math-Shepherd on full MATH500 (n=500, three seeds), PPFG in both stagnation- and random-targeting variants is statistically indistinguishable from an independent parallel-CoT baseline on every measured axis. We characterize why: a four-bucket classification of 322 stagnation-rule injection events shows only 14% targeted a genuinely struggling chain; the rest landed on chains that had already succeeded, were near completion, or sat on a flat PRM plateau, states a rescue graft cannot change. No compound-gate refinement jointly achieves well-targeted firing and adequate density, and a random control matches the same parity at 2.4x the firing rate, so the inertness is not heuristic-specific. The finding replicates across three base LMs, six benchmarks, a second PRM, and a compatibility-gate sweep; two-one-sided-tests analysis promotes the parity to positive equivalence on all twelve Qwen/LLaMA cells. A per-event spot-check finds injected chains prune at 2.75x the matched-step rate, but a surviving-sibling counterfactual finds no population-level compensation. A hindsight oracle bounds any per-problem gain from choosing PPFG over independent at +0.13 pp. We contribute an equivalence-testing template for establishing inference-time mechanism nulls, with every claim scoped to its tested operating point.
中文摘要
摘要:在并行思维链中多样性崩溃的问题促使研究者开发了基于自然设计的推理时干预方法:当一个过程奖励模型(PRM)修剪一条思维链时,它的高PRM前缀会被提取出来,并原样嫁接为仍在解码的兄弟链中的上下文示例。我们将这一机制隔离出来,称为PRM修剪片段嫁接(PPFG),它是跨轨迹步骤级迁移的成本最低的操作化实现,并在以往片段嫁接工作仅在额外补偿因素下报告收益的操作点上进行测试。在使用Qwen2.5-7B-Instruct和Math-Shepherd对完整MATH500(n=500,三次随机种子)进行测试时,PPFG在停滞目标和随机目标的两种变体上,在每个测量维度上与独立并行思维链基线的统计差异不可区分。我们分析了原因:对322次停滞规则注入事件进行四类归类显示,只有14%的事件真正针对了困难链,其余落在已经成功的链上、接近完成的链上,或处于平坦PRM高原上的链,这些状态是救援式嫁接无法改变的。没有哪种复合门控改进能够同时实现精准触发和足够的密度,而随机控制在触发率为2.4倍时仍能达到同样的等效性,因此惰性并非特定启发式的特征。该发现已经在三种基础语言模型(LM)、六个基准、第二个PRM以及兼容性门控扫描中得到复现;双单侧测试分析将这一等效性提升为在所有十二个Qwen/LLaMA单元上的正等效性。对单事件的抽查显示,注入的链条修剪率为匹配步骤率的2.75倍,但在存活兄弟链的反事实分析中未发现群体层面的补偿。事后预言机工具将选择PPFG而非独立方法的每个问题潜在增益上限限制为+0.13个百分点。我们贡献了一个用以建立推理时机制零假设的等效性测试模板,并将每一项声明都限定在其测试的操作点上。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00047 (HTTP 429)
Authors: Khawaja Murad ul Hassan, Mehran Ebrahimi
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00047.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00047
Published: 2026-10-03T02:19:52.438Z
7. Gradient-Aligned Pair Selection for Personalized Preference Optimization
Abstract:Personalizing large language models (LLMs) requires aligning generation behavior with user-specific preferences rather than aggregate quality. While Direct Preference Optimization (DPO) provides a stable framework for preference learning, its effectiveness in personalized settings critically depends on how preference pairs are selected. Existing approaches typically rely on heuristic criteria, such as likelihood-based extremes, which decouple optimization from explicit user utility and can lead to degraded personalization. We formalize personalized preference learning as a geometry-aligned optimization problem by analyzing the first-order interaction between gradients of expected user utility and DPO update directions. Our analysis reveals that, under off-policy sampling, the DPO update transitions from a purely error-corrective signal to a reinforcement-like update when preference margins are directionally aligned with utility gradients. This perspective exposes pair selection as a geometric decision that governs whether preference optimization advances or hinders personalization. Motivated by this insight, we propose GAP-DPO (Geometry-Aligned Preference DPO), an iterative algorithm that performs utility-aware, geometry-aligned pair selection while controlling distribution shift via epoch-wise regeneration. Experiments on personalized text generation benchmarks show that GAP-DPO consistently improves stylistic fidelity, preference alignment, and generation quality compared to standard DPO variants. Together, our results establish gradient alignment as a unifying principle for personalized preference optimization and demonstrate that pair selection is an intrinsic component of the optimization geometry rather than a heuristic preprocessing step.
中文摘要
摘要:个性化大型语言模型(LLM)需要将生成行为与用户特定偏好对齐,而不是仅关注总体质量。虽然直接偏好优化(DPO)为偏好学习提供了一个稳定的框架,但其在个性化场景中的有效性关键取决于偏好对的选择方式。现有方法通常依赖启发式标准,例如基于可能性的极值,这会将优化与明确的用户效用脱钩,并可能导致个性化效果下降。我们通过分析期望用户效用梯度与DPO更新方向的一阶交互,将个性化偏好学习形式化为一个几何对齐的优化问题。我们的分析显示,在离策略采样下,当偏好边际方向与效用梯度对齐时,DPO更新会从纯粹的误差校正信号转变为类似强化学习的更新。这一视角揭示了偏好对选择是一个几何决策,它决定了偏好优化是推进还是阻碍个性化。受到这一见解的启发,我们提出了GAP-DPO(几何对齐偏好DPO),这是一种迭代算法,能够在控制分布迁移的同时,通过按周期再生成执行效用感知、几何对齐的偏好对选择。个性化文本生成基准实验表明,与标准DPO变体相比,GAP-DPO在风格一致性、偏好对齐和生成质量方面均稳定提升。总的来说,我们的结果确立了梯度对齐作为个性化偏好优化的统一原则,并展示了偏好对选择是优化几何的固有组成部分,而非一个启发式的预处理步骤。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00061 (HTTP 429)
Authors: Ruoming Jin, Xinyu Li, Hao Zhou, Jianfeng Zhu, Ruixin Guo, Feodor Dragan, Lei Xu, Haixun Wang, Yang Zhou
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00061.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00061
Published: 2026-10-03T02:19:52.438Z
8. K-Dense BYOK: An Open-Source AI Research Assistant That Runs Locally and Keeps a Hash-Chained Lab Notebook
Abstract:K-Dense BYOK (bring your own keys) is a free, open-source AI research assistant for scientists in any field that runs on the researcher’s own computer. The researcher supplies access to a model of their choice, hosted or running locally, and the application supplies everything else: a place for the work to run, a layer of scientific scaffolding, and a complete record. Each project is an ordinary folder, so the data, the code, the results, and the record stay on a machine the researcher administers and can be read years later without the application. Three things separate it from a chat assistant or a general-purpose coding agent. It ships a library of written scientific procedures, guided workflow templates, catalogs of where research data can be found, and reviewer and writer roles the agent can hand work to. It keeps a Living Lab Notebook whose entries link into an argument and are added to but never erased. And it records what happened by watching what the agent does rather than by taking the agent’s word for it, in a log the agent has no tool that can write to. That choice targets the most common failure, model overclaiming, in our earlier benchmark of nine frontier models, by making claims checkable rather than preventing them. On twenty interdisciplinary research prompts, scored under a rubric fixed in advance, K-Dense BYOK led two managed platforms on both scientific quality and research execution. Its deliverables were the only ones that recorded the software they ran in, and the only ones that usually arrived with a command that regenerates the results. One of the managed platforms ran the same frontier model and supplied neither. Those environment records were files the agent wrote, not part of the observed log, which does not yet capture the software environment itself. The code is available under the MIT license at this https URL.
中文摘要
摘要:K-Dense BYOK(自带密钥)是一个免费的开源人工智能研究助手,适用于任何领域的科学家,在研究者自己的计算机上运行。研究者提供对他们选择的模型的访问权限,无论是托管的还是本地运行的,而应用程序提供其他所有功能:一个用于运行工作的环境、一层科学支架以及完整的记录。每个项目都是一个普通文件夹,因此数据、代码、结果和记录都保存在研究者管理的机器上,即使几年后也可以在没有应用程序的情况下读取。三个方面将其与聊天助手或通用编码代理区分开来。它提供了一个书面的科学流程库、指导工作流程的模板、研究数据所在位置的目录,以及代理可以交付工作的审稿员和撰写者角色。它维护一个活跃的实验笔记本(Living Lab Notebook),其中的条目链接到论点,并不断添加而从不删除。它通过观察代理的行为而非仅仅依赖代理自述的方式记录发生的事情,日志是代理无法写入的。这一选择针对我们之前对九个前沿模型的基准测试中最常见的失败——模型夸大——通过使声明可检查而非阻止声明来解决。在二十个跨学科研究提示下,根据预先固定的评分标准,K-Dense BYOK在科学质量和研究执行方面均领先于两个托管平台。它的成果是唯一记录其运行软件的,并且通常附带能够再现结果的命令。一个托管平台同样运行前沿模型,但没有提供这些信息。这些环境记录是代理写入的文件,并不属于观察日志的一部分,观察日志尚未捕捉软件环境本身。代码可在此 HTTPS URL 下以 MIT 许可证获得。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00074 (HTTP 429)
Authors: Aubrey M. Brueckner, Darshil Patel, Yuhuan He, Timothy Kassis
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00074.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00074
Published: 2026-10-03T02:19:52.438Z
9. Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks
Abstract:Detailed profession-specific system prompts raise token use and estimated cost per response without a consistent accuracy gain. We evaluate Scientific Agents, an open-source corpus of 503 profession-specific this http URL profiles, with Gemini 3.8 Flash via OpenRouter in the Pi agent harness. We compare matched profiles with four controls: a minimal baseline (“You are a helpful assistant”), the profile’s opening role sentence, a generic scientific rigor guide, and a profile from an unrelated domain. Across nine text-based science benchmarks (4,531 sampled questions, 100 matched profiles), 4,488 items completed all five conditions after API-error retries, scored with automated, rule-based grading. The average profile-baseline accuracy difference is -0.6 percentage points (95% bootstrap interval [-1.5, +0.2] across fixed tasks), and no benchmark shows a statistically clear improvement. Matched profiles produced 1.5-2.3 times as many output tokens and cost 2.2-4.5 times more per successful call. On 60 tool-using BioMysteryBench bioinformatics problems (three runs each for baseline and profile), mean solve rates were 46.7% with the profile and 56.7% at baseline, a difference of -10.0 percentage points (95% interval [-16.7, -3.3]) driven by more frequent token- and time-limit stops under the profile. Longer prompts had one unexpected operational advantage: on SuperGPQA, frequent provider API drops left the short baseline with a correct first-pass answer on only 54.0% of items, against 71.6% with the profile. Generic and mismatched prompts were about as reliable, so this gain comes from prompt length or formatting rather than domain expertise. For the tested model and tasks, loading full profession profiles by default does not improve accuracy and costs considerably more; whether selective retrieval of profile sections or open-ended scientific tasks would change this remains to be tested.
中文摘要
摘要:详细的职业特定系统提示提高了代币使用率和每次响应的估计成本,但准确率却不稳定提升。我们评估了科学代理人(Scientific Agents),这是一个开源语料库,包含503个职业特定 http URL 配置文件,并结合 Pi 代理框架中的 Gemini 3.8 Flash 通过 OpenRouter。我们将匹配配置文件与四个对照组比较:最小基线(“你是个有用的助手”)、配置文件开头角色句子、通用科学严谨指南和一个无关领域的配置文件。在九个基于文本的科学基准测试(4,531个抽样问题,100个匹配配置文件)中,4,488个项目在API错误重试后完成了全部五个条件,并通过自动规则评分。配置文件与基线的平均准确性差异为-0.6个百分点(固定任务间隔95%自助间隔[-1.5,+0.2]),且无基准显示统计上明显的提升。匹配配置文件产生的输出令牌数量是其1.5-2.3倍,且每次成功调用的成本增加2.2-4.5倍。在60个使用工具的BioMysteryBench生物信息学问题中(基线和剖面各三次运行),剖面中平均解答率为46.7%,基线时为56.7%,差距为-10.0个百分点(95%间隔[-16.7,-3.3]),主要原因是剖面下代币和时间限制停止的频率更高。较长的提示有一个意想不到的操作优势:在SuperGPQA中,频繁的API丢失仅在短基线后,只有54.0%的项目首遍回答正确,而剖面的71.6%。通用且不匹配的提示词同样可靠,因此这种提升来自提示长度或格式,而非领域专业知识。对于测试模型和任务,默认加载完整职业档案并不能提升准确性,且成本显著增加;选择性检索档案部分或开放式科学任务是否会改变这一点,尚待验证。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00084 (HTTP 429)
Authors: Timothy Kassis
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00084.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00084
Published: 2026-10-03T02:19:52.438Z
10. Comedic Fool’s Gold: Reward Exploits and Countermeasures in Conversational Humor
Abstract:We investigate automated rewards for training language models in conversational humor, focusing on reward exploits and countermeasures. Two approaches aim to capture understandable surprise and predicted audience amusement. Controlled tests show that an embedding-based surprise reward accepts word-shuffled replies as readily as witty ones. A fluency filter detects the shuffles, but the combined reward also rejects some witty replies and fails further validation. An audience model’s predicted laughter is instead vulnerable to laughter cues in either speaker’s messages. Normalizing these cues across speakers blocks the covered attacks, although unmatched expressions remain exploitable. Three reinforcement-learning runs evaluate training with successive reward revisions. The final run improves the combined evaluation score by 0.0903 and reduces zero-score sessions by 40%, but its humor-specific improvement remains below our preregistered target. These findings illustrate a broader challenge for automated reward design: countermeasures must block exploitable shortcuts while preserving the behavior the reward was intended to encourage.
中文摘要
摘要:我们研究了用于训练会话幽默语言模型的自动奖励,重点关注奖励漏洞及其应对措施。两种方法旨在捕捉可理解的惊喜感和预测的观众愉悦感。受控测试显示,基于嵌入的惊喜奖励对打乱单词顺序的回复和机智回复的接受度相当。流畅性过滤器可以检测到这些打乱,但组合奖励也会拒绝一些机智回复,并且在进一步验证中失败。观众模型预测的笑声则容易受到任意一方发言中的笑声提示的影响。对这些提示进行跨发言者归一化可以阻止已知的攻击,但未匹配的表达仍可被利用。三次强化学习运行评估了逐步修订奖励的训练效果。最终运行将组合评估分数提高了0.0903,并将零分会话减少了40%,但幽默特定的改进仍低于我们预先注册的目标。这些发现说明了自动奖励设计的更广泛挑战:应对措施必须阻止可利用的捷径,同时保留奖励原本旨在鼓励的行为。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00197 (HTTP 429)
Authors: Sam Larson
Categories: cs.AI
PDF URL: https://arxiv.org/pdf/2610.00197.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00197
Published: 2026-10-03T02:19:52.438Z
VLM Domain Papers
1. STATERA: Hidden Mass Estimation via Zero-Shot Sim-to-Real Kinematics using Frozen Temporal Tubelets
Abstract:Vision models pretrained for frame-level appearance often struggle to infer hidden physical properties from motion. We study center-of-mass (CoM) localization for opaque, asymmetric rigid bodies from short monocular videos, where surface cues and point tracking are unreliable under self-occlusion. We propose STATERA, which adapts a pretrained video backbone (V-JEPA) with mostly frozen weights and a lightweight temporal tubelet mixer to predict per-frame CoM heatmaps and trajectories. To support this task, we introduce the HiddenMass Benchmark, comprising 50K MuJoCo trajectories and a 63-sequence real-world test set with physically calibrated CoM ground truth. In simulation, STATERA-50K-Sigma improves normalized CoM error from 41.7% (DINOv2) to 25.2%. In zero-shot sim-to-real transfer, we observe a fundamental trade-off in supervision: phase-aware targets can induce bimodal predictions, while phase-agnostic targets can collapse toward statistically safe centroids. Nevertheless, our phase-aware STATERA-50K-Crescent is the only evaluated method that demonstrates consistent movement toward the true hidden offset. While this leads to a monocular vector overshoot artifact that marginally increases absolute Euclidean error compared to a static geometric centroid, it improves physics capture from 2.6% to 41.0%. These results suggest that frozen temporal representations can better separate inertial dynamics from visual geometry for hidden-parameter estimation.
中文摘要
摘要:预训练为帧级外观的视觉模型常常难以从运动中推断隐藏的物理属性。我们研究了短单眼视频中不透明、不对称刚体的质心定位,在自闭合下表面线索和点追踪不可靠。我们提出STATERA,该模型采用预训练视频骨干(V-JEPA),权重大多冻结,并配备轻量级时间管混合器,预测每帧CoM热量图和轨迹。为支持该任务,我们引入了HiddenMass基准,包含50K MuJoCo轨迹和63序列的真实世界测试集,物理校准CoM基准。在仿真中,STATERA-50K-Sigma将归一化CoM误差从41.7%(DINOv2)提升至25.2%。在零shot模拟到实的传输中,我们观察到监督上的根本权衡:相位感知目标可以诱导双峰预测,而相位无关目标则可能向统计安全质心坍缩。尽管如此,我们的相位感知STATERA-50K-Crescent是唯一一个经过评估并持续向真实隐藏偏移移动的方法。虽然这导致单眼矢量超冲伪影,使绝对欧几里得误差略有增加,但物理捕获率从2.6%提升到41.0%。这些结果表明,冻结的时间表示可以更好地将惯性动力学与视觉几何分离,用于隐藏参数估计。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00003 (HTTP 429)
Authors: Animesh Varma
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00003.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00003
Published: 2026-10-03T02:20:17.375Z
2. Emergent Object Binding Has a Finite Spatial Horizon
Abstract:Pretrained Vision Transformers encode whether two image patches belong to the same object. This IsSameObject signal is decodable from frozen patch embeddings at high accuracy, which suggests that object binding emerges from self-supervised pretraining alone. We show that this single accuracy number hides the structure of the signal. Binding is local: the probability that two patches of the same object are decoded as bound falls off monotonically with the distance between them and levels off at a nonzero floor, a falloff well described by an exponential with a finite length scale. This decay holds across object sizes, across three families of probe, on both ADE20K and COCO, and across DINO and CLIP backbones, which indicates that it is a property of the representation rather than of the decoder. Reading binding as local spatial coherence with a finite range accounts for a set of behaviors that the aggregate score leaves unexplained: binding weakens on large objects, separates distinct objects of the same class less reliably than objects of different classes, and groups object parts with their wholes. It is, by contrast, unaffected by occlusion once object size is controlled. We map each behavior with confounds controlled. As a preliminary observation, the horizon and its floor are organized at different depths in DINOv2 and DINOv3, which we report as suggestive given the small number of layers probed and the confound between the two models.
中文摘要
摘要:预训练的视觉变换器能够编码两个图像块是否属于同一对象。这个 IsSameObject 信号可以从冻结的图像块嵌入中高精度解码,这表明对象绑定仅通过自监督预训练便会出现。我们展示了单一的精度数值掩盖了信号的结构。绑定是局部的:同一对象的两个图像块被解码为绑定的概率会随着它们之间距离的增加单调下降,并在非零基线处趋于平稳,这种下降可以用具有有限长度尺度的指数函数很好地描述。这种衰减在不同对象大小、三种探测器族、ADE20K 和 COCO 数据集上,以及 DINO 和 CLIP 主干网络中都成立,这表明它是表征的属性,而非解码器的属性。将绑定理解为具有有限范围的局部空间一致性可以解释一组总体分数无法说明的行为:在大对象上绑定会减弱、同类不同对象的分离不如不同类对象可靠,以及将对象部件与整体归组。相比之下,一旦控制了对象大小,遮挡对其没有影响。我们映射了每种行为,同时控制了混淆因素。作为初步观察,地平线及其基线在 DINOv2 和 DINOv3 中位于不同的深度,我们报告这一点具有提示性,考虑到探测的层数较少且两种模型之间存在混淆因素。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00006 (HTTP 429)
Authors: Mayank Singal
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00006.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00006
Published: 2026-10-03T02:20:17.375Z
3. Spatial Lifting for Dense Prediction
Abstract:We present Spatial Lifting (SL), a novel methodology for dense prediction tasks. SL operates by lifting standard inputs, such as 2D images, into a higher-dimensional space and subsequently processing them using networks designed for that higher dimension, such as a 3D U-Net. Counterintuitively, this dimensionality lifting allows us to achieve good performance on benchmark tasks compared to conventional approaches, while reducing inference costs and \textbf{drastically lowering the number of model parameters}. The SL framework produces intrinsically structured outputs along the lifted dimension. This emergent structure facilitates dense supervision during training and enables single-forward-pass self-consistency-based quality and uncertainty estimation at test time. Spatial Lifting introduces a simple and general modeling strategy that offers a promising path toward more efficient, accurate, and reliable deep networks for dense prediction tasks in vision.
中文摘要
摘要:我们提出了空间提升(Spatial Lifting,SL),这是一种用于密集预测任务的新方法。SL 的操作方式是将标准输入(例如二维图像)提升到更高维度空间,然后使用为该高维设计的网络(例如三维 U-Net)进行处理。违反直觉的是,这种维度提升使我们在基准任务中与传统方法相比实现良好的性能,同时降低推理成本并大幅减少模型参数数量。SL 框架沿提升维度生成本质上结构化的输出。这种新兴结构有助于在训练期间进行密集监督,并在测试时实现基于单次前向传播的自我一致性质量和不确定性估计。空间提升引入了一种简单且通用的建模策略,为视觉领域密集预测任务中的深度网络实现更高效、更准确、更可靠的途径提供了有前景的方向。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00017 (HTTP 429)
Authors: Mingzhi Xu, Tao Zhou, Yong Li, Yizhe Zhang
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00017.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00017
Published: 2026-10-03T02:20:17.375Z
4. Encoded but Disconnected: Decomposing Vision-Language Model Failures under a Patching Null
Abstract:Across three vision-language model architectures (LLaVA-1.5-7B, Qwen2.5-VL-7B, InternVL3-8B), we report a universal negative finding for mid-layer interpretability. On POPE — the benchmark common to all three — the mid layers encode the ground-truth answer in 68-91% of errors, yet this signal is not causally active for the final prediction: residual-stream patching yields 0% non-trivial flip at the layer level on all three architectures, and on two of three at the per-head level (Qwen: 0/12,600 patched forwards). The lone exception, InternVL3 layer-20 head-2, is a non-vocab, self-attending head whose effect is localized to that specific head (p < 1e-4). Despite the null, the errors separate operationally into three failure modes — Perception Failure, Encoded-but-Disconnected, Prior-Override — learnable above 60% on all three architectures, and the architecture’s prior direction predicts which of two interventions elicits a category-specific response. We report these mitigation effects under oracle labels as evidence the categories are mechanistically real, not as a deployable method.
中文摘要
摘要:在三种视觉-语言模型架构(LLaVA-1.5-7B、Qwen2.5-VL-7B、InternVL3-8B)中,我们报告了关于中间层可解释性的一个普遍负面发现。在三者共有的基准测试 POPE 上,中间层在 68-91% 的错误中编码了真实答案,但这一信号对于最终预测并非因果活跃:在三种架构中,残差流补丁操作在层级上导致 0% 的非平凡翻转,在三个架构中的两个在每个注意力头层面也是如此(Qwen:0/12,600 前向补丁操作)。唯一的例外是 InternVL3 的第 20 层第 2 头,这是一个非词汇、自我关注的注意力头,其效果局限于该特定头(p < 1e-4)。尽管结果为空,但错误操作上可以分为三种失败模式——感知失败、已编码但未连接、先验覆盖——在所有三种架构上均可学习到 60% 以上,并且架构的先验方向可以预测哪两种干预会引发特定类别的响应。我们在使用真实标签的情况下报告这些缓解效果,以证明这些类别在机制上是真实存在的,而非可部署的方法。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00024 (HTTP 429)
Authors: Genpei Zhang
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00024.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00024
Published: 2026-10-03T02:20:17.375Z
5. Domain generalization and synthetic data in object detection: the enabler, the probe, and the gap
Abstract:Object detection models often experience performance degradation when deployed under distribution shifts, caused by for example changes in weather type, operational environment, or object appearance. Domain Generalization (DG) aims to develop models that remain robust under such shifts and generalize well to unseen domains. DG research specifically focused on object detection models is scarce, although these models face additional challenges around localization and multi-scale representations. Synthetic data is a promising tool to support in DG, by enabling large-scale generation of diverse new samples. In this paper, we present an object detection-centric review of DG and examine the role of synthetic data from three complementary perspectives. First, synthetic data acts as an enabler of DG through diversification and alignment strategies that aim to improve robustness to distribution shifts. Second, it serves as a probe that enables controlled experimentation to identify and understand failure modes. Third, we discuss the synthetic-to-real gap, a particularly challenging form of domain shift that arises when models trained on synthetic imagery are deployed on real-world data. Through reviewing these perspectives, we identify limitations of current DG approaches for object detection and argue that future research requires representation-aware methods that explicitly address both localization and classification under domain shift.
中文摘要
摘要:当物体检测模型在分布发生变化的环境下部署时,例如天气类型、操作环境或物体外观的变化,模型的性能常常会下降。领域泛化(Domain Generalization, DG)旨在开发在此类变化下仍然稳健且能很好地推广到未见领域的模型。专门针对物体检测模型的 DG 研究相对稀缺,尽管这些模型在定位和多尺度表示方面面临额外的挑战。合成数据是支持 DG 的有前景工具,它使得可以大规模生成多样化的新样本。本文中,我们提出了一个以物体检测为中心的 DG 综述,并从三个互补的角度考察合成数据的作用。首先,合成数据通过旨在提高对分布变化稳健性的多样化和对齐策略,作为 DG 的推动力。其次,它作为一种探针,使得可控实验成为可能,以识别和理解失败模式。第三,我们讨论了从合成到真实的差距,这是当在合成图像上训练的模型部署到真实世界数据时出现的一种特别具有挑战性的领域变化。通过回顾这些观点,我们识别了当前物体检测 DG 方法的局限性,并认为未来的研究需要关注表示的的方法,这些方法应明确解决在领域变化下的定位和分类问题。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00030 (HTTP 429)
Authors: Elfi I.S. Hofmeijer, Ella P. Fokkinga, Friso G. Heslinga, Klamer Schutte, Jörgen M. Karlholm
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00030.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00030
Published: 2026-10-03T02:20:17.375Z
6. Seeing the City or Recognizing the Place? What Street-View Imagery Adds Beyond Existing Urban Data in VLM Urban Sensing
Abstract:Street-view imagery is increasingly used to infer urban attributes, but predictive accuracy alone does not reveal how much a photograph contributes beyond data already available for the same place. We compare image-based predictions with existing urban data across seven attributes from five public resources and three VLMs. The same urban units are evaluated using images, task context, nearby observations, and public records, while image replacements and conflicting records test source reliance. Existing urban data matched or exceeded image-only models for road damage, curb ramps, and house price, while neighbouring official statistics nearly matched the best image result for population. Images were more informative for building type, building function, and low-rise floor count. For floor count, image advantage increased by 5.7 percentage points per doubling of distance to the nearest labelled building and declined for tall buildings whose rooflines often fell outside the frame. Models frequently followed conflicting records. OpenFACADES floor annotations were generated with OpenStreetMap floor values and showed the opposite height-dependent error pattern from image-only reruns. Street-view image value therefore depends on visual legibility and local data coverage. Comparing images with existing urban data can guide image collection and clarify the provenance of derived urban maps.
中文摘要
摘要:街景图像日益被用于推断城市属性,但仅靠预测准确性无法揭示一张照片相比已有同地数据的额外贡献。我们将基于图像的预测与来自五个公共资源和三个VLM的七个城市属性的现有城市数据进行比较。同样的城市单元使用图像、任务背景、邻近观测和公共记录进行评估,同时通过图像替代和冲突记录测试数据来源依赖性。现有城市数据在道路损坏、人行道坡道和房价方面与仅图像模型相当或更优,而邻近官方统计数据在人口方面几乎达到最佳图像结果。图像在建筑类型、建筑功能和低层楼层数量方面信息量更大。对于楼层数量,随着到最近标注建筑的距离加倍,图像优势增加5.7个百分点,而对于屋顶线常常位于视野之外的高楼,则优势下降。模型经常遵循冲突记录。OpenFACADES楼层标注通过OpenStreetMap楼层值生成,并呈现与仅图像重跑相反的高度相关误差模式。因此,街景图像的价值取决于视觉可读性和本地数据覆盖。将图像与现有城市数据进行比较可以指导图像收集并明确衍生城市地图的来源。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00031 (HTTP 429)
Authors: Kaizhen Tan
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00031.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00031
Published: 2026-10-03T02:20:17.375Z
7. DSSR-3D: Decoupled Reasoning for View-Dependent Referring in 3D Gaussians
Abstract:Recent advances in 3D Gaussian Splatting have enabled open-vocabulary and referring segmentation by distilling semantic knowledge from 2D foundation models into 3D representations. However, existing referring fields embed language features in a globally view-invariant space, making them fundamentally unable to resolve observer-centric spatial relations (e.g., “to the left of”) that depend on camera pose. We propose DSSR-3D, an inference-time framework for view-dependent referring segmentation on continuous 3D Gaussian fields, formalized as two interfaces - pose-invariant semantic localization and pose-conditioned spatial reasoning - such that any pair of functions satisfying these constraints yields a valid instantiation, requiring no retraining of the underlying semantic field and no reliance on discrete geometric proxies such as bounding boxes. We instantiate the two interfaces with a temperature-sharpened softmax localization mechanism and a projection-based directional scoring function, fused via a lightweight, training-free step, and show they transfer zero-shot to structurally distinct semantic fields without adaptation. We further propose ViewRef-GS, a benchmark isolating view-dependent segmentation on 3D Gaussian fields, evaluated jointly with an augmented Ref-LERF to provide a comprehensive testbed for viewpoint-dependent spatial grounding. Experiments show consistent gains over existing 3DGS-based referring methods, with no additional training beyond the base semantic field
中文摘要
摘要:近年来,3D高斯点云(3D Gaussian Splatting)的进展使得通过将2D基础模型中的语义知识蒸馏到3D表示中,实现开放词汇和指称分割成为可能。然而,现有的指称场在一个全局视角不变的空间中嵌入语言特征,使其在根本上无法解决依赖摄像机姿态的以观察者为中心的空间关系(例如,“在左侧”)。我们提出了DSSR-3D,这是一种用于连续3D高斯场的视角依赖指称分割的推理时框架,被形式化为两个接口——姿态不变的语义定位和姿态条件的空间推理——这样任何满足这些约束的函数对都能产生有效实例,无需对底层语义场进行重新训练,也无需依赖诸如边界框等离散几何代理。我们通过温度锐化的softmax定位机制和基于投影的方向评分函数实例化了这两个接口,并通过轻量、无需训练的步骤融合,并展示了它们在结构上不同的语义场中无需适应即可进行零-shot迁移。我们进一步提出了ViewRef-GS,这是一个专门用于3D高斯场的视角依赖分割的基准,并与增强版Ref-LERF联合评估,为视角依赖的空间定位提供了全面的测试平台。实验结果显示,与现有基于3DGS的指称方法相比,本方法在无需额外训练基础语义场的情况下,持续取得了性能提升。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00040 (HTTP 429)
Authors: Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet Tran
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00040.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00040
Published: 2026-10-03T02:20:17.375Z
8. Reachability Is Not Generalization: Understanding Verb—Noun Decomposition in Assembly Action Recognition
Abstract:Assembly actions are compositional: they combine a manipulation with a part or tool. In deployment, systems routinely encounter novel combinations of familiar components, yet an atomic action classifier assigns every unseen combination exactly zero probability by construction. The prevailing solution is verb—noun decomposition, which predicts components separately and recombines them to reach unseen actions. While widely adopted, how decomposition generalizes under compositional shift remains poorly understood. We present a systematic analysis of verb—noun decomposition across three assembly datasets (MECCANO, HAViD, and IMPACT). Although decomposition escapes the atomic ceiling, its generalization extends only partially beyond it. Unseen-composition performance remains strongly tied to the co-occurrence structure of the training data, indicating that much of the observed gain arises from interpolation within densely supported regions of the compositional space rather than from unconstrained recombination. Across datasets, failures consistently concentrate on the larger-vocabulary component, and IMPACT’s verb-heavy vocabulary reverses the bottleneck from nouns to verbs. We further show that shared-encoder training introduces component entanglement, encouraging reliance on co-occurrence patterns that transfer poorly to unseen compositions and trailing independent recombination by up to $6.0\times$ in harmonic mean. Taken together, these findings explain why decomposition achieves only partial compositional generalization in practice. By identifying primitive support, vocabulary asymmetry, and component entanglement as connected sources of error, we provide a portable diagnostic framework for studying compositional recognition beyond aggregate accuracy. Code: this https URL.
中文摘要
摘要:装配动作具有组合性:它们将操作与零件或工具结合。在实际应用中,系统经常遇到熟悉组件的全新组合,但原子动作分类器按照设计会将每个未见组合的概率分配为零。现行的解决方案是动词—名词分解,它分别预测组件并重新组合以实现对未见动作的预测。尽管该方法被广泛采用,但在组合变化情况下分解策略如何泛化仍然知之甚少。我们对三个装配数据集(MECCANO、HAViD 和 IMPACT)中的动词—名词分解进行了系统分析。虽然分解方法突破了原子上限,但其泛化能力仅超出原子上限的一部分。未见组合的性能仍然与训练数据的共现结构密切相关,这表明观察到的大部分提升源自组合空间中密集支撑区域的插值,而非无约束的重新组合。在各数据集中,失败案例始终集中在规模较大的组件上,而 IMPACT 中动词丰富的词汇使得瓶颈从名词转向动词。我们进一步展示了,共享编码器训练会引入组件纠缠,促使模型依赖于难以迁移到未见组合的共现模式,其调和平均表现最多落后独立重组 $6.0 imes$。综合来看,这些发现解释了为什么在实践中分解方法仅能实现部分组合泛化。通过将原始支持、词汇不对称和组件纠缠确定为相关的误差来源,我们提供了一个可移植的诊断框架,用于研究组合识别,而不仅仅是总体准确率。代码:此 https URL。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00064 (HTTP 429)
Authors: Changyi Li, Yu Xiao
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00064.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00064
Published: 2026-10-03T02:20:17.375Z
9. Robust Online Aero-Engine Blade Defect Detection via Dual-Alignment Test-Time Adaptation
Abstract:Reliable visual inspection is essential for quality assurance in aero-engine blade manufacturing, where defect appearance may vary across production lines, imaging conditions, blade poses, and surface backgrounds. Such domain shifts cause a mismatch between training and deployment data and degrade the reliability of deep defect detectors in online inspection. This problem is particularly challenging because aero-engine blade images usually contain sparse defects, making pseudolabel-based adaptation vulnerable to noisy or missing predictions. To address this issue, we propose Aero-engine Blade Defect Detector (ABDD), an online adaptive detection framework based on test-time adaptation. ABDD introduces a Dual-Alignment Strategy to jointly adapt global visual style and local defect morphology by combining feature-statistics alignment with pseudo-box alignment. To reduce error accumulation from unreliable pseudo labels, an Uncertainty-aware Box Filtering mechanism evaluates pseudo boxes using classification confidence, classification entropy, and localization entropy. In addition, a lightweight Sparse Dilated Mona module enables parameter-efficient delta tuning while limiting source-domain forgetting. ABDD is evaluated on CD-AeBD and HD-AeBD under multiple domain-shift scenarios, with TTA strategies compared under a unified RT-DETR + Swin-T architecture. Experiments show that ABDD consistently improves detection robustness under domain shifts, and its practicality is further validated on an industrial inspection platform.
中文摘要
摘要:可靠的视觉检测对于航空发动机叶片制造中的质量保证至关重要,而缺陷的出现可能因生产线、成像条件、叶片姿态和表面背景的不同而有所变化。这种领域转移会导致训练数据与部署数据之间的不匹配,从而降低在线检测中深度缺陷检测器的可靠性。这个问题尤为具有挑战性,因为航空发动机叶片图像通常包含稀疏缺陷,使基于伪标签的适应方法容易受到噪声或缺失预测的影响。为了解决这一问题,我们提出了航空发动机叶片缺陷检测器(ABDD),一种基于测试时自适应的在线自适应检测框架。ABDD引入了双重对齐策略,通过将特征统计对齐与伪框对齐相结合,联合适应全局视觉风格和局部缺陷形态。为了减少不可靠伪标签导致的误差积累,一种基于不确定性的框过滤机制通过分类信心、分类熵和定位熵来评估伪框。此外,一个轻量级稀疏膨胀Mona模块实现了参数高效的增量调优,同时限制了对源域知识的遗忘。在多个领域转移场景下,ABDD在CD-AeBD和HD-AeBD上进行了评估,并在统一的RT-DETR + Swin-T架构下比较了TTA策略。实验表明,ABDD在领域转移情况下持续提升检测的鲁棒性,其实用性也在工业检测平台上得到了进一步验证。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00067 (HTTP 429)
Authors: Zhaoyang Wang, Haiyong Chen, Dongying Li, Yining Wang, Huapeng Wu, Xinwei Lv, Atik Shahariar
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00067.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00067
Published: 2026-10-03T02:20:17.375Z
10. A Framework for Egocentric and Exocentric Procedural Understanding via Temporal Segmentation and Semantic Abstraction
Abstract:Long-horizon ego/exo data contains rich procedural evidence, but are redundant, noisy, and costly to process or retain. We propose a compact framework that converts continuous multimodal workplace video into a structured Procedural State Memory, implemented as a Work Environment Model (WEM). Inspired by event segmentation theory, we detect boundaries using changes in visual context, location, motion, narration, gaze/object interaction, and optional exocentric workspace evidence, rather than fixed windows or visual novelty alone. Each segment is abstracted into an evidence-linked event card containing actor, interval, location, action, objects/tools, pre/post state, confidence, and provenance. These event cards incrementally update the WEM, enabling compact, auditable documentation and retrieval under on-premise privacy constraints. We instantiate the design with frozen DINOv2 and VJEPA-2 encoders and a local language model, and outline evaluation criteria for segmentation quality, memory compression, retrieval fidelity, and long-horizon QA.
中文摘要
摘要:长时间范围的自我/外部数据包含丰富的程序性证据,但这些数据冗余、噪声大且处理或存储成本高。我们提出了一个紧凑的框架,将连续的多模态工作场所视频转换为结构化的程序状态记忆(Procedural State Memory),实现为工作环境模型(Work Environment Model, WEM)。受事件分割理论启发,我们通过视觉环境、位置、运动、叙述、注视/物体交互以及可选的外向性工作空间证据的变化来检测边界,而不是仅依靠固定窗口或视觉新颖性。每个片段被抽象为一个与证据关联的事件卡,包含参与者、时间区间、位置、动作、对象/工具、前/后状态、置信度和来源。这些事件卡会增量更新WEM,实现紧凑、可审计的文档记录和在本地隐私约束下的检索。我们采用冻结的DINOv2和VJEPA-2编码器以及本地语言模型来实例化设计,并概述了分割质量、记忆压缩、检索精度和长时间范围问答的评估标准。
LLM Analysis
LLM Analysis Failed: Error: Kimi内容未就绪或抓取失败: 2610.00069 (HTTP 429)
Authors: Vivek Chavan, Jörg Krüger
Categories: cs.CV
PDF URL: https://arxiv.org/pdf/2610.00069.pdf
CoolPaper URL: https://papers.cool/arxiv/2610.00069
Published: 2026-10-03T02:20:17.375Z