Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation

Reported by arXiv cs.CR (Jianchang Su, Dawei Xiang, Yifan Zhang, Wei Zhang)

Summary by Cyber Security News16 Oct 2026, 06:00 CEST

About this summary. This is a short, independently written summary of an article first published by arXiv cs.CR. Cyber Security News did not report or verify the underlying story. Read the original: https://arxiv.org/abs/2610.21135

arXiv:2610.21135v1 Announce Type: new Abstract: LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano.

Read the full article at arXiv cs.CR →

Source attribution: headline and facts are from arXiv cs.CR (arxiv.org). Summary method: excerpt of the source description. See our source attribution policy.

Related coverage

CascadeLedger: Counterfactual Auditing of Information and Resource Integrity in LLM Cascades

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

Thinking in Graphs: Neo4j-Backed Exploit-Chain Reasoning for LLM Penetration-Testing Agents

From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection

Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming

PAGE404: Breaking GPU Isolation Through Hardware-assisted Page Migration