收录解读
运行时防护是大型语言模型可靠部署的关键,现有方法依赖外部独立模型,带来额外推理成本、延迟安全信号以及容量不匹配问题。
本文提出SingProbe,一种轻量级内在运行时防护方法,直接复用LLM推理过程中的隐藏状态,在自回归解码的同时连续预测查询意图、响应安全性和幻觉风险,仅需约2M参数和不到0.5%的额外开销,实现“免费午餐”式的防护。
全文复核的关键证据:约2M参数的内生探针在Ling-3.0-flash上将8个响应安全基准的平均F1提升至0.8728,超过最佳独立基线0.8604。;在三个流式安全测试集上取得平均R-AUC 0.9887、T-AUC 0.9481,T-AUC比最强Qwen3Guard流式基线高约4.9个百分点。;解码阶段跨1–32并发的ITL额外开销仅0.18%–0.45%,但预填充模式TTFT增加0.86%–2.25%。。
该工作把基座模型隐藏状态转化为低开销、逐词可用且能联动解码控制的通用安全原语,并以广泛基准、消融和服务延迟实验验证,但部分在线与医疗结论依赖LLM伪标签、组件化延迟估算及有限模型家族,尚不足以支持范式级结论。
原始摘要
Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external models that introduce additional inference cost, delayed safety signals, and a capacity mismatch with increasingly capable base models. To address these issues, we introduce SingProbe, a lightweight intrinsic runtime guard that directly reuses hidden states produced during LLM inference and operates alongside autoregressive decoding. Within a unified framework, SingProbe continuously predicts query intent, response safety, and hallucination risk at the token level with negligible additional guardrail inference overhead, offering a "free-lunch" solution. We further introduce SingStreamBench, a benchmark designed to assess whether streaming guardrails remain inactive on benign prefixes while promptly detecting emerging unsafe content. Extensive experiments show that SingProbe achieves competitive or superior performance compared with substantially larger standalone guardrails and specialized hallucination detectors, with only $\approx$2M parameters and $<0.5\%$ extra overhead. Beyond passive detection, we also show that SingProbe scores can anticipate future generation risk and guide constrained safe decoding. We further extend this paradigm to medical generation through SingProbe-Med, which selectively activates risk-directed decoding interventions only when clinically relevant risks emerge. Together, these results demonstrate that internal model representations provide an effective and efficient interface for generation-time monitoring and control.