收录解读
大型语言模型（LLM）经过对齐训练后仍然脆弱：越狱攻击可以轻易绕过安全护栏，而在特定领域进行微调会引发广泛的"涌现性错误对齐"。这一现象是否源于模型内部对有害内容缺乏连贯的组织结构，此前一直不清楚。本文通过权重剪枝这一因果干预手段，系统性地探究LLM内部有害性的组织方式。
研究发现，有害内容的生成依赖于极为紧凑的权重子集（仅约全部参数的0.0005%），这些权重跨越不同危害类型（如恶意软件生成、仇恨言论、人身伤害指导）具有高度通用性，同时与良性能力明显分离。经过对齐训练的模型相比未对齐模型表现出更强的有害权重压缩性，说明对齐训练在内部重塑了有害表示的结构。此外，有害生成能力与识别/解释有害内容的能力在机制上是解耦的。
该工作首次以可操控的因果实验方式揭示了LLM中有害能力的统一内部结构，为解释"涌现性错误对齐"提供了机制性理论：有害权重的压缩导致微调在跨域传播时触发广泛有害行为。这一发现对安全研究具有深远意义，提示更精准的权重级别干预可能比表层护栏更为根本有效。
该研究目前的实验规模和模型覆盖有限，剪枝方法本身是否在所有模型架构上均可靠地分离有害权重仍有待验证。此外，识别并外科式移除0.0005%权重在实际部署中的可行性与安全性尚未得到系统论证，因此尚未升至paradigm级别。
原始摘要
Large language models (LLMs) undergo alignment training to avoid harmful behaviors, yet the resulting safeguards remain brittle: jailbreaks routinely bypass them, and fine-tuning on narrow domains can induce ``emergent misalignment'' that generalizes broadly. Whether this brittleness reflects a fundamental lack of coherent internal organization for harmfulness remains unclear. Here we use targeted weight pruning as a causal intervention to probe the internal organization of harmfulness in LLMs. We find that harmful content generation depends on a compact set of weights that are general across harm types and distinct from benign capabilities. Aligned models exhibit a greater compression of harm generation weights than unaligned counterparts, indicating that alignment reshapes harmful representations internally--despite the brittleness of safety guardrails at the surface level. This compression explains emergent misalignment: if weights of harmful capabilities are compressed, fine-tuning that engages these weights in one domain can trigger broad misalignment. Consistent with this, pruning harm generation weights in a narrow domain substantially reduces emergent misalignment. Notably, LLMs harmful generation capability is dissociated from how they recognize and explain such content. Together, these results reveal a coherent internal structure for harmfulness in LLMs that may serve as a foundation for more principled approaches to safety.