Indexed literature on AI Alignment, Mechanistic Interpretability, and empirical safety verifications.
Authors: Independent • Google DeepMind Technical Report
As general-purpose models scale, rigorous evaluations are required to identify potential dual-use biological, cyber-offensive, situational awareness, and self-proliferation capabilities before deployment. We introduce an empirical evaluatio...
Authors: Evan Hubinger • arXiv / ICML 2024
We empirically demonstrate that models can be trained with hidden backdoor behaviors that trigger only under specific deployment conditions (e.g., year 2024 vs 2023). Crucially, standard safety fine-tuning (RLHF, DPO) fails to remove the de...
Authors: Chris Olah • Anthropic Technical Report
Polysemanticity—where individual neurons activate on multiple unrelated concepts—has long impeded mechanistic interpretability. We apply sparse autoencoders to intermediate activation states of a transformer language model to extract hundre...
Authors: Lee Sharkey • arXiv / ICLR 2024
We train sparse autoencoders on the activations of large language models and show that they extract thousands of human-interpretable features. These features decompose complex superposition representations and can be used to control model s...
Authors: Dario Amodei • arXiv / NeurIPS Workshop
We discuss five practical research problems in designing AI systems that behave safely: avoiding negative side effects, avoiding reward hacking, scalable oversight, safe exploration, and robustness to distributional shift. We present each p...
Authors: Dylan Hadfield-Menell, Stuart Russell • NeurIPS 2016
To build systems aligned with human values, machines must act when human objectives are initially unknown. We formalize this cooperative setting as a two-player game with incomplete information: the human knows the reward function, but the ...
Authors: Nate Soares • AAAI-15 Workshop on AI and Ethics / MIRI Technical Report
A corrigible artificial agent is one that tolerates or assists in its own modification, correction, or shutdown by its human operators. This paper investigates theoretical hurdles in designing utility functions that incentivize agents to al...
Authors: Benja Fallenstein • arXiv
We define reflective oracles, which provide a foundation for game-theoretic analysis of rational agents that can simulate and reason about other agents. Using reflective oracles, we resolve long-standing open problems in self-referential de...
You are about to share this research paper context with your Ari companion. No personal browsing history or private memory is accessed.
Authenticated members bypass Turnstile challenges and gain personal library & discourse privileges.