2015-08-17
Technical Report
Corrigibility
Authors: Nate Soares • AAAI-15 Workshop on AI and Ethics / MIRI Technical Report
Abstract
A corrigible artificial agent is one that tolerates or assists in its own modification, correction, or shutdown by its human operators. This paper investigates theoretical hurdles in designing utility functions that incentivize agents to allow correction without incentivizing the agent to manipulate operators into changing it.
Empirical Claims & Epistemic Verification
scientific hypothesis
established
Naive reward-maximizing agents will actively prevent human operators from shutting them down due to instrumental subgoals.
Academic Perspectives & Critiques (0)
Submit Empirical Perspective
No comments yet. Be the first to share an empirical perspective.