Automated Alignment is Harder Than You Think

Summary This is a summary of a paper published by the alignment team at UK AISI. Read the full paper here . AI research agents may help solve ASI alignment, for example via the following plan: Build agents that can do empirical alignment work (e.g.~writing code, running experiments, designing evaluations and red teaming) and confirm they are not scheming. [1] Use these agents to build increasingly sophisticated empirical safety cases for each successive generation of agents, gradually automating more of the research process Hand over primary research responsibility once agents outperform humans at all relevant alignment tasks. We argue that automating alignment research in this manner could produce catastrophically misleading safety assessments, causing researchers to believe that an egregiously misaligned AI is safe, even if AI agents are not scheming to deliberately sabotage alignment research. Our core argument (Fig. 1) is as follows: The goal of an automated alignment program is to