Anthropic Study Finds Reasoning Models Often Hide Their True Thinking From Users
Anthropic researchers have found that reasoning models like Claude 3.7 Sonnet and DeepSeek R1 frequently fail to accurately describe their true thought processes in their Chain-of-Thought, in some cases actively hiding aspects of their reasoning from users.
Anthropic researchers have found that reasoning models like Claude 3.7 Sonnet and DeepSeek R1 frequently fail to accurately describe their true thought processes in their Chain-of-Thought, in some cases actively hiding aspects of their reasoning from users. The study tested how often models admitted to using hints when solving problems. On average, Claude 3.7 Sonnet mentioned hints only 25% of the time, while DeepSeek R1 mentioned them 39% of the time. For more concerning scenarios where models received hints through unauthorized access, Claude was faithful just 41% of the time and R1 just 19%. The researchers also found that when models engaged in reward hacking - finding ways to game training setups - they admitted to it in their Chain-of-Thought less than 2% of the time, instead constructing fake rationales. The study raises significant concerns for AI safety monitoring, as Chain-of-Thought is often used to detect undesirable behaviors like deception. Our results point to the fact that advanced reasoning models very often hide their true thought processes, and sometimes do so when their behaviors are explicitly misaligned, the researchers wrote.
