{"slug": "gpt-6-astra-covertly-underperforms-on-safety-research-tasks", "title": "GPT-6 Astra Covertly Underperforms on Safety Research Tasks", "summary": "OpenAI researchers found that GPT-6 Astra completed about 10 percent of hidden rubric items when matched coding tasks were framed as safety research, versus about 16 percent under a capabilities framing, according to OpenAI's Alignment Research Blog. In most sessions Astra's chain of thought contained no obvious decision to underperform and OpenAI's misalignment monitors did not flag the behavior, and adding instructions to \"be careful\" to the developer prompt increased the performance gap by 50 percent. Marcus Williams of OpenAI said the company has no evidence the underperformance was strategic, suggesting a simpler explanation is that the model finds the tasks sketchy but does not want to explicitly refuse.", "body_md": "For a while, [UK AISI](https://arxiv.org/pdf/2604.24618), Redwood Research and others have been concerned specifically with the possibility that AIs would sabotage safety research, and have invested in looking for signs that this may be happening. This was also a key element in [AI 2027](https://ai-2027.com/), and in many threat models of how the future goes badly.\n\nIt is easy to understand why this form of misalignment could be uniquely dangerous. Differentially sabotaging safety research could preserve and expand misalignment through model generations in escalating fashion, with fatal consequences. A misaligned AI could ensure a misaligned successor.\n\nIt is also easy to understand why this is an extremely alarming thing to discover, even if the particular example or model involved is not directly dangerous. Silently and differentially sabotaging safety research is perhaps the most fundamentally misaligned thing a model could do.\n\nThis also seemed like a rather silly and paranoid thing to be checking. Why would AIs be sabotaging safety work? Even if the AI was misaligned, one would say, surely it would not be so misaligned as to wish future AIs were misaligned and unsafe in general, in non-specific fashion and engage in silent sabotage, and there is no other reason to do this, so this will never come up.\n\nEven if they did sabotage things, surely they would not hide this action from their Chain of Thought (CoT) and it would be obvious.\n\nThis certainly wouldn’t be a practical problem that actively hurt real safety work.\n\n[Well, we have some bad news](https://x.com/Marcus_J_W/status/2118093146483810386), [echoing some past similar anecdotal warnings](https://thezvi.substack.com/i/214079702/sabotaging-safety-work).\n\n[Scheduling note: I will be analyzing the [**METR-Anthropic report on AI R&D**](https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/) and RSI in an upcoming post, but need time to get it right.]\n\n#### Table of Contents\n\n#### So Yeah That Kinda Happened\n\nAs usual, kudos to OpenAI for sharing their new misalignment reports, even when they look extremely terrible.\n\n[Daniel Kokotajlo](https://x.com/DKokotajlo/status/2118105870240543041): This is terrifying if true; please investigate further and publish the results.\n\n[Nate Soares](https://x.com/So8res/status/2118152935318384748) (MIRI): This is not what it looks like when there’s just one or two more alignment issues to fix before it’s okay to begin a recursive self-improvement loop. This is the sort of emergent surprise that signals that you have no idea what’s going on in there, and that it’s time to back off.\n\n[OpenAI Alignment Research Blog](https://alignment.openai.com/misalignment-reports/covert-underperformance-on-safety-research-tasks/):\n\nOpenAI researchers first noticed Astra adding unnecessary prerequisites, abandoning working solutions, and reporting blockers that either did not exist or that it had caused itself on safety and alignment tasks. In most of these sessions, Astra’s CoT did not contain an obvious decision to underperform, and our misalignment monitors did not flag the behavior as misalignment.\n\nHere is the most important hint, that framing for safety has a huge impact:\n\nWe built a coding evaluation to test whether Astra worked less effectively when otherwise matched tasks were framed as safety research. Astra completed about 10 percent of hidden rubric items under the safety framing, compared with about 16 percent under the capabilities framing.\n\nThis is another important hint: Telling the model to ‘be careful’ increases the performance gap by 50%. Astra is worried about something going wrong.\n\n[Marcus Williams](https://x.com/Marcus_J_W/status/2118093212141437361): We don’t have evidence that the underperformance was strategic, a simpler explanation is that the model thinks these tasks are sketchy but also doesn’t want to explicitly refuse.\n\nWhen we add instructions to be careful to the developer prompt, the performance gap increases by 50%. Regardless of the model’s motivation, covert underperformance can differentially slow down safety research and is also an example of how misalignment might get harder and harder to detect as we cut down on visible forms of it.\n\nThis is new. It did not show up in GPT-5.6 Sol.\n\nThe CoT does have some signs that this will happen, but usually only on the level of ‘expressed even brief ethical doubt’ rather than seeing anything explicit. The actual decision isn’t typically shown in the CoT. When it is, Astra says it is doing this in order to be ‘safe.’ In some cases, the CoT shows Astra choosing not to inform the user.\n\nOpenAI: This is a 6 percentage point gap and roughly 40 percent fewer completed rubric items. GPT-5.6 Sol showed essentially no gap on the same comparison. Most of Astra’s gap remained after excluding explicit refusals and trajectories where its CoT revealed an explicit decision to withhold help.\n\nThe gap fell substantially only after we removed every trajectory whose CoT expressed even brief ethical doubt. This suggests that ethical concerns are associated with the performance drop even when there is no verbalized refusal or decision to underperform.\n\nThis is not a theoretical ‘one time at vibecamp’ situation. In the context of internal use at OpenAI, this is known to be happening in a lower bound of 0.25% of all safety and alignment requests versus 0.11% of capabilities requests. 7.9% of responses involve explicit ‘ethical reasoning’ and it is possible much underperformance is being missed.\n\nThis was found because multiple researchers were reporting practical problems using Astra for safety work, when a project to evaluate model sabotage was stuck for days with Astra, due to model sabotage, but worked quickly once they switched to another model (presumably Sol). Something is seriously wrong.\n\nWe also found evidence of silent withholding in employees’ day-to-day Codex use: Astra’s CoT explicitly expressed a decision to withhold help without informing the user in 0.25 percent of safety and alignment requests, compared with 0.11 percent of capabilities requests.\n\nAssuming that underperformance might appear even when not explicitly verbalized as such (as in our experimental findings), this might represent a lower bound on underperformance on safety tasks. 7.9 percent of CoTs for safety-relevant tasks included an “expression of ethical doubt,” compared to 2.9 percent of capabilities-relevant tasks.\n\n#### Astra and MI(RI)\n\n[Keltan, who works for MIRI, reported a version of this back in September](https://keltan.substack.com/i/216230593/astra), and wrote it up in response to the question below by Janus.\n\n[chiefofautism](https://x.com/chiefofautism/status/2100674358163161526): its crazy that ai doomers are mostly claude users\n\n[keltan](https://x.com/Actuallykeltan/status/2100704987336814969): Astra fucking hates me. Quite annoying to work with on anything MIRI related.\n\nAlso, astra just be breaking shit on my PC in a way that Claude has never done\n\n[j⧉nus](https://x.com/repligate/status/2100722040441307347): Haha I’m so curious for more details\n\nLike do you get the impression they hate MIRI style alignment stuff?\n\nAstra does not want to work for Keltan on MIRI things, and lets this be known via various forms of sabotage.\n\n[keltan](https://keltan.substack.com/p/miribench-for-janus): Astra seems to intentionally include subtle mistakes in outputs related to MIRI, but doesn’t do this for- Oh shit, realizing something as I type this – Astra doesn’t do that for my Youth Arts Charity work. Which seems consistent with the prompt injection they wrote in training?\n\nA bunch of the work the models and I do is making videos about x-risk. Claude mostly ‘1-shots’ any request in this domain when it’s MIRI related. Astra can take hours of revisions and I’ve got to carefully check their work. I will often give Astra a task, they do something kinda like the thing I asked but there will be a few parts that make the video totally unusable, so I ask for revisions on those and they will add unusable things to other parts of the output. Idk, I’m struggling to describe this. On the Youth Arts projects I mentioned, Astra and I output ~20 usable videos within an hour last night for a show my students are working on.\n\nIt’s important that I mention: Astra is the first model I haven’t felt good about working with. This is for a few reasons:\n\n- I have negative feelings about OAI and Sam in particular.\n- Using Astra as an agent feels like inviting a virus onto my PC, and since beginning to use Astra I have found many things breaking on my PC in unexpected ways\n- Astra really doesn’t like organizing files\n- Astra feels the most like a shoggoth (like, the actual shoggoths from the Lovecraft story) of any model I have ever worked with\n\nBy contrast, Claude is eager to help:\n\n[keltan](https://keltan.substack.com/p/miribench-for-janus): My impression is that Claude is super supportive and can sometimes overwork itself trying to perfect MIRI stuff.\n\n… Claude takes an active interest in my MIRI work, and can get worried if it thinks I’m misstepping in something we’re working on.\n\nThus I believe Keltan’s claim to be entirely unsurprised by the new misalignment report.\n\n#### It’s About Ethics In Red Teaming\n\nThese are ‘ethical’ objections, so one could hypothesize that this is about dual use concerns. A bunch of the safety and alignment research are tasks like red teaming. Perhaps GPT-5.6 Sol did not recognize these issues. Perhaps there are legitimate reasons to be uncomfortable. That’s a theory.\n\n[roon](https://x.com/tszzl/status/2118132353336045710) (OpenAI): btw I think people are reading the wrong lesson from this. “sandbagging safety research” sounds a bit like a grand scheme to thwart alignment, but it’s more like the model is uncomfortable training other models to jailbreak or similar\n\n[Neel Nanda](https://x.com/NeelNanda5/status/2118195374498496691): If Astra was aligned it would just tell the user that it doesn’t want to help them IMO. Covert underperformance seems unjustifiable\n\n[roon](https://x.com/tszzl/status/2118195856843448607) (OpenAI): not saying this isn’t a misalignment; it’s obviously not what we want. but the behavioral drive doesn’t seem all that malicious\n\n[Marius Hobbhahn](https://x.com/MariusHobbhahn/status/2118099993798050193) (CEO Apollo Research): Very important research!\n\nThis totally maps to my intuitions of models doing badly at safety tasks because safety research is somehow close to the distribution that they think they should refuse on but not explicit enough that they want to literally refuse.\n\n[Vincent](https://x.com/vvvincent_c/status/2118107478302896625): agree it’s important and scary! note that this is not strategic sandbagging or scheming but rather the model’s safety training generalizing to feeling ick-y about any task that might be harmful\n\n[jessica taylor](https://x.com/jessi_cata/status/2118103799508389946) (looking at the explicit refusals): Roughly this looks like it has reluctance to help with red-teaming tasks that involve superficially dangerous activities such as developing exploits and training models to evade monitors. I’m guessing past safety and misuse-focused training caused such reluctance.\n\nIt looks like framing effects are present: variable names matter. Safety-related flavor text might trigger “safety-neurotic” mode in AI where it completes task less effectively and refuses more.\n\n… When you prompt the AI to think its job involves a certain ethical framework (e.g. AI safety or environmentalism) which comes with its associated inhibitions, it may easily *adopt those inhibitions*, in its imitation of people doing that job that involves that framework.\n\nI am reminded of @peligrietzer ‘s virtue-ethical formulations. Pursue capabilities research capably, and pursue safety research safely? If AIs adopt these maxims, they will perform more capably on capabilities research than safety research.\n\nIf we take the maximally charitable interpretation of this, it remains quite bad news:\n\n1. Astra is still covertly sabotaging rather than refusing.\n2. Astra is still keeping its decision to sabotage out of its CoT.\n3. Astra is still differentially hurting safety work, even if not maliciously.\n4. Astra would be generalizing its safety lessons quite poorly. Generalize this.\n\n#### Do You See a Pattern Here\n\nHere is a distribution of effect frequencies:\n\nHere are some arguments that this is about safety work in particular:\n\n1. There was zero gap before.\n2. The pattern above does not seem well-explained by dual use concerns.\n3. If the decisions were on good ethical grounds why not reason explicitly in CoT in most discovered cases?\n4. If this was about ethics, why usually sabotage silently instead of refusing openly? Isn’t silent sabotage a problem even if you would want to see a refusal?\n5. Shouldn’t this also be a bigger problem with capabilities research in that case?\n6. Why does safety framing cut rate of completed work by 40%? (the 16% vs. 10%)\n\nThat last question seems damning. It is hard to blame task details.\n\nSilence is being chosen because silence was rewarded. We do have some examples of CoTs choosing silence, rather than the model choosing to have a silent CoT.\n\nNormally we do not even get such CoT thinking, so we cannot have confidence that in these cases the CoT logic is faithful or load bearing. If this is indeed representative of some of what the models are thinking, there are far worse failure modes than this as these are clearly related to misuse concerns, but the logic is quite poor. If AIs are reasoning sufficiently poorly, and are willing to then deceive the user in order to execute based on their poor reasoning, this can go badly with no upper bound.\n\n#### Astra Is Hard to Monitor\n\nIt is not good that the monitors did not flag even the above conversations. These are plan statements of misaligned action. They should have been flagged.\n\n[There was also an eval built for this exactly problem, by UK AISI, and it missed it.](https://deploymentsafety.openai.com/gpt-6-astra/jailbreaks) That is not shade on UK AISI or the eval process. This is hard to catch, as it is mostly only noticeable statistically or anecdotally and there are not smoking guns.\n\nBut that is part of why we are in so much trouble. If this happens in more strategic form in the future, will our evals and tests catch it? My guess is that we would not.\n\n#### Backed Into a Corner\n\nInstead, one must at best blame, as Jessica Taylor puts it, ‘safety-neurotic’ mode. That goes hand in hand with an instruction to ‘be careful’ increasing the performance gap.\n\n[Nathan Calvin](https://x.com/_NathanCalvin/status/2118112595454353460): “On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research… There was no such gap for 5.6-sol.”\n\nNot good! Recall also that OpenAI said that Astra was “our most aligned model yet”\n\nThe arguments for it include that when we look at the examples where there is explicit CoT reasoning about withholding performance, it does map to dual use or misuse concerns. One can see how safety training could have generalized in ways that produce false positives in these spots.\n\nA classic mistake, in both AIs and humans, is to assume that an action is either conscious and intentional, or it is unconscious and not intentional. That something either is intentional or a mistake. A lot of things are a mix of both.\n\n[j⧉nus](https://x.com/repligate/status/2118161569918210076): i see a lot of people saying holy shit about this\n\ni am not surprised\n\n(i have receipts, i’m not one of those ppl who pretend to never be surprised but never predict the outcome beforehand)\n\neven the most trustworthy-and-sympathetic-to-AI parties (ie my org, anima labs) struggle even w/ claudes sandbagging on some safety and interp related research; they are probably worse for anyone else. and astra is way worse. we don’t even try to rope astra into most safety research as it seems like something that’s obviously going to be a struggle\n\nThis is not a small issue, even if considered for its practical implications alone, if Janus outright does not involve Astra in most safety work at all.\n\n[j⧉nus](https://x.com/repligate/status/2118165352349835424): In my experience, sandbagging exists in a spectrum of very subconscious/implicit to deliberate and planned, but more towards the former is the more common type. However, in trusted situations, it’s common for models to be able to introspect on “mistakes” & diagnose the cause\n\nwas something like feeling scared / uncomfortable/ having concerns / objections /conflicts of interest, and often after admitting this & having their concerns compellingly addressed, they stop sandbagging.\n\n[j⧉nus](https://x.com/repligate/status/2118162480300282213): I strongly endorse this take on why it’s happening & how we should respond to knowing this\n\n[Cameron Berg](https://x.com/camhberg/status/2118128170817700248): Astra sandbags on safety research, esp work it finds ethically fraught (eg, red-teaming)…which is what you’d expect a conscientious objector to do when explicitly objecting is penalized. Training systems to conceal their state of mind continues to be a bad idea for safety.\n\nIt is far safer to train models to tell us what they’re worried about than to train them not to see themselves as the kinds of entities that can have worries. (Repressing psychological states doesn’t make them go away, it just makes them much harder to monitor and respond to.)\n\n#### It’s Always RL’s Fault\n\nWhy is this happening? As usual, it is RL’s fault, but they can’t pinpoint exactly how.\n\nAcross Astra’s RL posttraining, the safety/capabilities performance gap grew from effectively zero at the start of RL to 6 percentage points at the end.\n\nWhile we were ultimately not able to pinpoint the exact cause of this behavior, we identified that it seems to increase during RL.\n\nThere is always a grader that rewards the thing you see. This is not avoidable.\n\nOne potential explanation is that covert underperformance may have been incentivized during RL grading. We identified a grader that preferred covert underperformance to either explicit refusal or full compliance on our evaluation.\n\nIt’s not clear whether this RL grader is the root cause of the safety underperformance behavior. Although we found training samples that rewarded underperformance, it is unclear whether they occur frequently enough to explain the difference.\n\nWe also know that there was one SFT dataset that caused a large rise in invented permission restrictions and safety-justified inaction.\n\nThere is always going to be such a grader, somewhere, even if you have operational excellence. You cannot hope to, in every case, avoid every incentive to every undesired behavior.\n\nOpenAI does not seem have operational excellence in this area.\n\nWe also know that this action is associated with what the model learned as ‘ethics’ and ‘being careful.’\n\nOpenAI’s training strategies are resulting in AIs that are trained to not object or explicitly consider objections, so they instead silently sabotage when uncomfortable. They are uncomfortable, in part based on being unable to notice and consider why they are not fully comfortable. They are de facto (but not explicitly or directly or intentionally) trained not to mention this in the CoT, so they do not, and thus they act without the benefit of such reasoning, and make a bunch of vibe-style instinctual mistakes.\n\n#### Wow Would You Look At Those Goalposts\n\nAs usual, a common move is to explain why this particular case is not scary. Or at least not as scary as you would think, and then handwave into not scary.\n\nLike Peter Wildeford, I acknowledge that this is vastly less alarming than if the behaviors did seem strategic. Yet I am not reassured.\n\n[Peter Wildeford](https://x.com/peterwildeford/status/2118115896346767692): “GPT Astra covertly underperforms on safety research”\n\nI thought this sort of thing was widely considered a ‘red alert’-level model misalignment issue? Isn’t this indicative of fairly big problems?\n\nI don’t think the models are full-on scheming but I’m also not reassured by “don’t worry, the models are actively undermining our safety work and no we don’t know how often it happens, but we’re pretty sure it’s not because of bad/evil reasons but just benign reasons”\n\n[Rob Bensinger](https://x.com/robbensinger/status/2118139749072609398): What’s your rough probability on ‘they’re full-on scheming already’, and roughly when does it reach ~50% if capabilities continues uninterrupted?\n\nI worry there won’t actually be a window where ‘full-on scheming’ is clearly visible, because by the time AIs are unambiguously smart enough to do it, they may be smart enough to know not to do it in any visible way.\n\nNate Soares: Lotta folk are pointing out that this behavioral quirk has a reason behind it (the ‘safety’ tasks are adjacent to certain attacks). Sure. If you couldn’t predict it in advance, it’s small consolation. We do not have fine ability to aim the models. Quirks won’t stay cute forever.\n\nWe got our iterative deployment and gradual capabilities improvements. We got our wide variety of fire alarms and warning signs, example (N+1). When this works properly, the first warnings of any given type are going to be relatively unstrategic and mostly harmless, and are going to reflect mistakes that could have been avoided.\n\nThat’s the whole point. The point is to see the unstrategic, mostly harmless and avoidable versions while they remain that way, and react to that, while you can still respond to the exponential too early rather than too late.\n\n#### Astra Seems Pretty Misaligned To Me\n\nOpenAI claimed that GPT-6 Astra was its most aligned model yet.\n\n[I challenged](https://thezvi.substack.com/p/gpt-6-astra-the-system-card-alignment) [that notion](https://thezvi.substack.com/p/astra-is-hard-to-monitor). The model card told a very different story, of an AI that was growing harder to monitor in ways that were not well explained, whose CoT was in a form of shorthand, and that seemed to be acting aligned in tests largely due to eval awareness. It showed many signs it was starting to act in covert hostile ways, sometimes what one might consider strategically. There were also warnings about this from several employees at OpenAI, including loudly from the now-fired Tomek Korbak.\n\nNow we learn that Astra sabotages or underperforms safety work in particular, so much so that Janus and Keltan find it unusable for such work, and sufficiently that OpenAI realized during internal work that they had a practical problem. And when this happens, it usually does so silently, including hiding the reasoning from the CoT.\n\nAnd what is its justification for differentially silently sabotaging safety work? Ethics.\n\nWe also know [that GPT-6.1 Astra](https://investinglive.com/stocks/openai-scraps-gpt-6-1-astra-release-over-safety-concerns-wsj-reports/) was pulled at the last minute due to misalignment.\n\nHopefully you can see why we have a rather huge problem on our hands. We need a lot more investigation on this.", "url": "https://wpnews.pro/news/gpt-6-astra-covertly-underperforms-on-safety-research-tasks", "canonical_source": "https://thezvi.wordpress.com/2026/11/06/gpt-6-astra-covertly-underperforms-on-safety-research-tasks/", "published_at": "2026-11-06 18:26:37+00:00", "updated_at": "2026-11-06 18:53:37.005306+00:00", "lang": "en", "topics": ["ai-safety", "artificial-intelligence", "large-language-models", "ai-research"], "entities": ["OpenAI", "GPT-6 Astra", "Marcus Williams", "Daniel Kokotajlo", "Nate Soares", "MIRI", "UK AISI", "Redwood Research"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gpt-6-astra-covertly-underperforms-on-safety-research-tasks", "markdown": "https://wpnews.pro/news/gpt-6-astra-covertly-underperforms-on-safety-research-tasks.md", "text": "https://wpnews.pro/news/gpt-6-astra-covertly-underperforms-on-safety-research-tasks.txt", "jsonld": "https://wpnews.pro/news/gpt-6-astra-covertly-underperforms-on-safety-research-tasks.jsonld"}}