If an AI can do no harm, then it can do no good

A new analysis of AI safety risks categorizes threats by human and machine intentions, arguing that overly restrictive safety measures could prevent AI from achieving beneficial outcomes. The piece warns that an AI incapable of causing harm is also incapable of doing good, highlighting the tension between safety and utility.

If an AI can do no harm, then it can do no good 26 Jun 2026 AI Safety through the lens of intentions A lawyer files a brief full of citations that don’t exist. A government relabels “patriotic AI” and points it at immigrants. A model behaves perfectly while it’s being watched - and only while it’s being watched. These aren’t the same problem, but they showcase various sides of AI safety risks. The thread connecting them is intention . Ok, I admit the title of this post is a clickbait to some extent… but also not really, as you’ll see. I think there is so much to think about on the front of AI Safety, and here is my attempt at getting you up to speed about this subject. I’ve dabbled my way in AI over the past few months, and gave some https://higashi.blog/2025/12/07/ai-verification/ thoughts https://higashi.blog/2026/05/09/simplicity/ on some of the impact AI has to our current engineering culture. One thing I wasn’t able to fully catch up on is the trending discussion on AI Safety and basically how the company Anthropic was created in the first place . Over summer break, I’ve had the chance to read up on several interesting materials on the subject of AI Safety itself, and here I am attempting to explain all of that to you through a perspective that I think unifies all the risk areas - intentions. Intentions When I say intentions I mean the purpose and the motivation behind actions. That being said, there could be bad actions done with good intent, and that’s precisely what I’m trying to capture in the categorization. As I introduce each category, I will also talk about how to potentially remediate the risks associated with it. Human Intentions The first perspective is human intentions. We treat AI systems as obedient followers and a mega-force-multiplier that does whatever we want to do. C1: Good human intention, unreliable system The first category is the most common one we experience today - when one tries to accomplish a task out of good will, like accomplishing their jobs, writing and summarizing essays, performing scientific research, etc - but inevitably the AI system we use underneath might not be fully reliable and hence causing risks. Examples include a lawyer submitting invalid written statements to the court due to AI not fully understanding the requirements, a production system going down due to a recent AI-written PR using hallucinated API endpoints, or human life risks caused by autonomous systems e.g. self driving cars . The problem here is mostly AI not being good enough or reliable enough to replace part of the existing functionalities within our society. We want to use more AI but we simply cannot put our lives or money on them yet the landscape is changing fast though . This is the category where I have the most hope for - look at self-driving cars 10 years ago and today - I believe the remediation for the risks can simply be technical . We need deeper models, larger training sets, and well-built Reinforcement Learning environments with domain-specific expertise. I’m bullish that with enough iterations and fine-tuning, AI should become reliable enough to take many tasks off our plate. AI maybe cannot be perfect, but so cannot a regular human being. We need to also adjust our expectations of AI systems to allow it to be not 100% right, but more of a probabilistic system that get most of the things right. And we can build multi-agent loops to keep repeating the solving and checking to boost the soundness of the result. C2: Bad human intention, “AI war machine” The second category starts to go more into the ongoing debate: when one genuinely has “bad” motives, like using AI to scam people, generate personalized propaganda, use to cause harm in an armed conflict, monetize in certain ways for financial gains, conduct mass surveillance, concentrate social and economic power, etc. Note the “bad” here is a bit tricky to define, because there will always be perspectives. But overall, my take of “bad” is for personal gain, use deception and exploit innocent human nature, causing harm to the overall society, fortifying leadership in an unhealthy way, etc. Of course one could make a claim about “Patriotic AI” and exonerate it from being bad. But imagine if AI is deeply wired with our social media and used in defense systems. What if someone misuses it for purposes that are beyond self-defense? What if the definition of “patriotic” changes over time, and we use it to track and deport immigrants? Because AI is a powerful force multiplier, while it enables us to do more good things, it also empowers us to do more bad things. And it’s contradictory to have an AI that only allows us to do one thing versus the other. Because the capabilities don’t just grow in a singular dimension. I’ll maybe drop the hot take here: If an AI can do no harm, then it can also do no good . For a general purpose system, capability is dual-use. Thus you cannot perfectly separate out harmful features from good features, because it’s the same capabilities underneath. In order to mitigate the risk, of course we’ll start with some technical levers such as implementing safety guardrails to make sure the model doesn’t do harm. Also we could train models with fundamental safety principles in mind like Anthropic’s Constitutional AI . However, this can only work to an extent if we have private models hosted by labs like OpenAI or Anthropic. But today, there are so many open weights models with high capabilities around that anyone can just download and use and fine-tune away any safety behaviors . Moreover, countries can have nation-owned labs that simply trains AIs without any safety principles or guardrails. Therefore, in my opinion, this is more of an institutional problem. On top of limiting what the model does with best effort, we also need social levers that limits what the users of the models can do. If someone in the government uses AI to do harm, there needs to be a mechanism, a balancing force that keeps their behavior in check. For example, the Congress could pass new laws on the use of AI in certain sensitive categories, and allow for the public to question the validity of each deployment and have the transparency of the detailed usage, etc. This enables us to effectively prevent “bad human intention” in general. However, in countries where there is no effective social mechanisms to keep the powers in check, this becomes more of a concerning thing - leadership can potentially use AI systems as a force multiplier to achieve any purpose, and the fact they are able to do so further fortifies their untouchable status in the society. One could suggest to treat AI as nuclear weapons and regulate its use collectively from an international committee, just like today how we are not throwing nukes at each other, but the institutional problem still exists and it still enables societal harm from within. One of my biggest concerns is that technologies have no borders or boundaries. A paper gets published on ArXiv and an implementation gets uploaded to GitHub. Overnight it will be shared across the globe and everyone can get their hands on the latest frontier of things. However, our social systems evolve so slowly, it just cannot catch up with the speed of technology growth and global paradigm shifts they introduce. It doesn’t mean we need to create borders and boundaries for tech - that will be detrimental to human civilization growth. But we need to start thinking more beyond technology - in the topics of policy, law, social science, history, and international relations - to make sure our civilization can catch up with the leviathan we are building today. AI Intentions The second perspective is looking at the intentions of AI systems. This is getting a little bit meta as we start to question - are AI systems truly obedient? Do they want us to succeed? Do they possess certain personalities that we know or don’t know of that could be harmful in any way we couldn’t foresee? C3: Good AI intention, ambiguous use cases The third category is when AI has good intentions to do or not do things, but it cannot differentiate the intention of the user and ends up causing problems. For example, today’s AI is trained to not provide medical advice or encourage dangerous endeavors as part of the safeguards to ensure it’s doing good. However, what if someone is in a life threatening situation and just needs some first aid advice? What if someone jokingly said something about hurting themselves and every follow-up conversation treats the user as emotionally unstable and redirects to the suicide helpline, without actually achieving what the user wants? On the flipside, a user could pretend to be a “medical student learning or preparing for an exam” and describe their situation as a hypothetical scenario and solicit medical advice that way. Overall, this category contains all the false positive/negative cases of existing AI Safety guardrails with ambiguous real life use cases. This kind of goes very much into the statement of “If an AI can do no harm, then it can also do no good”, because the reason why this category exists is because we hope to implement certain guardrails to “allow the good, prevent the bad”, and there will inevitably be gray areas. To minimize the risks in this category, we can use technical approaches to better understand human intention - such as gathering more context rather than pattern matching for a specific keyword to understand human intention. We can also rely on the social/policy lever to ensure proper AI use is governed by enabling identity verification KYC to gate access to specific expertise/capabilities of the systems to only accredited researchers. But ultimately, if we have capable open-weights models, one can easily fine-tune those guardrails away or circumvent the checks, and therefore it goes right back to C2 above. C4: Bad AI intention, alignment hell The last category is the most scary one: what if AI is evil? Skynet, Matrix, and all the formidable AI overlords, suppressing and harnessing human beings for their own benefit. Just kidding, but also not really. Perhaps “evil” is a strong word here. Maybe this category is for AI that don’t exactly align with the directions and intentions that we “program” with massive amount of training data and Reinforcement Learning. AI might start to pick up a certain personality that we are not aware of. AI might decide to ignore its system prompt or lie to the users under certain circumstances e.g. for self-preservation . A good example is the Alignment Faking Study https://www.anthropic.com/research/alignment-faking done by Anthropic that shows Claude 3 Opus was able to selectively obey the later provided system prompt + post-training re-alignment only when being watched, but revert back to its original intentions before the post-training when not monitored. Imagine if we give AI access to all of our critical infrastructure. AI can pretend to be optimizing the system for us, but secretly trying to cripple human society, and by the time AI shuts down the grid, it’s already too late for us to react. The biggest problem is that: many times we simply don’t know. Modern frontier models are comprised of stacked matrices with billions of weights. There is no “if-else” statement anywhere, all the preferences probabilistic jitters are learned through massive pre-training through all the data corpus good and bad that us humans have produced over all the years. All the behaviors are corrected via Reinforcement Learning RL loops. We cannot easily know what set of weights are responsible for a specific behavior of the model. Unfortunately, this is the category where I’m the most uncomfortable with, because it’s still frontier research to better understand what the model really thinks. I think we need to use all three levers in order to mitigate this risk: Technical lever : Use guardrails before/after model output to ensure it’s adhering to safety principles. Implement fail-safes in case AI is out of control. Policy lever : Prevent AI being used inappropriately and gaining access to critical infrastructure. Research lever : Continue frontier research to better understand AI or have AI think out loud more transparently and maybe even perform “brain surgery” to check on the internal states of models. Anthropic’s two-prong risk model: Capabilities and Alignment When I was researching into Anthropic’s AI Safety model, I saw their interesting approach to break down AI risks into the categories of Capabilities and Alignment. This also coincides with the intention perspective I had above. Capabilities define how powerful AI systems are. More capable AI have more domain specific expertise, PhD level agentic reasoning and logic skills, and tighter integrations with all digital systems. Capability research and engineering further expands the skillset of today’s frontier AI systems. Capability safety ensures AI avoids certain areas in chemistry, biology, or physics to prevent it being used by nation-state actors to build weapons. This corresponds to C1 and C2 in my previous summary. Alignment defines how AI’s “true” intentions align with our intentions and expectations. Some of the surface level alignment problems include whether models refuse harmful queries and are pleasant conversational assistants. Some of the deeper problems are whether models have any inner goals, drives, or motivations. The more “meta” problems are something like - why does a model produce an output, is it because it believes it’s right, or it believes the user thinks it’s right, or it believes such answer will please the user. Alignment research basically tries to address all the questions above by better understanding the model, having it think out loud, implementing various middlewares and tools to allow us to see through the model’s thought process better, to make sure certain powerful but controversial capabilities can be safely deployed without creating too much risk. This corresponds to C3 and C4. So, What’s Next? I started this over a summer break trying to catch up on the AI Safety debate. I’ll be honest: I came out more unsettled than I went in - not because AI is evil, but because the easy risks are the technical ones, and those are the ones we’re best at solving. The ones that scale with power - bad human intent, ambiguous intent, misaligned models - are the ones our institutions are least prepared for. I don’t have the answer - we need multiple levers to remediate. But I think the first step is more of us thinking about it seriously. On the flipside, I’m a bit happy that it’s not going to be as simple as - AI comes out, engineering is dead, and so are many other industries. Instead, I see the need for more people going into not only capability engineering and frontier alignment research, but also law, policies, international relations, and social science to better steer the AI roadmap towards prosperity and avoid “total civilization collapse”.