AI Safety Ecosystem Research notes

A researcher mapping the AI safety ecosystem for MATS Research discovered unexpected organizations, including the Human Line Project, which collects stories of AI psychosis, and Impact Academy, which helps professionals transition into AI safety careers. The findings highlight the broadening scope of AI safety beyond traditional existential risk concerns.

These are some personal notes taken and later dressed up a bit to make into a post. Dunno how much value is here for people already familiar with the AI Safety Ecosystem. Over several weeks in the spring of 2026 I attempted to map out the entire AI Safety ecosystem as a project for MATS Research. This entailed finding every organization working on AI Safety whether it be via research, policy, pipeline, or other methods and determining or estimating their headcount and annual spending. It’s a snapshot, catching a period of time just before the incoming flood of 2026 funding. We worked almost entirely from public information. I believe MATS will be publishing the results in a formal report later this year. I will leave the unveiling of our findings to them. However while I was doing this research I stumbled across various orgs or tidbits that made me think “Huh, that’s interesting ” I’m not an insider here or at least, I’m a periphery insider if anything so many of these things may be old hat to people in the know. But they were interesting enough for me to jot down for my own notes, so I’m sharing them here. I was surprised by a number of things I found. MATS paid for the many hours that the collection of all this data took, and they own the resulting dataset. This post is not endorsed by MATS or representative of their views in any way. I worked as an independent contractor, and the following opinions are entirely my own. These are just thoughts that struck me while working, not things that are in the dataset or will impact the final report. So if an opinion below is bad, it’s on me. XD Human Line Project https://www.thehumanlineproject.org/ is an organization that collects stories of AI Psychosis. It keeps these in some sort of repository, shares some of them, and presumably does advocacy work. When I think “AI Safety” I think “stop the extinction or disempowerment of the human species” first, and “stop the disintegration of freedom and life as we know it” second. It didn’t occur to me that groups would form concerned about documenting every case of AI Psychosis to address mental health issues? But in retrospect this is exactly the sort of thing that they archetypal Suburban Soccer Mom would feel strongly about getting involved in. I know that sounds dismissive, and I don’t really mean it to be, because people’s concerns are important and everyone matters. But yeah, I guess it is a little dismissive. It seems on the far non-effective side of the effective interventions spectrum. But I could be wrong, maybe it’ll make all the difference at some key moment. At least I’m kinda embarrassed about feeling dismissive about it. Impact Academy https://www.impactacademy.org/ is entirely about helping high impact professionals transition from their current boring, non-AI Safety career into one that is with an AI Safety org or in the AI Safety ecosystem. I’ve mostly seen people transitioning from work in related fields primarily programmers , or going directly into AI as they enter the workforce, or moving into it from the rationalist community. IE: people who already were aware this was A Thing and had directly relevant skills or contacts. But as AI has become a full-blown industry, it now needs a lot of people and anyone who’s really smart and good at professional work can join. Getting someone who was a lawyer or actuary to pivot their career into AI Safety is a great idea and a serious lift. I’m really glad there are people making it their job to find folks who are smart and competent but total outsiders and bring them into the project. Impact Academy is not the only such group, but it was the first one I found with this sort of flip-a-professional mission as I was going along. Apropos of nothing… there’s a Journal of Science Fiction https://publish.lib.umd.edu/index.php/scifi ? And they want first-publishing rights while offering nothing in return? No wonder I’ve never heard of them. Still, it’s kinda neat that people can always find new rarefied environments to compete in. : Nick Fitz - listed in a high position at Lucid Computing, Seldon Labs, Andon Labs, Workshop Labs. There’s several people like this, Nick was the first one that caught my eye because his name POPS in my attention. Nick Fitz I didn’t notice the second time I saw the name, but by the third I was like “waitaminnit…..” and did some back-searching. He’s not the only such person in the ecosystem. Upon previewing this post Ryan informed me this is because he’s been a central funder of many safety orgs for a long time and other repeated names were similarly frequent funders which often led to them having leadership or advisory roles in orgs they funded. No funny business or anything, but it did highlight to me how tricky it is to get a true headcount of a full ecosystem… some people can end up counted multiple times. of course adjusting for contractors is equally hard. eg - I’m not counted anywhere in the AI Safety ecosystem. : Windfall Trust https://windfalltrust.org/ - An org dedicated to economic research & policy to ensure AI “benefits reach everyone, not just the few.” Personally I distrust that wording 1 , but it’s neat that there’s people gaming out economic futures and how they can be responded to with policy. Like my friend John Bennett says, the way to change the future is to There are a million sites that are someone who got excited one weekend and started something and then never did anything further. They look a lot like Legal Safety Lab https://legalsafetylab.org/ with a statement about how important the problem is, a call for action + contact info, and then nothing. I’ve been told this one may actually be doing some things quietly, which means this example will age badly, but there’s many like it. Compare to The Alliance for Secure AI Action https://secureainow.org/ . I have no clue who they are, and on the face of it they seem to basically be a fancier version of the previous site: call to action, taking people’s email addresses, republishing news articles. Maybe write about how urgent the problem is. But they keep at it. There are more opinion pieces regularly, and the news articles are recent. They have a staff of nine, and they launched a “six-figure advertising campaign” in 2025. I’m curious how this happened, because I don’t understand the value proposition of this group. Where’s that money come from? Will they become something bigger? I guess good for them for keeping at it and getting a snowball rolling. Buddhism & AI Initiative is here to provide Buddhist contemplative insights informing AI development & ethics. They have a hell of an online resources page https://www.engagedbuddhists.ai/field-framework . And their substack https://buddhismai.substack.com/ is fire. The four members seem to be serious, legit professionals. They have seed funding, likely in the six-digits. But all in the service of… Buddhist contemplative insights? What?? Do we need an org for Astrology Insights to AI, and Southern Baptists Insights to AI and all the rest? I realize people’s religion is important to them, and there’s enough buddhists in the AI world that they’d be happy to fund how their take on existence meshes with AI. I guess it’s as legit as the DEI-AI advocacy groups, and less harmful. /shrug I ran into a few similar sites and started a mental category of “unhinged AI mysticism” for sites like the Cyborgism wiki https://cyborgism.wiki/ . It’s not Time Cube https://www.timecube.net/ , but it gives a mysticism-amateur like myself Time Cube vibes. Boy will I be embarrassed if they turn out to be tapped into the true nature of… something. DeepSeek https://deepseek.com/ and xAI https://x.ai/ basically don’t have safety teams. I was legit surprised by the poverty of their safety efforts. I love that Musk is doing amazing things for humanity and unlocking space for us… but for such major players to be doing this little is wildly irresponsible. While we’re on the topic of massive efforts by the leading lights of the industry, Ilya Sutskever’s Safe Superintelligence https://ssi.inc/ is extremely secretive. With $3B raised in 2 years and an estimated 50 employees their entire public output appears to be a 217 word manifesto and an link to apply. I didn’t believe this was their actual online presence at first, I thought someone was domain squatting and I had to confirm through two different channels. My eyebrows are thoroughly hoisted. 2 https://www.lesswrong.com/feed.xml fnfhk4goxlj6 Woah, check out this EA Forums post on pursuing a career in AI Alignment https://forum.effectivealtruism.org/posts/7WXPkpqKGKewAymJf/how-to-pursue-a-career-in-technical-ai-alignment from mid 2022 . Holy shit, what a blast from the past. I imagine the people who followed this back then are some of the most impactful people in the world now. They jumped in on the groundfloor, when everyone else was thinking “wtf even is this? There’s no jobs in this. What would I do? How do I learn anything?” A 20-yr-old with an interest in this and no relevant technical skills could now be one of the foundational people in AI alignment, and making comfortable six-figures. That whole thing about OpenAI being the largest theft in human history https://thezvi.substack.com/p/openai-moves-to-complete-potentially ? OpenAI may be the largest, but the model isn’t unique to them, Ought https://ought.org/ did it two years earlier. Ought started out as a 501c3 non-profit doing AI Alignment Research, with a mission to “scale up good reasoning.” Ought spun off Elicit as an independent public benefit corporation in September 2023, selling its IP and transferring most of its staff to the new entity. Elicit raised a $22 million Series A at a $100 million valuation in early 2025. It’s a for-profit built on research that a non-profit’s donors paid for. I wonder how those donors feel about that. It’s funny how much bubbles don’t overlap. I hadn’t heard of Americans for Responsible Innovation https://ari.us/ , but they appear to be pretty dang big in DC. Funded by several big orgs including Coefficient Giving, they’ve got 34 employees, likely $20M+ annually, and were founded by a former Congressman. They crushed Ted Cruz’s AI initiative. They are bipartisan. I guess I’m just completely disregarding the policy side of things, which does sound like me tbh. Guys, Chinese AI researchers are bootstrapping coordination, and I think they’re doing it better than us. Regarding CnAISDA https://cnaisi.cn/ - “an internationally connected group of policy entrepreneurs within China developed a body that could engage in global AI governance conversations on frontier AI risks while fitting within China’s domestic political context.” It provides a formal platform for influential experts by forming a coalition “of multiple existing Chinese AI-focused institutions to represent China abroad, as well as to advise the government. This avoids the need for the government to “pick winners” in China’s policy ecosystem.” It’s “credibly claimed that the organization has the Chinese government’s support” despite not being a formal organization, and instead being a network for orgs and experts. all from Carnegie https://carnegieendowment.org/research/2025/06/how-some-of-chinas-top-ai-thinkers-built-their-own-ai-safety-institute It has no staff and almost no budget but works anyway. Note that I wrote the previous paragraph before Anthropic announced Project Glasswing. I’m now a little less surprised about private entities coordinating without government involvement to solve civilization-level problems. It’s interesting how much our upbringing molds our pre-verbal expectations. Despite having become deeply disenchanted with government and swinging hard into centralist liberal, I was really surprised by both CnAISDA and Glasswing. My core psychology was raised on leftist doctrine and my instincts still believed only authoritarian governments could do that sort of thing. I’m happy to see my intuitions proven false. Not double-counting is hard, part two: Berkeley Existential Risk Initiative https://www.existence.org/collaborations-tais collaboration-tabs “is supporting Coefficient Giving in administering grants” and “manages research support expenses for 26 participating research groups” and “also provides living stipends to some researchers.” But they work on a lot of existential risk projects that are not AI. Deciding how much of the org’s budget should be counted towards AI Safety is tricky. Similarly, as AI Safety becomes a regular, mainstream concern, it’s harder to disentangle the resources that go just into AI Safety when an org is concerned about humanity as a whole. eg - Non-Trivial https://www.non-trivial.org/ is here to support young researchers getting into impactful careers. “Join the brightest young researchers tackling the world’s most pressing problems. Online incubator with mentorship, up to $10,000 scholarships, and zero required fees.” Just straight up doing good with a broad brush, helping get people started on a wide variety of important research. Biosecurity, climate, global health. AI risk is included, and estimates are 10-25% of their funding and efforts goes to people working on that. How do I count the people working at this org, and the costs associated with running it? Certainly a fraction of their efforts are directly working on/funding AI safety, but it’s weird to say the AI Safety Field includes 20% of an org that is actually a high-impact interventions fellowship/scholarship program. This is going to become far more common over the next years. There’s a number of groups working on AI Safety hardware, which it hadn’t occurred to me is a part of this. The govt regulates compute in theory, but how does one do that in practice? Groups like FlexHEG Flexible Hardware-Enabled Guarantees https://flexheg.com/ are trying to make AI-chip governance technically enforceable and auditable on the hardware level. The idea is to build hardware/software mechanisms into AI-relevant chips or accelerators so that folks can make verifiable, privacy-preserving claims about how powerful AI compute is being used. The Center for Shared AI Prosperity https://www.csaip.org/ top - these guys are working on directly moving the ball forward on redistribution from AI to humans. Currently paying for strong candidates to pitch their policy ideas https://www.csaip.org/rfi which including a realistic proposal and bio+CV of team members. I’m happy there’s someone actually working on this rather than everyone just saying “Yeah, the future govt will fix it by implementing UBI or something.” it triggers “these are the same words used by the confiscation gangs” vibes for when raising just isn’t enough