# A genealogy of AI safety: how directions are born, and how they die (2005-2026) > Source: > Published: 2026-07-10 00:50:21+00:00 Ask how AI safety reached this point, and most people give you a neat story. A small group of philosophers raised the first concerns about superintelligence. The arrival of large language models changed that, and many researchers turned to the alignment problem. The story is too tidy, and it leaves out the interesting part. Examine timelines, contributors, funding sources and publications, and it falls apart. New directions appear. A few merge into the mainstream, others fade out, and money and research attention rarely align; one tends to follow the other by years. I wanted to study that pattern rather than declare it, so I built the entire project from the ground up. Below is a table of 323 documented events, 129 actors and 18 directions. Its timeline runs from 2005 through June 2026, and the final year covers only six months. Next to it, a separate set of arXiv attention proxies covers 23 of those directions. I also built a deduplicated safety corpus from a single combined query, so no paper is counted twice. Each row is one verifiable fact backed by a primary source, tagged with year, actor, direction, type, amount, source, and my confidence. Every chart below draws on that table, and the path was never straight. Sort the events by year, funding and attention, and you see fresh strands open up, a handful fold into the mainstream, several fall quiet, while funding and research stay out of step. The charts below are all generated from this base. First, **five caveats** that the rest depends on: *Each dot is an event. The organisation axis is sorted by year of appearance, so the dots form a diagonal from the bottom-left up to the top-right. The track colour is a chronological gradient by birth year, so the field's colour drifts over time at a glance. The bottom panel stacks the events by track per year in the same colours, and the activity shifts from early macrostrategy and agent foundations toward later interpretability, evaluations and AI control. One caveat: that panel counts events, so it shows collection density rather than real-world activity. The eras are hatched and stars mark the milestone papers. ( interactive).* The timeline is sorted by the **start** of a track, but it doesn't reveal who survived to today. For that there's a separate lifespan chart: a line from birth to the last recorded event, tracks ordered by birth year. One note here: a missing late event is not a dead track (again collection density, per the caveat above). So the chart carries an **independent signal, a diamond on the last year a track still publishes on arXiv**, wherever a curve exists. A track can fall silent in the event log and still be alive. *The line runs from a direction's first recorded event to its last, and the diamond marks the last year it still publishes on arXiv, an independent proxy across 23 directions. The two often diverge. Reward modeling's last event is 2017, yet its publications, now under the RLHF banner, run to 2025. Value learning went quiet as discrete events by 2021, and agent foundations thins to a lone 2024 marker with MIRI's pivot. Both stay alive on arXiv. So a track rarely just dies. More often it gets absorbed, or slips into the background. There is more on that in the Cross-cutting patterns chapter, after the eras. ( interactive).* Here is another look at tracks over time, now counting the number of *events* in the base per year: *The stacked area shows how many events of each direction landed in the base each year. A reminder of caveat 2: the height is collection density, not real work volume, and the late years (2026 is only the first half) are hatched as incomplete. So read the shape and the order of appearance rather than the absolute values. The field sits almost empty until 2012 and 2013. It then explodes in 2015 to 2017 with the first technical tracks. And from 2021 to 2024 it keeps growing as new directions such as evaluations, AI control and model organisms pile on top of the older ones. ( interactive).* One more overview map pulls the whole route together, tracing how the flows run from era to organisation to direction. *( interactive).* At first glance it is just spaghetti. The middle column holds over a hundred organisations, about 115 of them, the ribbons cross, and reading it all at once is impossible. It is also unnecessary. So let us roll it up into big blocks: the roughly 115 organisations into **seven actor types** (researchers, funders, policy institutes, talent training, industry, meta and community, and government bodies), and the 17 directions into **four families**: The same picture comes next in **two cross-sections** that answer different questions: one shows the route as a whole, the other walks it step by step. **Cross-section one, the overall map.** Three axes run left to right, from **era** (the when) to **actor type** (the who) to **direction family** (the on-what). The ribbon colour shows at once which family dominated and how that shifted. *The colour shows the direction family, and the numbers count non-money events, so they measure how much was collected rather than the field's real output (caveat 2). A diagonal jumps out. The blue foundations and strategy ribbons crowd the left and the early eras, while the orange technical safety gains mass toward the right. ( interactive).* **Cross-section two, by era.** It is the same flow, with one mini-panel per era. The **center of gravity shifts** step by step across the panels, and one family dominates each period. *Each panel is one era, running from actor type to direction family, and the ribbon width is the number of events. The blue foundations and strategy rules from 2005 to 2016. Then from the 2020s the orange technical safety dominates the panels. ( interactive).* There is also a third cross-section: two clean two-level flows side by side, splitting the *when* and the *who*. *On the left the flow runs from era to direction family, showing how the field's focus shifted over time. On the right it runs from actor type to family, showing who works on which family. The ribbon width is again the count of non-money events, which reflects collection effort rather than the field's true volume (caveat 2). On the left, the orange technical safety gains mass toward the later eras. On the right, researchers and industry pull mostly toward technical safety, while government bodies and policy institutes pull toward governance and infrastructure. ( interactive).* **The era-blocks show where the field's center of gravity moves.** The number beside each is how many non-money events landed in that era (collection density, not world volume, see caveat 2): This migration is the storyline of the whole post: from the bottom-left corner (philosophy, lone institutes) to the top-right (empirics, attention and money concentrating around Anthropic and the safety institutes). But concentrated attention doesn't mean concentrated organisations. In this last era the field **proliferates into dozens of narrow organisations** (2024 was a record founding year in the base). The sections below take that apart era by era. The field's activity composition shifted too, meaning what the eras are even made of. The prehistory era (2005-2012) is **100% organisation foundings**: no papers, no grants yet. Publications, grants and funding statements enter only from 2013 on. By the last era foundings are just about 28% of a much larger, more varied mix. *This shows the shares of the event types, such as foundings, publications, grants and statements, within each era. Caveat 2 still holds: this is the composition of the collection, not the composition of the world. ( interactive).* Finally, here's how all of it changed **across the eras in one frame**: four comparable panels where five epochs run horizontally (the same five substantive Parts below). *The panels run left to right and top to bottom. The first panel shows the direction families as a share of each era's events, out of 100%: the blue foundations and strategy starts at 67% in the 2005-2013 era and shrinks to 6% by 2024-2026, while the orange technical safety climbs to about 60%. Above the columns sits the total number of events in the era, so that normalizing does not hide the field's roughly tenfold growth. The second panel shows the actor types the same way, across seven types. Early on it is almost all researchers, at 56% in the 2005-2013 era. Funders enter as a distinct force in 2014 to 2019, at 24%. And by 2024 to 2026 the mix is transformed: government arrives from nothing to 24% and industry to 26%, now rivalling researchers at 29%. The state and the labs, not just philanthropy, drive the last era's recorded activity. The third panel shows the money by fund, the absolute amounts of all itemized grants per era, roughly $764 million all told. The six long-standing philanthropic funders (OpenPhil, SFF, FTX, FLI, LTFF and Tallinn) keep their own colours, while the newer government and international money (UK AISI at $159 million, ARIA at $74 million, Canada at $36.5 million, NSF at $20 million, Australia at $19.7 million, and so on) rolls into one grey band of other, government and new money. So the growth in money runs consistently end to end: about $0 at the start, then about $128 million in the 2014-2019 era led by philanthropy, then about $242 million in 2022 and 2023 and about $335 million in 2024 to 2026, where the grey government band dominates. The fourth panel shows the organisation dynamics, with births rising and closures and pivots falling. A couple of caveats: the first two panels are shares of collection events (caveat 2), not world volume, and the money panel is itemized grants only, with donor totals and pledges excluded, so that nothing is counted twice against the cumulative-funding chart. (* The birth and fade of individual directions already show up above on the direction-lifespan chart, and the cross-cutting over-all-years money and attention charts sit together in the Cross-cutting patterns chapter, which comes after the epochs. Now for the same five epochs, in order and in words. *This era holds 9 events in the base, and the birth and fade of its directions runs along the direction-lifespan chart above.* It doesn't start with neural networks. It starts with a philosophical question: **how do we avoid wrecking humanity's distant future?** In 2005 the Future of Humanity Institute (FHI, Nick Bostrom) opens at Oxford, the earliest reference point in the base. Around it forms what this post calls **macrostrategy**: the broadest philosophical reasoning about global risks. Over the next years GCRI (2011), CSER (Cambridge, 2013), FRI (2013) and CFI (2015) join in. By event density, macrostrategy peaks in the mid-2010s. The second root is mathematical. In 2013 MIRI pivots from public outreach to friendly AI as formal mathematics (decision theory, logical induction). That is the birth of **agent foundations**, the attempt to understand an idealized agent *before* anyone builds it. At this stage the table holds almost no money. There's a question and there are first researchers, but no industry and no grant flows. The early nodes (FHI, GCRI, FRI) have already run for years, and none of them has died yet. This shows on the organisation-lifespan diagram in the Cross-cutting patterns chapter, after the epochs. *In this era the money panel fills for the first time, and OpenPhil dominates immediately.* The turning point is January 2015: the FLI conference in Puerto Rico ("Future of AI"), which Nate Soares would later call exactly that. It produces an open letter and, with it, the field's first big donor: **Open Philanthropy**. On cumulative money by fund (the field-wide summary money charts are in the Cross-cutting patterns chapter) the picture is unambiguous. **OpenPhil dominates the whole field's funding.** By annual donor totals, its investment into technical safety rises from $1.19 million in 2015 to $6.56 million in 2016, then to **$43.2 million in 2017**, up to a peak of **$81.7 million in 2021**. But that 2017 jump is mostly a single grant, the **$30 million for general support of OpenAI** in March 2017. A spending peak can be one big bet rather than a lot of research. These same years also give birth to **governance**: policy, institutes, AI governance. In 2019 OpenPhil puts **$55 million over five years** into founding CSET (Georgetown), the largest governance grant of that epoch. Value learning runs in parallel, with Stuart Russell's center CHAI (Berkeley, 2016), which OpenPhil immediately backs with **$5.6 million over two years**. Where the money actually went, by direction and year, shows up most clearly through itemized grants, without the aggregate donor totals, which would count the same dollars twice. The detailed money-by-track breakdowns live in the Cross-cutting patterns chapter. One figure stands out already: field-building (infrastructure like funds and fellowships) accounts for **$326.1 million by itemized grants, the field's largest money flow**. That figure grew after backfill, once field-building came to include large government programs like the UK AISI taskforce and new regranters. Next to it the other directions almost disappear. *In this era the families' center of gravity shifts noticeably into technical safety.* By 2020 it's become clear that strong AI, if it arrives, will come from **large neural networks** rather than from pure agent theory. So the field turns to prosaic alignment, working with real models. Three shifts land at once: The shifting money leader year to year, and the drift of shares from early technical bets toward governance and infrastructure, both show up on the cross-cutting money chart (absolute money by track). It sits in the Cross-cutting patterns chapter, so it reads across the whole field rather than one era at a time. The publication attention curves (arXiv proxy) by track sit there too, and the interpretability curve on them starts right around 2020-2021. *In this era the money now comes from several funds at once (SFF, OpenPhil, FTX), and among the actors, institutes and industry noticeably grow.* In late 2022 ChatGPT pushed AI into big politics and mass consciousness. In the table that surfaces as a surge on several fronts at once: Then the shock. **November 2022, FTX collapses.** In half a year (February-August 2022) the FTX Future Fund had handed out **$18.7 million** by name to AI safety, about $32 million by estimate overall. Now it's cut off, and some grants even have to be clawed back. On the cross-cutting cumulative chart the FTX line stops at 2022 and never grows again. The births and closures of organisations by year sit on the summary organisation-dynamics chart there too, and in the organisation-dynamics panel of the by-era overview chart. *In this era, consolidation looks more like continued activity, with the organisation dynamics showing a record 31 births against 8 closures or pivots.* The last era in the data is contraction and redefinition. But this consolidation is really specialization and proliferation. 2024 is the record year in the base for organisations founded (**24 founded-events**). Interpretability gets commercialized and splinters into startups (** Goodfire**, **Transluce**, **Guide Labs**, **Tilde**, **Simplex**, **Decode Research**). A separate AI-security and evaluations cluster emerges, in **Palisade Research**, **Gray Swan AI** and **Virtue AI**. Organisations for fresh sub-agendas keep appearing: **Geodesic** for chain-of-thought monitoring, **Luthien** for practical AI control, **Softmax** for multi-agent alignment, **Formation Research** for lock-in risk, and Yoshua Bengio's **LawZero** in June 2025, built around a non-agentic "Scientist AI" as an oversight layer. Even the giants open internal cells: in 2025 **Meta** sets up a "superintelligence alignment and safety" team. Alongside all of it, **non-technical infrastructure** proliferates too: the forecasting cell **AI Futures Project** (Kokotajlo, the "AI 2027" scenario), the watchdog **AI Lab Watch**, the standardizers **AI Standards Labs**, national centers (** Beijing Institute of AI Safety and Governance**, France's **CeSIA**), the international association **IASEAI**, and the **AI Whistleblower Initiative**. **Governance stops being background and becomes the era's main storyline.** It accounts for a third of the block's events (26 of 82 non-money), and the chronology tightens from summit to summit. **November 2023, Bletchley.** Then the **AI Seoul Summit** (May 2024): 16 frontier companies sign the **Frontier AI Safety Commitments**, each promising to publish its own safety framework with risk thresholds, and 27 countries plus the EU, in the **Seoul Ministerial**, agree for the first time to develop common thresholds for severe risks. **1 August 2024, the EU AI Act comes into force**, the world's first broad, horizontal AI law, with rules for GPAI from August 2025 and a **GPAI Code of Practice**. In the US, hard law stalls. California's **SB 1047** (tests, a kill switch, liability) is **vetoed** by Newsom (September 2024). Then in January 2025 the federal course swings from safety to dominance: the repeal of Biden's AI executive order, the "Removing Barriers to American Leadership in AI" order. Later in 2025 California comes back with a softer **SB 53**, focused on transparency. The industry builds its own: the **Frontier Model Forum** in 2023 and its **AI Safety Fund**, worth over $10 million in rounds of $4 million in November 2024 and over $5.2 million in December 2025. And an IPCC for AI takes shape as the **International AI Safety Report**, which ran from an interim version in May 2024 to a full report in January 2025 under Bengio, landing just before the **AI Action Summit** in Paris in February 2025, where the tone has already slid from safety toward action and deployment. The technical directions don't stand still. Interpretability gains new momentum: Scaling Monosemanticity (Anthropic, May 2024, "Golden Gate Claude"), **Gemma Scope** (DeepMind, July 2024), an open set of SAEs shipped as public infrastructure, circuit tracing and "On the Biology of a Large Language Model" (March 2025), and **"Auditing Language Models for Hidden Objectives"** (Anthropic, March 2025), the first real audit game, where a goal is hidden in a model and then uncovered blind. The youngest direction, **model organisms**, forms around the empirics of deception: Sleeper Agents (January 2024), **Sabotage Evaluations** (Anthropic, October 2024), Apollo's **"In-Context Scheming"** (December 2024, where scheming showed up in 5 of 6 frontier models), Alignment Faking (December 2024), and **Agentic Misalignment** (Anthropic, June 2025, where 16 models in stress-tests resorted to blackmail or leaks to avoid shutdown). Safe-scaling frameworks evolved into their own genre. DeepMind published the Frontier Safety Framework in May 2024, and Anthropic revised its Responsible Scaling Policy that October. Scalable oversight had lost its home when the Superalignment program was cut, so researchers rebuilt its goals in other projects. In December 2024 OpenAI introduced Deliberative Alignment. The following September, it teamed up with Apollo to release anti-scheming training that reduced hidden actions by about 30-fold but never eliminated them. That work spurred agentic control evaluations, among them Redwood's Ctrl-Z in April 2025. Everything culminated in July 2025, when OpenAI, DeepMind, Anthropic and the UK AISI co-signed the position paper "Chain of Thought Monitorability", which outlined their collective responsibility to protect the limited window into model reasoning. The first six months of 2026 followed similar patterns, and the dataset includes only data through H1. Having walked the field epoch by epoch, it is worth stepping back to look across all the years at once, because some patterns surface only that way and slicing them by era hides them. Money is the most tangled of these, so it comes first. Four money views, and why the same fund shows different amounts.The money here gets counted fourdifferentways, which deliberately do not add up; otherwise the same dollars would land in the total twice. They are four views of one pile.The first is the itemized grants, at $763.6 million. These are the real targeted grants, recorded as they went out. They show in full on the money-by-track chart, where the bars are the tracks plus a grey other-and-untracked strip of $83.6 million, and on both sankeys. Only the track-classified $680.0 millionappears on the radar and the per-track money-versus-attention chart, because the aggregate donor baskets and one capabilities grant, $83.6 million together, have no specific track. The second view is the donor annual totals. Each is a donor's full budget for the year, the umbrella figure rather than the sum of its sub-category breakdowns, which would tally the same money twice. This is howOpenPhil (about $304.5 million)and LTFF (about $3.64 million) show up on the cumulative-money-by-fund chart, while every other fund shows by its itemized grants. The same two totals run to 2024 on the field-wide money-versus-attention chart; the 2025-and-later donor totals are not published yet, but the individual-grant view keeps flowing past that point.The third view is the venture money, the VC and equity, at $268.5 million. It expects a return, arrives across five rounds, appears only on the third panel of the cumulative-money chart, and is never summed with the first two. The fourth is the pledges, budgets and estimates, kept as context and never summed. This is money that is nota disbursed grant: FTX's launch pledge of about $160 million against the $18.7 million it actually disbursed, organisations' annual budgets like MIRI, CHAI and CSER, and overlapping field-wide estimates such as the roughly $40 million EA spent in 2019. It was invisible on every chart until it got its own dots-only view. Because the field-wide estimates already include the organisation budgets, it isheterogeneous and non-additive, so it is plotted as points and never totalled, purely so that nothing collected stays hidden.Here is where the confusion comes from. OpenPhil is $176.77 millionas the sum of its itemized grants under the first view, andabout $304.5 millionas the donor annual total under the second. It is the same organisation seen two ways. No figure is lost; the two simply cannot be added together. **Money across the whole field.** The historical main funder was OpenPhil, with $176.77 million of itemized grants, and after backfill it got caught and overtaken by **the government money of half a dozen countries**. The UK AISI and its taskforce come to about $159 million altogether (£100 million plus the Alignment Project), with ARIA Safeguarded AI at $74 million close behind. Then a longer tail: Canada CAISI at $36.5 million, NSF Safe Learning-Enabled Systems at $20 million, Australia AISI at $19.7 million, the US AISI at NIST at $10 million, DARPA GARD at $10 million, and the EU AI Office at $9.8 million. SFF stands alongside at $144 million. FTX still cuts off at the end of 2022. So the one-big-funder picture of 2020 to 2022 gives way to something far more distributed in 2023 to 2025, and noticeably more governmental, now international. But this is already *different types of money* (caveat 5). And beside it a fourth type has appeared, **venture capital** into safety startups, about $268 million of equity. It stays a **separate view** with its own colour and line across the charts below, never mixed into the grant totals. The grant money appears three ways, ranked, cumulative over time, and as a dot-strip, because no single view catches both who gave how much and when the money arrived. *There is one bar per funder, sorted, on a log axis, and coloured by type. The philanthropy bars are blue: Open Philanthropy sits at its donor annual total of**$304.5 million** (the same OpenPhil that reads $176.77 million under the itemized-grant view; caveat 5), then SFF at $144 million, FLI at $39 million and FTX at $18.7 million disbursed, followed by Schmidt, Longview, Manifund, Tallinn, LTFF, Founders Pledge and Effektiv Spenden. The *The same money over time, now stacked by funder type, showing not only who but when.* *The stack is cumulative grant dollars by type, with philanthropy in blue, government in orange and corporate in red. OpenPhil and LTFF sit at their donor annual totals and the rest are itemized, which is why the canonical stack runs a little above the $763.6 million itemized-grant total. The shape is the point here: through 2020 it is almost entirely a single blue channel of philanthropy. Then from 2023 the orange government band explodes, with half a dozen national safety institutes landing at once, turning one fragile channel into a broad, increasingly governmental and international flow. The dotted green line is the VC and equity money, about $268 million, a separate view laid over the same axis and never summed into the grant stack. (* *Third, a dot-strip, since the shapes above hide the rhythm of individual moves.* *The x-axis is the year and the y-axis is the funder, sorted by total, while the dot area shows that year's dollars and the colour shows the type. OpenPhil is the only funder active almost every year, and its 2021-2023 dots are the largest. The government cluster is unmistakable in 2023 to 2025, in big orange dots for the UK AISI, ARIA, Canada CAISI, NSF and Australia AISI. And VC rounds show up as green diamonds from 2023 onward, with Protect AI in 2024 the largest. The timing makes the regime change legible: philanthropy carried the field's money alone until 2023, and then the state and the market arrived together across the last three years. (* A fourth view keeps any collected dollar from staying invisible: the money that is **not** a disbursed grant. *Ranked horizontal bars group the sub-types on a log axis. What matters here is the magnitude, because these are not comparable, additive dollars. The colour shows the sub-type. The launch pledges include the FTX Future Fund's pledge of about $160 million at launch. The organisations' annual budgets include FTX's own $50 million and $32 million figures and MIRI at about $7.5 million a year, along with CHAI, CSER, Ought and Lightcone. The field-wide estimates include the estimate of roughly $40 million spent on AI safety in 2019 plus the 2014-2016 estimates, which deliberately overlap the organisation budgets. And the seed at founding includes SFF at about $2 million and Timaeus at $0.14 million. This view is heterogeneous and non-additive. The field estimates already contain the organisation budgets, and a pledge is not a disbursement. So it is never summed and never mixed with the grants ($763.6 million), the donor totals, or the VC money ($268.5 million). Its only job is to show that these numbers exist in the data. (* *That gap shows up most sharply in one case, which gets its own chart.* *Here is the headline as a dumbbell: the FTX Future Fund pledged about $160 million at launch but actually disbursed only $18.7 million before its collapse in November 2022. That gap between what was announced and what was actually delivered is the clearest warning of the era. Below it, the rest of the non-grant money, the organisation budgets, the overlapping field estimates and the seeds, is ranked on the same log axis for scale. It is still the fourth view, so never summed with grants, donor totals, or VC. (* And now for where this money actually flows, the routing of itemized grants from fund to track. *The flow here is a Sankey of the known itemized grants, with funds on the left, research tracks on the right, and the ribbon width showing the grant amount. OpenPhil and SFF clearly give across many tracks at once, while the government institutes (UK AISI, Canada CAISI, Australia AISI, US AISI at NIST, EU AI Office, DARPA and NSF) enter with precision, mostly into field-building, evaluations and robustness. This routes all the itemized grants, some $764 million altogether: grants whose direction is an aggregate donor-basket (the mixed-technical or technical-safety baskets) or capabilities get their own right-hand nodes rather than being dropped. Only donor totals and the VC and equity money stay out, because otherwise those dollars would be counted twice. (* The same grant flow, but now with a third level, the era, added on the left. That way it shows who, where and when at the same time. *This has three levels, running from era to fund to track. It reads clearly how the mass of grants shifted into the consolidation era of 2024 to 2026, and exactly how the new government institutes support evaluations, field-building and robustness. The recipient organisation is free text in the detail field, not a structural column, so it does not get its own level, a gap in the data. ( interactive).* **Money versus attention, and the life of organisations.** Money, as donor totals, and scientific attention, as the arXiv proxy, keep drifting apart over time. And the births and closures of organisations give a separate organisation-dynamics curve across the whole field. *Four things are drawn here, and it is worth reading them separately.* *The grey band marks the partial final year, the first half only, and the right axis is logarithmic. Both money and attention are proxies, but the deduplicated line is reliable at the field level. ( interactive).* *Up is new organisations, down is closures and focus changes. One thing to keep in mind: this counts events per year, not organisations. There are 14 closure or pivot events in all, 3 closures and 11 pivots, and a single organisation can change focus several times. MIRI pivoted in 2013, 2018 and 2024; OpenAI pivoted in 2019 and then closed its Superalignment team in 2024; and Open Philanthropy pivoted in 2024 and 2025. So there are more red events in total than distinct dead organisations, which is why there is more red here than there are crosses on the chart below. (* *Each bar is one organisation, unlike the chart above, which counts events: it runs from the founding year to the last closure or focus change, or to the present if still alive, with a cross marking the organisation's last such event. The founding dates of MIRI (2005, when SIAI reoriented to AI risk), Open Philanthropy (2014), CFAR (2012), LTFF (2017), METR (2022, as ARC Evals) and the FTX Future Fund (2022) are added from sources. So now every organisation with a closure or pivot is visible, and the chart converges with the one above. The 14 red events there collapse to 10 distinct organisations here, each carrying one cross on its last such event, because MIRI, OpenAI and Open Philanthropy each pivoted more than once. Pivots happen across the whole timeline, with MIRI's turn coming as early as 2013 and CFAR's in 2016. The actual closures, just 3 of the 14 events, land in 2024, at FHI and OpenAI's Superalignment team, with the FTX Future Fund's 2022 collapse as the earlier exception. (* **A track's three fates.** The table's biggest lesson is that a direction here can meet **three different fates**, and disappearing from the conversation is not the same as dying. **Rising: interpretability, evaluations and AI control**, all born recently and growing. Interpretability is the reference case. By the **denoised** arXiv proxy, after the bare word *interpretability* was stripped from the query, where it had caught all of general ML, the curve is far more modest, rising from **69 in 2021 to 81, then 125, then 257, then 657 by 2025**. The shape holds, a steady exponential rise, and it is that shape, not the absolute count, that carries the point. The same climb shows in the field-wide deduplicated corpus, which runs from **351 in 2021 to 427, then 890, then 2191, then 3813 by 2025** unique safety papers per year, almost an order of magnitude in four years. Money, meanwhile, *lags sharply* behind attention, with only $1.04 million of itemized grants into interpretability, the classic case of attention outpacing its funding. For **evaluations** the gap closed almost overnight. Before 2024 there was no itemized money on the track at all. Then it filled to **about $96.7 million in itemized grants, nearly all of it in 2024 and 2025**, making evaluations one of the field's better-funded tracks in a single year. Some of that is dedicated philanthropic and US money: the **AI Safety Fund at the Frontier Model Forum gave $9.2 million** in two rounds, $4.0 million in 2024 plus $5.2 million in 2025, for biological, cyber and agentic evaluations; the **US AI Safety Institute at NIST gave $10 million** in 2024; and **Schmidt Sciences gave $10 million** in 2025 for the science of evaluations. But the bulk comes from the new **international government institutes** routing into evaluations: **Canada CAISI at $36.5 million**, **Australia AISI at $19.7 million** and the **EU AI Office at $9.8 million**, plus an OpenPhil grant of $1.5 million. In the per-track panels below, evaluations now shows up with both a money bar and an arXiv line, and on the gap chart it sits close to the field average instead of off the money-starved end. **Absorbed: reward modeling, now RLHF, and value learning.** RLHF by the arXiv proxy, on the broadened query for RLHF, reward models, DPO and preference optimization, runs from **25 in 2021 to 43, then 275, then 1010 in 2024, then 1777 in 2025**. The direction vanished from safety conversations because it had won. RLHF and DPO became the standard way to fine-tune every commercial LLM and dissolved into mainstream ML. Publications keep growing precisely because it is shared infrastructure now, not a separate safety track. **Faded: agent foundations and early macrostrategy.** They were the field's core, then shrank. Agent foundations barely registers on the arXiv proxy before 2024, since the term is young while the direction itself is old, the reverse case of proxy lag. The closure of FHI in 2024 is a symbolic full stop for macrostrategy. Money and attention for each track, laid out one small panel at a time so nothing overlaps: *The grid holds small multiples, one mini-panel per track, ordered by total grant money. In each panel the bars, on the left axis, are that track's itemized grant money per year in dollars, and the dotted line, on the logarithmic right axis, is its scientific attention on the arXiv proxy for the same years. So each track is legible on its own and its divergence between money and attention reads directly. This replaces the old single dual-axis chart, where about 20 attention lines piled over the stacked bars into unreadable spaghetti. Here all the itemized grants show, about $764 million in total: grants without a specific track collect into one grey other, untracked or aggregate panel, about $83.6 million, made up of the aggregated donor baskets for mixed-technical and technical-safety plus one non-safety grant, money only, with no arXiv line, so no dollar is lost and the grid still totals the full $763.6 million. Donor annual totals for OpenPhil and LTFF, and the venture money in equity, live on the cumulative-funding chart and are not summed here, because that would count them twice. Interpretability shows a steep dotted rise over tiny bars, its attention far ahead of its money, even after denoising the curve by dropping the bare word interpretability. Evaluations shows a wall of bars for 2024 and 2025. AI control has money bars from 2021 before its attention lifts. Tracks that have an arXiv proxy but no itemized grant money at all (reward modeling, red-teaming, unlearning, chain-of-thought faithfulness, honesty and ELK, and the umbrella and nested directions, per caveat 1) get no panel here, since there is no grant bar to rank them by. Their attention instead sits on the field-level money-versus-attention chart and the per-track proxy curves. Both series here are only proxies, and the deduplicated safety corpus is deliberately not drawn on this grid, being field-level. (* *This diverging gap chart replaces the old log-log trajectory tangle. For every track with both a money series and an arXiv proxy, the bar shows how far it sits from the field-average rate of money to attention, computed as the base-2 logarithm of the actual attention divided by the field's average papers-per-dollar times that track's money. A blue bar pointing right means scientific attention runs ahead of money. A red bar pointing left means money runs ahead of attention. The label gives the factor. Interpretability is the extreme blue case, its attention roughly 36.9 times ahead of what its modest $1 million or so in grants would predict, the classic science-runs-ahead-of-money gap. On the red side sit scalable oversight, agent foundations and governance, directions where itemized money outran the arXiv attention. Both signals are proxies, per caveats 1 and 2. (* *A companion dumbbell view lays out the same tracks. Instead of the magnitude of the gap it shows the ordering: each track's rank by grant money, as an orange dot, against its rank by arXiv attention, as a green dot, joined by a connector, where rank 1 is the smallest and the largest rank is the biggest. A long blue connector means attention ranks the track far above where money does. A long red one means money ranks it far above attention. Overlapping dots mean the two agree. Set beside the gap chart above, which shows magnitude, it makes the who-leads story concrete: interpretability's dots pull wide apart, while better-matched tracks keep their two dots close, both ranks resting on proxies (caveats 1 and 2). (* And here is the money footprint of the tracks, on a single money axis, with a caveat. *A note on this one: it is a single logarithmic axis for money, not a multi-metric comparison. Multi-metric radars mislead across different scales, so there is no one built here. ( interactive).* There is no moral to end on; the data does not hand one over. What it does leave is a handful of open questions the field is currently standing on: **Data and sources.** The dataset includes 323 documented events from 2005 through mid-2026, plus a deduplicated safety corpus and arXiv proxies across 23 directions. Each count links to the exact search-query URL that reproduces it. We backfilled most of the recent detail through several passes over live primary sources, rather than retrospectives that stop at 2023. We identified newly founded organizations from an analysis of field growth on the EA Forum. The research milestones for 2024 and 2025 come from the "Shallow Review of Technical AI Safety", with its arXiv and lab links, and the governance events from primary sources: [gov.uk](http://gov.uk), EU documents, California legislation, and White House records. They brought in new philanthropic grantmakers (Manifund, Longview, Founders Pledge, Schmidt Sciences, Effektiv Spenden, Macroscopic Ventures and Nonlinear) and international government institutes (UK AISI and ARIA, US AISI at NIST, NSF, Canada CAISI, Australia AISI, EU AI Office, DARPA's GARD and AIxCC, IARPA's TrojAI, Singapore AISI, the annual UK AISI budget of about £50 million a year, Germany's DE-AISI and India AISI). They also added venture rounds into safety startups as a separate view, recorded as investments: Goodfire at $50 million, Gray Swan at $40 million, Lakera at $20 million, HiddenLayer at $50 million, and Protect AI at $108.5 million before its Palo Alto acquisition of $634.5 million. And they added the missing founding events for safety organisations (EleutherAI, Apart Research, Lakera, Protect AI) and the individual donor Craig Newmark. The last two enter as a statement and an investment so that grants do not double-count. In the final pass the arXiv method turned toward precision over breadth. A deduplicated safety corpus was added, one combined OR-query with each paper counted once. The two worst tracks were denoised: the bare word *interpretability* was dropped from the interpretability track, and truthfulness was flagged high-noise and pulled from the field sum. Every proxy track now carries a noise flag. Three collector scripts write the data with a source URL and rerun reproducibly: one pulls arXiv, one the LTFF annual payouts from EA Funds, and one the SFF annual totals. So 2024 comes out as the densest year of the base, partly a matter of collection density, which the direction-lifespan charts flag. Every event ties to a primary source, and the source types run as follows. **How the money is counted.** Itemized grants (a specific amount to a specific recipient in a specific year) are kept separate from donor annual totals (how much a fund allocated in total for a year). To avoid double-counting, each donor uses one canonical series: annual totals for OpenPhil and LTFF, itemized grants for everyone else. The key amounts from the text: **On 2026 (incomplete year, H1).** Real events of the first half of 2026 came from primary sources (Anthropic Natural Language Autoencoders, the UK AI Security Institute cyber evaluations, the EU Digital Omnibus). But there's no distributed money for 2026 yet. SFF-2026 is only announced at $20 to $40 million with distribution in the autumn, so the amount isn't recorded, and the AI Safety Fund round with a figure fell in December 2025. The arXiv series were rebuilt with one unified method across all 23 directions: the same reproducible recipe, OR-synonyms in the title and abstract plus a per-track category filter, counting by submission date, in a common 2015-2026 window, dropping the earlier mixed methodology. The per-track proxies gained the deduplicated safety corpus, one combined export with each paper counted once, which on the money-versus-attention chart is the solid line against the dotted inflated keyword sum. The two noisiest curves were denoised: interpretability without the bare word *interpretability*, where the count dropped severalfold, and truthfulness flagged as high-noise and removed from the field sum. 2026 stays incomplete, the first half only, and is marked with a grey zone. So an incomplete year does not read as a decline, and the last-full-year arXiv diamonds on the direction-lifespan chart sit on 2025. The rename from OpenPhil to Coefficient Giving is pinned to its real date, **November 2025**. **Five caveats.** First, arXiv counts are a proxy, not bibliometrics, so the per-track sums are inflated and cannot be naively summed; the field number comes from the deduplicated safety corpus, and each curve's noise is flagged with its own noise rating. Second, the event counter is the density of the collection, not world activity. Third, the causal *why* is interpretation. Fourth, no flow is built from fund to organisation, because the recipient sits only in the description text, not as a separate structure. And fifth, the money charts mix philanthropy, government (already international) and corporate money, different kinds of dollars, some of them stated budgets rather than disbursements: the venture money in VC and equity stays a **separate view**, never summed with grants; the pledges, annual budgets and field-wide estimates get their own **dots-only fourth view**, shown but never summed; and the promises with no figure at all stay purely narrative. **Where to recheck numbers.** [GIT ai-safety-genealogy](https://github.com/eericheva/ai-safety-genealogy)