Alignment Research Engineer Accelerator This is where the ARENA course conten

The Alignment Research Engineer Accelerator (ARENA) hosts its full course content at arena.education, covering deep learning fundamentals, transformer interpretability, reinforcement learning, evaluations, and alignment science. The curriculum spans chapters from PyTorch prerequisites and CNNs through sparse autoencoders, activation oracles, and circuit analysis, with details on upcoming cohorts and applications available at arena.education.

Alignment Research Engineer Accelerator This is where the ARENA course content is hosted. For more information about the ARENA program, including upcoming cohorts and how to apply, visit arena.education https://arena.education . Fundamentals Build your foundation in deep learning, from prerequisites through CNNs, optimization, backpropagation, and generative models. Interpretability Dive deep into language model interpretability, from linear probes and SAEs to circuit analysis and toy models. RL Take a whirlwind tour through RL, starting from tabular learning and Atari, and ending with some of the cutting-edge techniques used in current LLM post-training. Evals Learn to build and run evaluations for large language models, including dataset generation and LLM agents. Alignment Science Case studies in misalignment, covering a range of topics and techniques both white-box and black-box . 0.0 Prerequisites Essential PyTorch basics, einops/einsum libraries, and tensor manipulation fundamentals. /chapter0 fundamentals/00 prereqs/intro 0.1 Ray Tracing Learn batched operations and linear algebra by rendering 3D meshes with raytracing. /chapter0 fundamentals/01 ray tracing/intro 0.2 CNNs & ResNets Build neural networks from scratch, from MNIST classifiers to ResNets for CIFAR-10. /chapter0 fundamentals/02 cnns/intro 0.3 Optimization Implement SGD, RMSprop & Adam optimizers, and use Weights & Biases for experiment tracking. /chapter0 fundamentals/03 optimization/intro 0.4 Backpropagation Build your own autograd system from scratch and train MLPs with custom backpropagation. /chapter0 fundamentals/04 backprop/intro 0.5 VAEs & GANs Implement GANs and VAEs, foundational architectures for generative image models. /chapter0 fundamentals/05 vaes gans/intro 1.1 Transformers from Scratch Build a transformer from scratch and load pretrained GPT-2 weights. /chapter1 transformer interp/01 transformers/intro 1.2 Intro to Mech Interp Learn TransformerLens to extract activations, apply hooks & find important attention heads. /chapter1 transformer interp/02 intro mech interp/intro 1.3.1 Linear Probes Train linear probes to detect deception in a model playing the game Coup. /chapter1 transformer interp/11 probing/intro 1.3.2 Function Vectors & Model Steering Steer model behaviour using activation interventions and the nnsight library. /chapter1 transformer interp/12 function vectors/intro 1.3.3 Interpretability with SAEs Use SAEs to decompose LLM activation space, monitor cognition & steer behaviour. /chapter1 transformer interp/13 saes/intro 1.3.4 Activation Oracles Implement activation oracles to reveal hidden knowledge and uncover forward-predictions. /chapter1 transformer interp/14 activation oracles/intro 1.4.1 Indirect Object Identification Reverse-engineer the IOI circuit in GPT-2 small following 'Interpretability in the Wild'. /chapter1 transformer interp/21 ioi/intro 1.4.2 SAE Circuits Apply SAEs to circuit analysis, decomposing computations and tracing features through layers. /chapter1 transformer interp/22 sae circuits/intro 1.5.1 Balanced Bracket Classifier Reverse-engineer the algorithm learned by a bracket-balancing transformer. /chapter1 transformer interp/31 brackets/intro 1.5.2 Grokking & Modular Arithmetic Discover Fourier circuits in modular arithmetic models and observe grokking in action. /chapter1 transformer interp/32 grokking/intro 1.5.3 OthelloGPT Investigate emergent world representations in a GPT model trained on Othello games. /chapter1 transformer interp/33 othellogpt/intro 1.5.4 Superposition & SAEs Replicate Anthropic's superposition paper and train SAEs to recover features. /chapter1 transformer interp/34 superposition/intro Monthly Algorithmic Problems 7 algorithmic challenges to test your interpretability skills in hackathon format. /chapter1 transformer interp/monthly algorithmic/intro 2.1 Intro to RL RL fundamentals: MDPs, policies, value functions, and multi-armed bandits. /chapter2 rl/01 intro rl/intro 2.2.1 DQN Implement DQN for CartPole and beyond. /chapter2 rl/21 dqn/intro 2.2.2 VPG Implement Vanilla Policy Gradient for CartPole. /chapter2 rl/22 vpg/intro 2.3 PPO Build a PPO agent from scratch and train it to master CartPole. /chapter2 rl/03 ppo/intro 2.4 RLHF Implement RLHF end-to-end, applying PPO to language model finetuning. /chapter2 rl/04 rlhf/intro 2.5 MCTS & AlphaZero Implement MCTS and AlphaZero to train agents for complex games. /chapter2 rl/05 mcts alphazero/intro 3.1 Intro to Evals Design threat models and specifications for evaluating model properties. /chapter3 llm evals/01 intro evals/intro 3.2 Dataset Generation Use LLMs to generate and refine high-quality evaluation datasets. /chapter3 llm evals/02 dataset gen/intro 3.3 Running Evals with Inspect Run standardised LLM evaluations using UK AISI's Inspect library. /chapter3 llm evals/03 running evals/intro 3.4 LLM Agents Build LLM agents with scaffolding to play Wikipedia Racing and other tasks. /chapter3 llm evals/04 llm agents/intro 3.5 AI Control Learn to monitor and control AI systems in a simulated environment. /chapter3 llm evals/05 ai control/intro 4.1 Emergent Misalignment Study emergent misalignment in finetuned models. /chapter4 alignment science/1 emergent misalignment/intro 4.2 Science of Misalignment Two case studies in black-box investigation to understand and characterize seemingly misaligned behaviour. /chapter4 alignment science/2 science misalignment/intro 4.3 Interpreting Reasoning Models Apply interpretability techniques to chain-of-thought reasoning models. /chapter4 alignment science/3 reasoning models/intro 4.4 LLM Psychology & Persona Vectors Explore persona vectors and psychological properties of language models. /chapter4 alignment science/4 persona vectors/intro 4.5 Investigator Agents Use AI agents for investigating model behaviours including petri & bloom . /chapter4 alignment science/5 investigator agents/intro