Alignment Research Engineer Accelerator This is where the ARENA course conten

Alignment Research Engineer Accelerator

        This is where the ARENA course content is hosted. For more information about the ARENA program,

    
        including upcoming cohorts and how to apply, visit [arena.education](https://arena.education).

Fundamentals #

Build your foundation in deep learning, from prerequisites through CNNs, optimization, backpropagation, and generative models.

Interpretability #

Dive deep into language model interpretability, from linear probes and SAEs to circuit analysis and toy models.

RL #

Take a whirlwind tour through RL, starting from tabular learning and Atari, and ending with some of the cutting-edge techniques used in current LLM post-training.

Evals #

Learn to build and run evaluations for large language models, including dataset generation and LLM agents.

Alignment Science #

Case studies in misalignment, covering a range of topics and techniques (both white-box and black-box). 0.0 Prerequisites Essential PyTorch basics, einops/einsum libraries, and tensor manipulation fundamentals.

0.1 Ray Tracing Learn batched operations and linear algebra by rendering 3D meshes with raytracing.

0.2 CNNs & ResNets Build neural networks from scratch, from MNIST classifiers to ResNets for CIFAR-10.

0.3 Optimization Implement SGD, RMSprop & Adam optimizers, and use Weights & Biases for experiment tracking.

0.4 Backpropagation Build your own autograd system from scratch and train MLPs with custom backpropagation.

0.5 VAEs & GANs Implement GANs and VAEs, foundational architectures for generative image models.

1.1 Transformers from Scratch Build a transformer from scratch and load pretrained GPT-2 weights.

1.2 Intro to Mech Interp Learn TransformerLens to extract activations, apply hooks & find important attention heads.

1.3.1 Linear Probes Train linear probes to detect deception in a model playing the game Coup.

1.3.2 Function Vectors & Model Steering Steer model behaviour using activation interventions and the nnsight library.

[1.3.3 Interpretability with SAEs

Use SAEs to decompose LLM activation space, monitor cognition & steer behaviour.](/chapter1_transformer_interp/13_saes/intro) 1.3.4 Activation Oracles Implement activation oracles to reveal hidden knowledge and uncover forward-predictions.

1.4.1 Indirect Object Identification Reverse-engineer the IOI circuit in GPT-2 small following 'Interpretability in the Wild'.

1.4.2 SAE Circuits Apply SAEs to circuit analysis, decomposing computations and tracing features through layers.

1.5.1 Balanced Bracket Classifier Reverse-engineer the algorithm learned by a bracket-balancing transformer.

1.5.2 Grokking & Modular Arithmetic Discover Fourier circuits in modular arithmetic models and observe grokking in action.

1.5.3 OthelloGPT Investigate emergent world representations in a GPT model trained on Othello games.

1.5.4 Superposition & SAEs Replicate Anthropic's superposition paper and train SAEs to recover features.

Monthly Algorithmic Problems 7 algorithmic challenges to test your interpretability skills in hackathon format.

2.1 Intro to RL RL fundamentals: MDPs, policies, value functions, and multi-armed bandits.

2.2.1 DQN Implement DQN for CartPole and beyond.

2.2.2 VPG Implement Vanilla Policy Gradient for CartPole.

2.3 PPO Build a PPO agent from scratch and train it to master CartPole.

[2.4 RLHF

Implement RLHF end-to-end, applying PPO to language model finetuning.](/chapter2_rl/04_rlhf/intro) 2.5 MCTS & AlphaZero Implement MCTS and AlphaZero to train agents for complex games.

3.1 Intro to Evals Design threat models and specifications for evaluating model properties.

[3.2 Dataset Generation

Use LLMs to generate and refine high-quality evaluation datasets.](/chapter3_llm_evals/02_dataset_gen/intro) 3.3 Running Evals with Inspect Run standardised LLM evaluations using UK AISI's Inspect library.

3.4 LLM Agents Build LLM agents with scaffolding to play Wikipedia Racing and other tasks.

3.5 AI Control Learn to monitor and control AI systems in a simulated environment.

4.1 Emergent Misalignment Study emergent misalignment in finetuned models.

4.2 Science of Misalignment Two case studies in black-box investigation to understand and characterize seemingly misaligned behaviour.

4.3 Interpreting Reasoning Models Apply interpretability techniques to chain-of-thought reasoning models.

4.4 LLM Psychology & Persona Vectors Explore persona vectors and psychological properties of language models.

4.5 Investigator Agents Use AI agents for investigating model behaviours (including petri & bloom).