{"slug": "pairsae-mechanistic-interpretability-from-pair-representations-in-protein-co", "title": "PairSAE: Mechanistic Interpretability from Pair Representations in Protein Co-Folding", "summary": "Researchers introduced PairSAE, a method for mechanistic interpretability of protein co-folding models that summarizes pairwise tensors into token-level features, avoiding the quadratic blow-up of standard sparse autoencoders. Evaluated on Boltz-2 activations, PairSAE produced interpretable features aligned with UniProt annotations and predicted affinity values, linking the model's latent space to structural concepts.", "body_md": "arXiv:2606.27440v1 Announce Type: new\nAbstract: Foundation models for structural biology have achieved remarkable performance in predicting biomolecular structure and show promise for the design of proteins and small molecules. Yet understanding which internal features drive their outputs remains challenging. Standard sparse autoencoders (SAEs), effective on transformer-style sequence embeddings, do not transfer cleanly to pairformer-like architectures: naively operating on pairwise representations yields a quadratic blow-up of features and obscures concepts distributed jointly across sequence and pair representations.\nWe introduce PairSAE, which summarizes pairwise tensors via an N-mode SVD into token-wise interaction roles, then uses a sparse autoencoder to learn a shared set of token-level features that decode into both sequence and pair representations. Evaluated on Boltz-2 activations for PLINDER protein-ligand complexes, PairSAE yields interpretable features that align with UniProt annotations and predict Boltz-2 affinity values. These results indicate that PairSAE links the latent space of foundation models for structural biology to interpretable structural concepts, clarifying what the model \"knows\" while avoiding pairformer-induced pitfalls that limit conventional SAEs.", "url": "https://wpnews.pro/news/pairsae-mechanistic-interpretability-from-pair-representations-in-protein-co", "canonical_source": "https://arxiv.org/abs/2606.27440", "published_at": "2026-06-29 04:00:00+00:00", "updated_at": "2026-06-29 04:09:13.566476+00:00", "lang": "en", "topics": ["machine-learning", "ai-research"], "entities": ["PairSAE", "Boltz-2", "PLINDER", "UniProt"], "alternates": {"html": "https://wpnews.pro/news/pairsae-mechanistic-interpretability-from-pair-representations-in-protein-co", "markdown": "https://wpnews.pro/news/pairsae-mechanistic-interpretability-from-pair-representations-in-protein-co.md", "text": "https://wpnews.pro/news/pairsae-mechanistic-interpretability-from-pair-representations-in-protein-co.txt", "jsonld": "https://wpnews.pro/news/pairsae-mechanistic-interpretability-from-pair-representations-in-protein-co.jsonld"}}