Connectionist Temporal Classification (CTC) is an algorithm for training deep neural networks on sequence problems like speech and handwriting recognition where input-output alignment is unknown. CTC …
The article explores morphogenesis, the self-organizing process by which a single cell develops into a complex multicellular organism, highlighting its robustness and regenerative capabilities. It dis…
This article from Distill explains graph neural networks (GNNs), a family of neural networks designed to operate on graph-structured data like social networks and molecules. It highlights the challeng…
Introductory overview of Graph Neural Networks (GNNs), explaining how neural networks can be adapted to process data structured as graphs. It covers the fundamental components of GNNs, explores why gr…
The article investigates the robustness of Neural Cellular Automata (CA) by training adversarial models to hijack the system's behavior. It explores two types of attacks: injecting adversarial cells i…
"weight banding," a structural phenomenon in neural networks where the weights in the final convolutional layer of vision models using global average pooling display a uniform spatial pattern, particu…
"branch specialization" as a large-scale structural phenomenon in neural networks, where layers split into branches and neurons self-organize into functional units similar to biological brain regions.…
Researchers discovered that artificial neural networks like CLIP contain "multimodal neurons" that respond to abstract concepts (such as "Spider-Man") across different input modalities like images and…
The article explores the use of Neural Cellular Automata (NCA) for texture synthesis, focusing on reproducing the general appearance of textures rather than exact copies. The authors argue that NCA ar…
Challenge of understanding neural networks by visualizing their weights, comparing it to reverse engineering compiled code or studying biological neural networks. It notes that despite the importance …
Reverse engineering of a learned algorithm from a neural network's weights, using its core principles to build a new artificial neural network from scratch. It also outlines the specific contributions…
Individual neurons within convolutional neural networks often form "transformed versions of the same basic feature," such as rotated, scaled, or color-shifted copies, creating a form of internal symme…
Distill thread exploring differentiable self-organizing systems, which use optimization to learn individual agent behaviors that achieve collective goals. It presents several research articles on topi…
"self-classifying MNIST task," where a grid of locally-communicating agents, operating under identical rules, must determine which digit their collective shape forms without any agent knowing its glob…
Bayesian optimization is a technique used to tune hyperparameters in machine learning algorithms by optimizing black-box functions. The article explains the process using a gold mining analogy, where …
Overview of the first five layers of the InceptionV1 neural network, categorizing its early vision neurons into "neuron families" that detect features ranging from raw pixels to sophisticated shapes a…
Deep neural networks can be understood as pipelines of simple functions, and that the intermediate values (or "activations") within these networks can be viewed as high-dimensional vectors. To analyze…