{"slug": "resnet-explained-why-skipping-layers-changed-everything", "title": "ResNet Explained: Why Skipping Layers Changed Everything", "summary": "Microsoft researchers introduced ResNet in 2015, a neural network architecture that uses skip connections to solve the degradation problem in deep networks. By allowing layers to learn residual mappings instead of full transformations, ResNet enabled training of very deep networks, becoming one of the most influential architectures in deep learning history.", "body_md": "For a long time, deep learning had one guiding belief:\n\n*If a neural network becomes deeper, it becomes smarter.*\n\nThis made sense. A shallow network learns simple patterns. A deeper network learns complex ones. In computer vision, early layers detect edges, middle layers detect shapes, and deeper layers recognize objects — dogs, cars, faces, buildings.\n\nSo researchers kept asking the natural next question: ** What if we just keep adding more layers? **🤔\n\nIt worked — until it didn’t.\n\nAfter a certain depth, adding more layers didn’t improve performance. It made the model *worse*. And the solution to this problem turned out to be surprisingly counter-intuitive:\n\nLet the network skip some layers. 😲\n\nBefore ResNet, Convolutional Neural Networks (CNN’s) were already powerful and getting more impressive every year.\n\nA CNN doesn’t look at an image as a flat grid of pixels.\n\nInstead, it learns visual patterns step by step — early layers catch fine details like edges and textures, middle layers assemble those into shapes like eyes or wheels, and deeper layers combine everything to recognize full objects. Show it a photo of a cat, and the early layers see curves and lines, the middle layers detect ears, whiskers, and fur, and the final layers understand: *this is a cat.*\n\nArchitectures like [AlexNet](https://patilswaraj22.medium.com/alexnet-pioneering-the-path-to-modern-deep-learning-16525572e266) and [VGG](https://patilswaraj22.medium.com/vgg16-and-vgg19-foundational-architectures-in-cnn-based-image-recognition-aaa4e91099bb) proved the concept at scale. VGG gained particular traction for its clean, consistent design — small convolutional layers stacked neatly on top of each other, going deeper and deeper. The logic felt airtight: more layers, more learning capacity, better performance. Then deep learning ran into a wall.\n\nAs researchers pushed CNN’s deeper, something unexpected happened. Performance stopped improving and then reversed.\n\nA 100-layer network performing worse than a 60-layer network. Not on new *training* data. This wasn’t overfitting. Over-fitting means a model memorizes training examples but fails to generalize. This was different. The deeper model wasn’t even learning its training data well. The extra layers weren’t helping. They were hurting. **Researchers called this the degradation problem**. The root cause was optimization. As networks got deeper, they became harder to train. There was also a related issue: vanishing gradients.\n\nNeural networks learn by sending error signals backward through the layers — telling each layer how to adjust. But in very deep networks, those signals weaken as they travel. By the time a gradient reaches the earlier layers, it can become so faint that it barely registers.\n\nImagine passing a whispered message through a hundred people. By the hundredth person, the message has lost most of its meaning. The same thing happens in deep networks. Earlier layers stop learning effectively. The model stagnates. The challenge was clear: *How do you build very deep networks without making them impossible to train?*\n\nResNet — short for Residual Network — was introduced by researchers at Microsoft in 2015. It became one of the most influential architectures in the history of deep learning.\n\nThe core idea is almost elegantly simple:\n\nInstead of forcing every layer to learn a completely new transformation of the input, ResNet lets the original input *skip ahead* and be added back to the output later.\n\nBelow is the difference between a standard block and a ResNet block\n\nSo the final output becomes: learned transformation + original input.\n\nThat addition, the shortcut, changes everything.\n\nThink about editing a document. You could rewrite it entirely from scratch every time you want to make a change. Or you could keep the original and only mark what needs to be different. The second approach is obviously easier. That’s exactly what ResNet does.\n\nInstead of asking each layer, *“Learn the full output,”* ResNet asks, *“Learn what needs to change from the input.”*\n\nIf a layer doesn’t need to do much, it can essentially pass the input forward untouched. The network only transforms what actually needs transforming. This **“what needs to change”** is called the **Residual, **hence the name **Residual Network.**\n\nIn practice, this matters because deep networks often don’t need every layer to make a dramatic transformation. Many layers are just making small refinements. Without skip connections, those layers are forced to reconstruct the full representation anyway. With skip connections, they can focus only on what’s new.\n\nSkip connections fixed both problems at once. During forward propagation, useful information flows through the network more easily. It doesn’t have to pass through every layer in strict sequence — it has a shortcut.\n\nDuring back-propagation, gradients flow backward through the shortcuts as well. Earlier layers receive stronger, cleaner learning signals. The result, deep networks became *trainable*.\n\nBefore ResNet, networks beyond a certain depth were essentially too difficult to optimize. After ResNet, researchers successfully trained networks with 50, 101, and even 152 layers — and performance kept improving with depth. That was the real breakthrough. ResNet didn’t just make networks deeper. It made depth *useful*.\n\nResNet re-framed how the field thought about network design. Before it, the dominant assumption was: ** add layers, get smarter**. Architecture was mostly about stacking and scaling. After ResNet, the question became more nuanced:\n\nThe impact was immediate and broad. ResNet significantly pushed the state of the art in image classification. It became a standard backbone for object detection, image segmentation, face recognition, and medical imaging. It showed up in autonomous driving systems analyzing roads and pedestrians, in industrial inspection models catching manufacturing defects, and in satellite image analysis.\n\nBut the deeper influence was conceptual. Residual connections became a general-purpose design principle, not just for CNN’s, but across deep learning. Modern transformer models, large language models, and vision transformers, and many of them use residual-style connections at their core.\n\nA standard deep network says: *Every layer must transform the data. *ResNet says: *Only transform what needs to change. Let the rest pass through.*\n\nThat single shift in perspective made very deep networks practical. It solved the degradation problem, tamed the vanishing gradient, and unlocked a new era of architecture design. Skipping layers sounds like taking a shortcut. In deep learning, it turned out to be the smarter path.\n\nResNet mattered because it solved a real, concrete problem and revealed something deeper about how we should think about building neural networks.\n\nDepth was never the goal. *Effective learning* was the goal. And sometimes, the way to learn better is not to add more complexity; it’s to create a cleaner path for information to travel. That lesson still echoes through every major architecture being built today.\n\n— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —\n\n*If this was useful, consider giving it a clap; it really helps. I write about ML, AI, and technology. Follow me here on Medium so you don’t miss the next one.*\n\n*📌 More from me:*\n\n*→ **Keras Implementation of LE-NET*\n\n*→ **AlexNet: Pioneering the Path to Modern Deep Learning*\n\n*→ **VGG16 and VGG19: Foundational Architectures in CNN-based Image Recognition*\n\n[ResNet Explained: Why Skipping Layers Changed Everything](https://pub.towardsai.net/resnet-explained-why-skipping-layers-changed-everything-6c3fc7d9da66) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/resnet-explained-why-skipping-layers-changed-everything", "canonical_source": "https://pub.towardsai.net/resnet-explained-why-skipping-layers-changed-everything-6c3fc7d9da66?source=rss----98111c9905da---4", "published_at": "2026-07-12 13:36:08+00:00", "updated_at": "2026-07-12 13:39:41.881853+00:00", "lang": "en", "topics": ["neural-networks", "computer-vision"], "entities": ["Microsoft", "ResNet", "AlexNet", "VGG"], "alternates": {"html": "https://wpnews.pro/news/resnet-explained-why-skipping-layers-changed-everything", "markdown": "https://wpnews.pro/news/resnet-explained-why-skipping-layers-changed-everything.md", "text": "https://wpnews.pro/news/resnet-explained-why-skipping-layers-changed-everything.txt", "jsonld": "https://wpnews.pro/news/resnet-explained-why-skipping-layers-changed-everything.jsonld"}}