{"slug": "alexnet-pioneering-the-path-to-modern-deep-learning", "title": "AlexNet: Pioneering the Path to Modern Deep Learning", "summary": "In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton published the AlexNet paper at NIPS, introducing a deep convolutional neural network architecture that revolutionized computer vision by winning the ImageNet competition. The architecture, which uses multiple convolutional layers, ReLU activations, and dropout, laid the foundation for modern deep learning and remains influential today.", "body_md": "In the [preceding article](https://patilswaraj22.medium.com/keras-based-implementation-of-lenet-5-db678b11c61a), we explored the utilization of the **LeNet-5** Architecture and the remarkable capability of Convolutional Neural Networks in recognizing handwritten characters within images. **Yann Le Cunn** introduced the **LeNet-5** architecture in **1998**, a time when computing resources with significant power were limited, resulting in a limited range of architectural modifications.\n\nIn the year 2012, there was a complete revolution observed in this field. A paper was published at the **NIPS** (Neural Information Processing Systems)** **conference named **“*** ImageNet Classification with Deep Convolutional Neural Networks*** ,” **authored by\n\nIn comparison with LeNet, AlexNet is a complex architecture that uses different Kernels of various sizes. Let’s take a look at the architecture below.\n\nInitially, this architecture might look terrifying, but a simplified or easy-to-understand image can be seen below.\n\n**Notable Points**\n\nThese were the most important points discussed in the paper, which are still used today.\n\n**P.S. -:** The input size mentioned in the paper is **224 x 224 x 3, **and the first convolutional layers output consists of **96 Kernels **with** size 11 x 11 **along with** stride = 3. **So according to this, the output should be\n\n([(224–11)/4] + 1 ) X ([(224 -11)/4] + 1) = ( 54 x 54 )\n\nThis does not match the output shown in the architecture, which is **55 x 55**. The possible explanation for this is that the author might have performed a padding operation on the image to increase the size of the image from **224 x 224** to **227 x 227. **Now, if we consider the input image size to be** 227 x 227, **then the entire calculation falls right into place.\n\n([(227–11)/4] + 1 ) X ([(227 -11)/4] + 1) = ( 55 x 55 )\n\nNow that we have gone through the crucial points let’s look at the architecture implementation of the same.\n\nThe entire Python code for the same, along with the Jupyter Notebook, can be accessed here:\n\n``` python\nimport numpy as npimport pandas as pdimport seaborn as snsimport matplotlib.pyplot as pltimport tensorflow as tffrom tensorflow.keras import layersfrom tensorflow.keras.datasets import mnistfrom tensorflow.keras.layers import Dense, Flatten, Input, Dropoutfrom tensorflow.keras.layers import Conv2D, MaxPooling2Dfrom tensorflow.keras.models import  Modelfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStoppingfrom sklearn.metrics import confusion_matrix # Configurable ParametersEPOCHS = 10BATCH_SIZE = 32LOSS = tf.keras.metrics.categorical_crossentropyOPTIMIZER = tf.keras.optimizers.SGD()METRIC = ['accuracy']def plot_errors(history, epochs):    # Plotting Train and Validation Loss    epochs_range = list(range(1, epochs + 1))    train_loss = history.history['loss']    val_loss = history.history['val_loss']    plt.figure(1, figsize=(10, 6))    plt.plot(epochs_range, train_loss, label='train_loss')    plt.plot(epochs_range, val_loss, label='validation_loss')    plt.xlabel('Epochs')    plt.ylabel('Categorical Cross Entropy')    plt.legend()    plt.show()def plot_confusion_matrix(y_test, y_pred, classes=[]):    cm = confusion_matrix(y_test, y_pred, labels=classes)    cm_df = pd.DataFrame(cm, index=classes, columns=classes)    plt.figure(1, figsize=(16, 8))    sns.set(font_scale=1.5, color_codes=True, palette='deep')    sns.heatmap(cm_df, annot=True, annot_kws={'size': 16}, fmt='d', cmap='YlGnBu')    plt.ylabel(\"True Label\")    plt.xlabel(\"Predicted Label\")    plt.title('Confusion Matrix')    plt.show()def model_training(model, X_train, y_train, X_cv, y_cv, epochs,                    batch_size,loss, optimizer, metrics):        # Initialise optimizers    model.compile(loss=loss, optimizer=optimizer, metrics=metrics)    # Enabling Early Stopping    es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)    # Enabling check point    mc = ModelCheckpoint(filepath='bestModel.h5', monitor='val_acc',                          mode='max', verbose=1, save_best_only=True)    # Model fitting    history = model.fit(X_train, y_train, validation_data=(X_cv, y_cv),                         epochs=epochs, batch_size=batch_size,                        verbose=1, callbacks=[es, mc])    return model, historyclass AlexNet:        def __init__(self, input_shape, output_shape):        self.input_shape = input_shape        self.output_shape = output_shape        def model_initialise(self):                # Define the AlexNet Architecture        ip = Input(shape= self.input_shape)                # 1st Convolutional Layer followed by Response Normalised and Pooling        x = Conv2D(filters=96, kernel_size=(11,11), padding=\"same\", strides=(4,4), activation =\"relu\")(ip)        x = layers.Lambda(tf.nn.local_response_normalization)(x)        x = MaxPooling2D(pool_size = (3,3),  padding=\"same\", strides=(2,2))(x)                # 2nd Convolutional Layer followed by Response Normalised and Pooling        x = Conv2D(filters=256, kernel_size=(5,5), padding=\"same\", strides=(2,2), activation = \"relu\")(x)        x = layers.Lambda(tf.nn.local_response_normalization)(x)        x = MaxPooling2D(pool_size = (3,3), padding=\"same\", strides=(2,2))(x)                # 3rd Convolutional Layer         x = Conv2D(filters=384, kernel_size=(3,3), padding=\"same\", strides=(1,1), activation = \"relu\")(x)                # 4th Convolutional Layer         x = Conv2D(filters=384, kernel_size=(3,3), padding=\"same\", strides=(1,1), activation = \"relu\")(x)                # 5th Convolutional Layer followed by Response Normalised and Pooling        x = Conv2D(filters=256, kernel_size=(3,3), padding=\"same\", strides=(1,1), activation = \"relu\")(x)        x = layers.Lambda(tf.nn.local_response_normalization)(x)        x = MaxPooling2D(pool_size = (3,3), padding=\"same\", strides=(2,2))(x)                # Flattening        x = Flatten()(x)                # 1st Fully Connected Layer with DropOuts        x = Dense(units=4096, activation=\"relu\")(x)        x = Dropout(0.5)(x)                # 2nd Fully Connected Layer with DropOuts        x = Dense(units=4096, activation=\"relu\")(x)        x = Dropout(0.5)(x)                # Output Layer        op = Dense(units=self.output_shape, activation=\"softmax\")(x)                # Define Model        model = Model(inputs=ip, outputs=op)                model.summary()                return modelif __name__ == '__main__':    # Loading the dataset and perform splitting    (X_train, y_train), (X_test, y_test) = mnist.load_data()    # Resize image to 32x 32    X_train = np.array([np.pad(X_train[i], pad_width=2) for i in range(X_train.shape[0])])    X_test = np.array([np.pad(X_test[i], pad_width=2) for i in range(X_test.shape[0])])    # Performing reshaping operation    X_train = X_train.reshape(X_train.shape[0], 32, 32, 1)    X_test = X_test.reshape(X_test.shape[0], 32, 32, 1)    # Normalization    X_train = X_train / 255    X_test = X_test / 255    # One Hot Encoding    y_train = tf.keras.utils.to_categorical(y_train, 10)    y_test = tf.keras.utils.to_categorical(y_test, 10)    # Define image size and number of classes    image_size = X_train.shape[1:]    classes = y_train.shape[1]    print(f\"Size of each image = {image_size}\")    print(f\"Nos of Classes = {classes}\")    # Create model instance and initialise Lenet Model    alexnet = AlexNet(image_size, classes)    model = alexnet.model_initialise()    epochs = EPOCHS    batch_size = BATCH_SIZE    loss = LOSS    optimizer = OPTIMIZER    metric = METRIC    model, history = model_training(model, X_train, y_train, X_test, y_test,                                     epochs, batch_size,                                    loss, optimizer, metric)        # Plot trainning and test error    plot_errors(history, min(epochs, len(history.history['loss'])))    # Perform Prediction    y_pred = model.predict(X_test)    # Get list of prediction    y_pred = np.argmax(y_pred, axis=1)    y_test = np.argmax(y_test, axis=1)    # Show Confusion Matrix    plot_confusion_matrix(y_test, y_pred, list(range(classes)))\n```\n\nThe entire Python code for the same, along with the Jupyter Notebook, can be accessed here:\n\n**References**:\n\n— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —\n\n*If this was useful, consider giving it a clap, it really helps. I write about ML, AI, and technology. Follow me here on Medium so you don’t miss the next one.*\n\n*📌 More from me:*\n\n*→ **Keras Implementation of LE-NET*\n\n[AlexNet: Pioneering the Path to Modern Deep Learning](https://pub.towardsai.net/alexnet-pioneering-the-path-to-modern-deep-learning-16525572e266) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/alexnet-pioneering-the-path-to-modern-deep-learning", "canonical_source": "https://pub.towardsai.net/alexnet-pioneering-the-path-to-modern-deep-learning-16525572e266?source=rss----98111c9905da---4", "published_at": "2026-07-12 13:29:54+00:00", "updated_at": "2026-07-12 13:39:47.838817+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "neural-networks"], "entities": ["AlexNet", "Alex Krizhevsky", "Ilya Sutskever", "Geoffrey Hinton", "NIPS", "ImageNet", "LeNet-5", "Yann LeCun"], "alternates": {"html": "https://wpnews.pro/news/alexnet-pioneering-the-path-to-modern-deep-learning", "markdown": "https://wpnews.pro/news/alexnet-pioneering-the-path-to-modern-deep-learning.md", "text": "https://wpnews.pro/news/alexnet-pioneering-the-path-to-modern-deep-learning.txt", "jsonld": "https://wpnews.pro/news/alexnet-pioneering-the-path-to-modern-deep-learning.jsonld"}}