{"slug": "keras-based-implementation-of-lenet-5", "title": "Keras Based Implementation of LeNet-5", "summary": "A developer published a Keras-based implementation of the LeNet-5 convolutional neural network architecture, originally proposed by Yann LeCun in 1989 for handwritten digit recognition. The implementation uses functional programming in Keras to replicate the original architecture, which includes convolutional, pooling, and fully connected layers.", "body_md": "Introduction\nLeNet-5\n \nis one of the first Convolutional Neural Network models proposed by Yann LeCun et al. in 1989. The architecture was first observed in the paper “\nGradient-Based Learning Applied to Document Recognition\n.” The authors have used this architecture to recognize handwritten digits. We won’t be going into the details of this paper; rather, we will focus more on implementing the architecture in Keras.\nThe LeNet-5 architecture has described in the paper is shown below\nLeNet 5 Architecture\nArchitecture Overview\nThe input image that is feed to the network is \n32 x 32\n. This is fed to the first convolutional layer with a filter size of \n5 x 5\n, \nstride =1,\n and the image is fed to \n6\n such filters ( filters are also called \nKernels\n), which generates an output \nC1\n. Kernels are nothing but a feature extractor of an image, and the extracted features are also known as “\nfeature maps\n.” The size of each feature map here is \n28 x 28,\n and \n6\n such feature maps are generated. Let’s understand how we obtained the dimensions \n28 x 28\n. This is one of the most important things in the understanding of architecture.\nLet us assume the size of image be n x n and the kernel size be k x k and stride be s\nSo the output size of the feature maps is obtained by the formula ( [ (n-k) / s ] + 1 ) x ( [ (n-k) / s ] + 1).\nIn our case the value of n = 32 and k = 5 so the feature map size (32–5+1) x (32–5+1) = 28 x 28.\nNow that we have understood this, the rest of the architecture is based on the above calculation.\nThe second layer is the pooling layer or subsampling layer. The subsampling layer performs sampling of the feature map to reduce its size. This is done to reduce the overall computation since the neighbouring pixels store more or less similar information. The subsampling layer used is of \nsize\n \n2 x 2\n with \nstride = 2\n. The output of the subsampling layer is the feature maps \nS2 \nwith size \n14 x 14\n (Calculation is the same as the above).\nThe third layer again is the convolutional layer with filter \nsize 5 x 5\n and \nstride = 1\n. The only difference here is we would be the number of kernels (\n16 kernels.) \nThe output \nC3\n would be of the size \n10 x 10\n with \n16 \nfeature maps.\n“ \nLayer C3 is a convolutional layer with 16 feature maps. Each unit in each feature map is connected to several 5 x 5 neighbourhoods at identical locations in a subset of S2’s feature maps. The first six C3\u0011feature maps take inputs from every contiguous subset of three feature maps in S2\u000e The next six take input from every contiguous subset of four The next three take input from some discontinuous subsets of four. Finally, the last one takes input from all S2 feature maps\n.”\nEach column indicates which units combine feature map in S2 in a particular feature map of C3.\u0004\nMapping of each S2 on C3\nThe obtained \nC3\n consists of \n16 \nlayers, each of \n10 x 10. \nAs earlier, again\n \nsubsampling layer here is used of \nsize\n \n2 x 2\n with \nstride = 2\n. The output here is a feature map \nS4\n with \n16\n layers of size \n5 x 5. \nThe feature map \nS4 \nis convoluted with \n120\n kernels each of size\n 5 x 5\n followed by a flattening layer to get the output \nC5 \nlayer. A fully connected layer \nF6\n of 84 layers is connected after the \nC5 \nlayers. At last, we have \n10 output\n layers obtained from the \nF6\n layer as the target variable has 10 distinct values. Now let's look at the implementation of the same architecture in Keras.\nImplementation\nBelow is the implementation of the entire architecture in Keras using Functional programming.\n\n```\n# Importsimport numpy as npimport pandas as pdimport seaborn as snsimport matplotlib.pyplot as pltimport kerasfrom keras.datasets import mnistfrom keras.layers import Dense, Flatten, Inputfrom keras.layers import Conv2D, MaxPooling2Dfrom keras.models import Sequential, Modelfrom keras.callbacks import ModelCheckpoint, EarlyStoppingfrom sklearn.metrics import confusion_matrix# Configurable ParametersEPOCHS = 20BATCH_SIZE = 32LOSS = keras.metrics.categorical_crossentropyOPTIMIZER = keras.optimizers.SGD()METRIC = ['accuracy']def plot_errors(history, epochs):    # Plotting Train and Validation Loss    epochs_range = list(range(1, epochs + 1))    train_loss = history.history['loss']    val_loss = history.history['val_loss']    plt.figure(1, figsize=(10, 6))    plt.plot(epochs_range, train_loss, label='train_loss')    plt.plot(epochs_range, val_loss, label='validation_loss')    plt.xlabel('Epochs')    plt.ylabel('Binary Cross Entropy')    plt.legend()    plt.show()def plot_confusion_matrix(y_test, y_pred, classes=[]):    cm = confusion_matrix(y_test, y_pred, labels=classes)    cm_df = pd.DataFrame(cm, index=classes, columns=classes)    plt.figure(1, figsize=(16, 8))    sns.set(font_scale=1.5, color_codes=True, palette='deep')    sns.heatmap(cm_df, annot=True, annot_kws={'size': 16}, fmt='d', cmap='YlGnBu')    plt.ylabel(\"True Label\")    plt.xlabel(\"Predicted Label\")    plt.title('Confusion Matrix')    plt.show()def model_training(model, X_train, y_train, X_cv, y_cv, epochs,                    batch_size,loss, optimizer, metrics):        # Initialise optimizers    model.compile(loss=loss, optimizer=optimizer, metrics=metrics)    # Enabling Early Stopping    es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)    # Enabling check point    mc = ModelCheckpoint(filepath='bestModel.h5', monitor='val_acc',                          mode='max', verbose=1, save_best_only=True)    # Model fitting    history = model.fit(X_train, y_train, validation_data=(X_cv, y_cv),                         epochs=epochs, batch_size=batch_size,                        verbose=1, callbacks=[es, mc])    return model, historyclass Lenet5:    def __init__(self, input_shape, output_shape):        self.input_shape = input_shape        self.output_shape = output_shape    def model_init(self, kernel_size=(5, 5), pool_size=(2, 2)):        if isinstance(kernel_size, tuple) and isinstance(pool_size, tuple):            # Define the LeNet-5 architecture            ip = Input(shape=self.input_shape)            # 1st Convolutional Layer followed by Max pooling (6 kernels)            x = Conv2D(filters=6, kernel_size=kernel_size, activation='tanh')(ip)            x = MaxPooling2D(pool_size=pool_size)(x)            # 2nd Convolution Layer followed by Max pooling (16 kernels)            x = Conv2D(filters=16, kernel_size=kernel_size, activation='tanh')(x)            x = MaxPooling2D(pool_size=pool_size)(x)            # 3rd Convolution Layer followed by Flattening (120 Kernels)            x = Conv2D(filters=120, kernel_size=kernel_size, activation='tanh')(x)            x = Flatten()(x)            # Fully Connected Dense Layer            x = Dense(units=84, activation='tanh')(x)            # Final Output Softmax(layer)            op = Dense(units=self.output_shape, activation='softmax')(x)            # Define Model            model = Model(inputs=ip, outputs=op)            model.summary()            return modelif __name__ == '__main__':    # Loading the dataset and perform splitting    (X_train, y_train), (X_test, y_test) = mnist.load_data()    # Resize image to 32x 32    X_train = np.array([np.pad(X_train[i], pad_width=2) for i in range(X_train.shape[0])])    X_test = np.array([np.pad(X_test[i], pad_width=2) for i in range(X_test.shape[0])])    # Performing reshaping operation    X_train = X_train.reshape(X_train.shape[0], 32, 32, 1)    X_test = X_test.reshape(X_test.shape[0], 32, 32, 1)    # Normalization    X_train = X_train / 255    X_test = X_test / 255    # One Hot Encoding    y_train = keras.utils.to_categorical(y_train, 10)    y_test = keras.utils.to_categorical(y_test, 10)    # Define image size and number of classes    image_size = X_train.shape[1:]    classes = y_train.shape[1]    # Create model instance and initialise Lenet Model    lenet = Lenet5(image_size, classes)    model = lenet.model_init()    epochs = EPOCHS    batch_size = BATCH_SIZE    loss = LOSS    optimizer = OPTIMIZER    metric = METRIC    model, history = model_training(model, X_train, y_train, X_test, y_test,                                     epochs, batch_size,                                    loss, optimizer, metric)    # Plot trainning and test error    plot_errors(history, epochs)    # Perform Prediction    y_pred = model.predict(X_test)    # Get list of prediction    y_pred = np.argmax(y_pred, axis=1)    y_test = np.argmax(y_test, axis=1)    # Show Confusion Matrix    plot_confusion_matrix(y_test, y_pred, list(range(classes)))\n```\n\nModel Summary\nVariations of train and validation loss on different epochs\nError Plots\nConfusion Matrix of all the classes\nThe entire python code for the same, along with the Jupyter notebook, can be accessed here:\nSP2203/Lenet5\nReferences:\nhttp://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf\nhttps://keras.io/api/\n— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —\nIf this was useful, consider giving it a clap, it really helps. I write about ML, AI, and technology. Follow me here on Medium so you don’t miss the next one.\n📌 More from me:\n→ \nSmoothening noisy GNSS data\nKeras Based Implementation of LeNet-5\n was originally published in \nTowards AI\n on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/keras-based-implementation-of-lenet-5", "canonical_source": "https://pub.towardsai.net/keras-based-implementation-of-lenet-5-db678b11c61a?source=rss----98111c9905da---4", "published_at": "2026-07-12 13:29:41+00:00", "updated_at": "2026-07-12 13:39:53.735063+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "neural-networks", "developer-tools"], "entities": ["Yann LeCun", "Keras", "LeNet-5", "MNIST"], "alternates": {"html": "https://wpnews.pro/news/keras-based-implementation-of-lenet-5", "markdown": "https://wpnews.pro/news/keras-based-implementation-of-lenet-5.md", "text": "https://wpnews.pro/news/keras-based-implementation-of-lenet-5.txt", "jsonld": "https://wpnews.pro/news/keras-based-implementation-of-lenet-5.jsonld"}}