In the preceding article, we explored the utilization of the LeNet-5 Architecture and the remarkable capability of Convolutional Neural Networks in recognizing handwritten characters within images. Yann Le Cunn introduced the LeNet-5 architecture in 1998, a time when computing resources with significant power were limited, resulting in a limited range of architectural modifications.
In the year 2012, there was a complete revolution observed in this field. A paper was published at the NIPS (Neural Information Processing Systems)** conference named “ ImageNet Classification with Deep Convolutional Neural Networks** ,” **authored by
In comparison with LeNet, AlexNet is a complex architecture that uses different Kernels of various sizes. Let’s take a look at the architecture below.
Initially, this architecture might look terrifying, but a simplified or easy-to-understand image can be seen below.
Notable Points
These were the most important points discussed in the paper, which are still used today.
P.S. -: The input size mentioned in the paper is **224 x 224 x 3, **and the first convolutional layers output consists of **96 Kernels with size 11 x 11 along with stride = 3. **So according to this, the output should be
([(224–11)/4] + 1 ) X ([(224 -11)/4] + 1) = ( 54 x 54 )
This does not match the output shown in the architecture, which is 55 x 55. The possible explanation for this is that the author might have performed a padding operation on the image to increase the size of the image from 224 x 224 to **227 x 227. Now, if we consider the input image size to be 227 x 227, **then the entire calculation falls right into place.
([(227–11)/4] + 1 ) X ([(227 -11)/4] + 1) = ( 55 x 55 )
Now that we have gone through the crucial points let’s look at the architecture implementation of the same.
The entire Python code for the same, along with the Jupyter Notebook, can be accessed here:
import numpy as npimport pandas as pdimport seaborn as snsimport matplotlib.pyplot as pltimport tensorflow as tffrom tensorflow.keras import layersfrom tensorflow.keras.datasets import mnistfrom tensorflow.keras.layers import Dense, Flatten, Input, Dropoutfrom tensorflow.keras.layers import Conv2D, MaxPooling2Dfrom tensorflow.keras.models import Modelfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStoppingfrom sklearn.metrics import confusion_matrix # Configurable ParametersEPOCHS = 10BATCH_SIZE = 32LOSS = tf.keras.metrics.categorical_crossentropyOPTIMIZER = tf.keras.optimizers.SGD()METRIC = ['accuracy']def plot_errors(history, epochs): # Plotting Train and Validation Loss epochs_range = list(range(1, epochs + 1)) train_loss = history.history['loss'] val_loss = history.history['val_loss'] plt.figure(1, figsize=(10, 6)) plt.plot(epochs_range, train_loss, label='train_loss') plt.plot(epochs_range, val_loss, label='validation_loss') plt.xlabel('Epochs') plt.ylabel('Categorical Cross Entropy') plt.legend() plt.show()def plot_confusion_matrix(y_test, y_pred, classes=[]): cm = confusion_matrix(y_test, y_pred, labels=classes) cm_df = pd.DataFrame(cm, index=classes, columns=classes) plt.figure(1, figsize=(16, 8)) sns.set(font_scale=1.5, color_codes=True, palette='deep') sns.heatmap(cm_df, annot=True, annot_kws={'size': 16}, fmt='d', cmap='YlGnBu') plt.ylabel("True Label") plt.xlabel("Predicted Label") plt.title('Confusion Matrix') plt.show()def model_training(model, X_train, y_train, X_cv, y_cv, epochs, batch_size,loss, optimizer, metrics): # Initialise optimizers model.compile(loss=loss, optimizer=optimizer, metrics=metrics) # Enabling Early Stopping es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3) # Enabling check point mc = ModelCheckpoint(filepath='bestModel.h5', monitor='val_acc', mode='max', verbose=1, save_best_only=True) # Model fitting history = model.fit(X_train, y_train, validation_data=(X_cv, y_cv), epochs=epochs, batch_size=batch_size, verbose=1, callbacks=[es, mc]) return model, historyclass AlexNet: def __init__(self, input_shape, output_shape): self.input_shape = input_shape self.output_shape = output_shape def model_initialise(self): # Define the AlexNet Architecture ip = Input(shape= self.input_shape) # 1st Convolutional Layer followed by Response Normalised and Pooling x = Conv2D(filters=96, kernel_size=(11,11), padding="same", strides=(4,4), activation ="relu")(ip) x = layers.Lambda(tf.nn.local_response_normalization)(x) x = MaxPooling2D(pool_size = (3,3), padding="same", strides=(2,2))(x) # 2nd Convolutional Layer followed by Response Normalised and Pooling x = Conv2D(filters=256, kernel_size=(5,5), padding="same", strides=(2,2), activation = "relu")(x) x = layers.Lambda(tf.nn.local_response_normalization)(x) x = MaxPooling2D(pool_size = (3,3), padding="same", strides=(2,2))(x) # 3rd Convolutional Layer x = Conv2D(filters=384, kernel_size=(3,3), padding="same", strides=(1,1), activation = "relu")(x) # 4th Convolutional Layer x = Conv2D(filters=384, kernel_size=(3,3), padding="same", strides=(1,1), activation = "relu")(x) # 5th Convolutional Layer followed by Response Normalised and Pooling x = Conv2D(filters=256, kernel_size=(3,3), padding="same", strides=(1,1), activation = "relu")(x) x = layers.Lambda(tf.nn.local_response_normalization)(x) x = MaxPooling2D(pool_size = (3,3), padding="same", strides=(2,2))(x) # Flattening x = Flatten()(x) # 1st Fully Connected Layer with DropOuts x = Dense(units=4096, activation="relu")(x) x = Dropout(0.5)(x) # 2nd Fully Connected Layer with DropOuts x = Dense(units=4096, activation="relu")(x) x = Dropout(0.5)(x) # Output Layer op = Dense(units=self.output_shape, activation="softmax")(x) # Define Model model = Model(inputs=ip, outputs=op) model.summary() return modelif __name__ == '__main__': # the dataset and perform splitting (X_train, y_train), (X_test, y_test) = mnist.load_data() # Resize image to 32x 32 X_train = np.array([np.pad(X_train[i], pad_width=2) for i in range(X_train.shape[0])]) X_test = np.array([np.pad(X_test[i], pad_width=2) for i in range(X_test.shape[0])]) # Performing reshaping operation X_train = X_train.reshape(X_train.shape[0], 32, 32, 1) X_test = X_test.reshape(X_test.shape[0], 32, 32, 1) # Normalization X_train = X_train / 255 X_test = X_test / 255 # One Hot Encoding y_train = tf.keras.utils.to_categorical(y_train, 10) y_test = tf.keras.utils.to_categorical(y_test, 10) # Define image size and number of classes image_size = X_train.shape[1:] classes = y_train.shape[1] print(f"Size of each image = {image_size}") print(f"Nos of Classes = {classes}") # Create model instance and initialise Lenet Model alexnet = AlexNet(image_size, classes) model = alexnet.model_initialise() epochs = EPOCHS batch_size = BATCH_SIZE loss = LOSS optimizer = OPTIMIZER metric = METRIC model, history = model_training(model, X_train, y_train, X_test, y_test, epochs, batch_size, loss, optimizer, metric) # Plot trainning and test error plot_errors(history, min(epochs, len(history.history['loss']))) # Perform Prediction y_pred = model.predict(X_test) # Get list of prediction y_pred = np.argmax(y_pred, axis=1) y_test = np.argmax(y_test, axis=1) # Show Confusion Matrix plot_confusion_matrix(y_test, y_pred, list(range(classes)))
The entire Python code for the same, along with the Jupyter Notebook, can be accessed here:
References:
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
If this was useful, consider giving it a clap, it really helps. I write about ML, AI, and technology. Follow me here on Medium so you don’t miss the next one.
📌 More from me:
*→ *Keras Implementation of LE-NET
AlexNet: Pioneering the Path to Modern Deep Learning was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.