GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension

An M.Tech thesis at IIT Bhilai implemented GPU acceleration for the MSCRED multivariate time-series anomaly detection model, achieving a 12.68× encoder speed-up by using CUDA and an im2col plus GEMM approach. The work, conducted by a student in the Data Science and AI program, reduced encoder runtime from 300 seconds on CPU to 23.664 seconds on GPU.

During my M.Tech in Data Science and Artificial Intelligence at IIT Bhilai 2021–2023 , I conducted thesis research on multivariate time-series anomaly detection and GPU acceleration using industrial process signals. My thesis, titled “Anomaly Detection of Multivariate Time Series Data and Acceleration Using GPUs,” explored an MSCRED—Multi-Scale Convolutional Recurrent Encoder-Decoder—and ConvLSTM-based pipeline for modelling relationships between signals and detecting abnormal temporal patterns. Although the model produced useful anomaly-detection results, its convolution-heavy encoder created a significant CPU bottleneck. I therefore implemented GPU-accelerated operations using CUDA, transformed convolution into an im2col plus GEMM workflow, investigated global-memory and shared-memory implementations, and integrated the accelerated operations with PyTorch. In the measured workload, encoder runtime decreased from 300 seconds on the CPU to 23.664 seconds on the GPU , representing an approximately 12.68× encoder speed-up . Research attribution:MSCRED and ConvLSTM are established architectures introduced by the researchers cited in the References section. This article describes how I used, adapted, profiled, and accelerated selected operations as part of my M.Tech thesis. Data confidentiality:The original research involved industrial process data. This article discusses the architecture, optimisation methodology, and aggregate benchmark results without publishing proprietary signals, internal process details, confidential datasets, or restricted source code. Industrial systems often generate many related sensor and process-control signals at the same time. A single signal may appear normal when viewed independently, while the relationship between several signals may indicate an abnormal operating condition. This makes multivariate time-series anomaly detection more difficult than analysing one signal at a time. The model must learn: The goal of the research was to detect unusual multivariate patterns while preserving the temporal and spatial relationships present in the data. MSCRED models relationships between multiple time-series signals by constructing signature matrices over different temporal windows. At a high level, the pipeline contains: The encoder processes the signature matrices through convolutional operations, while ConvLSTM captures how the encoded representations evolve over time. Multivariate signals ↓ Signature matrices ↓ CNN encoder ↓ ConvLSTM ↓ Decoder ↓ Reconstruction error ↓ Anomaly score A signature matrix represents pairwise relationships between signals over a selected time interval. By generating these matrices at multiple scales, the model can observe both short-term and longer-term relationships. This helps identify anomalies that may not be visible in an individual signal. The convolutional encoder extracts spatial patterns from the signature matrices. It reduces the raw matrix representation into compact feature maps that capture important relationships between signals. ConvLSTM models how those spatial representations change over time. Unlike a standard fully connected LSTM, ConvLSTM preserves the spatial structure of feature maps while learning temporal dependencies. The decoder reconstructs the expected signature matrices. When the reconstructed output differs significantly from the observed input, the reconstruction error can indicate an anomalous pattern. The purpose of the MSCRED and ConvLSTM pipeline was to detect abnormal relationships and temporal behaviour across multiple industrial process signals. The model learned normal multivariate patterns by reconstructing signature matrices. During evaluation, the difference between the observed signature matrices and the reconstructed outputs was used to calculate a reconstruction error. Observed signature matrix ↓ MSCRED reconstruction ↓ Reconstruction error ↓ Thresholding ↓ Normal or anomalous A larger reconstruction error indicated that the observed relationships differed from patterns learned during normal operation and could therefore represent an anomaly. The thesis evaluated anomaly-detection quality using the F1 score. Under the reported experimental setup, the best recorded F1 score was 0.941 . This result represents model-quality evaluation and should be interpreted separately from the CUDA runtime measurements presented later. The GPU memory optimisations were designed to reduce computation time, not to improve predictive accuracy. The encoder repeatedly applied convolutional operations to signature matrices. On the CPU, these operations involved substantial nested computation across: This made the encoder a strong candidate for GPU acceleration because many convolution calculations could be performed in parallel. The objective was not to redesign the anomaly-detection model. Instead, it was to preserve its expected computational behaviour while accelerating selected expensive operations. im2col and GEMM A convolution repeatedly applies a kernel to overlapping regions of an input feature map. The im2col , or image-to-column, transformation rearranges those local regions into columns of a matrix. After this transformation, convolution can be expressed as a general matrix multiplication: Output matrix = Weight matrix × im2col Input This conversion is useful because matrix multiplication is highly parallel and can be executed efficiently on a GPU. Input feature map ↓ Extract overlapping kernel windows ↓ Arrange windows as matrix columns ↓ Multiply by reshaped filter matrix ↓ Reshape result into output feature map GEMM stands for General Matrix-Matrix Multiplication . Modern GPU architectures are designed to perform large numbers of multiply-and-accumulate operations in parallel. Expressing convolution as GEMM allows the workload to use that parallelism. im2col introduces additional memory usage because overlapping input regions are copied into an intermediate matrix. The method is most effective when the computational benefit of parallel matrix multiplication outweighs the cost of creating and storing the intermediate representation. The CUDA implementation divided the computation across many GPU threads. Each thread was responsible for part of the output, while thread blocks grouped related work. Important kernel-design considerations included: The implementation required careful validation because incorrect indexing in GPU code can produce plausible-looking but incorrect outputs. The first GPU implementation relied primarily on global memory . Global memory provides access to the complete input, but its latency is considerably higher than that of on-chip shared memory. The optimised implementation staged reusable tiles inside shared memory . Threads within the same block could then reuse nearby values without repeatedly loading them from global memory. The shared-memory optimisation was intended to: Global memory ↓ Load reusable tile ↓ Shared memory ↓ Threads perform repeated calculations ↓ Write result to global memory Shared memory is limited in capacity, so tile dimensions and thread-block organisation must be selected carefully. Poorly selected configurations can: To use the accelerated operations inside the existing model pipeline, I connected the CUDA implementation to PyTorch through a custom extension. The integration enabled PyTorch tensors to be passed to the CUDA implementation and the computed results to be returned to the model pipeline. Important integration concerns included: Correctness validation was essential. A faster kernel is not useful if it changes tensor shapes, produces incorrect indexing, introduces unacceptable numerical differences, or behaves inconsistently for the tested configurations. The custom extension made it possible to retain PyTorch for model development while using lower-level CUDA operations for selected performance-critical components. The benchmark used the following environment: The timings reported here are measurements from my thesis experiments using the specified hardware and workload. The CPU and GPU implementations used the same input dimensions and batch configuration. The results should be treated as comparative experimental measurements rather than universal library benchmarks. Performance may vary with implementation details, CUDA configuration, tensor dimensions, data-transfer costs, compiler settings, and hardware. | Component | CPU | GPU: global memory | GPU: shared memory | |---|---|---|---| | Encoder | 300 s | 27.492 s | 23.664 s | | ConvLSTM | 1,200 s | 909.208 s | 847.260 s | | Total runtime | 2,700 s | 1,594 s | 1,481 s | The shared-memory implementation reduced encoder runtime from 300 seconds to 23.664 seconds , producing an approximately 12.68× encoder speed-up . For the complete measured pipeline, runtime decreased from 2,700 seconds to 1,481 seconds . The total-runtime measurement represents the complete measured pipeline rather than only the encoder and ConvLSTM components shown separately. The convolutional encoder benefited strongly from GPU execution because its operations provided substantial data parallelism. ConvLSTM achieved a smaller improvement because of: Consequently, the 12.68× encoder acceleration did not translate into a 12.68× end-to-end speed-up . After the encoder was accelerated, ConvLSTM became the dominant runtime bottleneck. This is an important performance-engineering lesson: accelerating one component can move the bottleneck to another part of the system. The CUDA implementation was compared with the reference implementation for the tested tensor shapes and configurations. The optimisation work focused on reducing execution time while preserving the expected computational behaviour. Model-quality evaluation and runtime evaluation were treated as separate concerns. Several engineering challenges appeared during the implementation. The CUDA kernel had to reproduce the same logical output as the reference convolution operation. A small indexing error could generate incorrect feature maps without necessarily causing a runtime failure. The im2col transformation created additional intermediate data. This required careful allocation, reuse, and release of GPU memory. Threads sharing data through shared memory had to synchronise at appropriate points. Missing or unnecessary synchronisation could cause incorrect results or reduced performance. Thread-block dimensions and tile sizes affected: The extension had to account for tensor shapes, data types, device placement, memory layout, compilation, linking, and output comparison. Improving one component changed the relative cost of other components. Profiling therefore had to be repeated after major optimisations. Once the encoder was accelerated, ConvLSTM represented a larger proportion of the remaining runtime. im2col Makes Convolution GPU-Friendly but Introduces Memory Overhead The transformation enables GEMM-based execution while creating a larger intermediate representation. Its benefit depends on access patterns, tile sizes, synchronisation, and thread-block configuration. Performance work should focus on measured bottlenecks rather than assumptions about which component is slow. GPU output should be compared against a trusted reference implementation before performance improvements are accepted. Device handling, tensor shapes, data types, compilation, numerical validation, and fallback behaviour all require careful attention. MSCRED stands for Multi-Scale Convolutional Recurrent Encoder-Decoder . It is an architecture designed to model relationships between multiple time-series signals and detect abnormal behaviour through reconstruction error. im2col do? im2col rearranges overlapping convolution windows into matrix columns, allowing convolution to be expressed as matrix multiplication. Shared memory allowed threads in the same block to reuse data without repeatedly loading it from slower global memory. This reduced memory-access overhead for reusable tiles. That was not the purpose of the optimisation, and this article does not claim that it did. The best recorded F1 score describes anomaly-detection quality in the thesis experiments. The CPU and GPU timing results describe computational performance. They should be interpreted separately. After the encoder became faster, ConvLSTM represented a larger share of total runtime. The overall system was therefore limited by the remaining bottleneck. No. Optimised PyTorch and CUDA libraries are already highly efficient for many operations. A custom kernel is most useful when: No. The original dataset contains industrial process information. A public reproduction would require synthetic or appropriately anonymised multivariate time-series data. No. Performance depends on GPU architecture, CPU configuration, tensor dimensions, batch size, kernel implementation, memory layout, CUDA and PyTorch versions, compiler settings, and workload characteristics. The reported numbers apply to the specific experimental setup described in this article. The reported results were obtained on a specific hardware configuration and workload. Different GPUs, tensor dimensions, batch sizes, memory layouts, CUDA versions, and kernel configurations may produce different results. The original dataset cannot be published because it contains industrial process information. A public reproduction would therefore require synthetic or appropriately anonymised multivariate time-series data. This article presents aggregate benchmark measurements and technical methodology rather than the complete industrial dataset or restricted implementation. Future work could focus on: This work was completed as part of my M.Tech thesis research at the Indian Institute of Technology Bhilai . It demonstrated how low-level GPU optimisation can be integrated into a deep-learning anomaly-detection pipeline. The convolutional encoder benefited substantially from: im2col -based loweringEncoder runtime decreased from 300 seconds to 23.664 seconds in the measured workload. The broader result also illustrated an important systems principle: once one stage is accelerated, another stage may become the new performance bottleneck. In this case, ConvLSTM limited the end-to-end improvement even after the encoder achieved a substantial speed-up. The work reinforced that successful performance engineering requires both low-level optimisation and system-level measurement. Zhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., and Chawla, N. V. 2019 . A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. Proceedings of the AAAI Conference on Artificial Intelligence , 33 01 , 1409–1416. https://doi.org/10.1609/aaai.v33i01.33011409 https://doi.org/10.1609/aaai.v33i01.33011409 Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-C. 2015 . Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Advances in Neural Information Processing Systems 28 , 802–810. https://proceedings.neurips.cc/paper files/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html https://proceedings.neurips.cc/paper files/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html NVIDIA. CUDA C++ Programming Guide. https://docs.nvidia.com/cuda/cuda-c-programming-guide/ https://docs.nvidia.com/cuda/cuda-c-programming-guide/ PyTorch. Custom C++ and CUDA Operators. https://docs.pytorch.org/tutorials/advanced/cpp custom ops.html https://docs.pytorch.org/tutorials/advanced/cpp custom ops.html If this article informs academic or technical work, please cite the original MSCRED and ConvLSTM papers above. To refer specifically to my thesis implementation, optimisation process, or reported benchmarks, you may cite this article as: Vutnoor, R. 2026 . “GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension.” DEV Community. If you found the article useful, a reaction, comment, or constructive technical question on DEV would be appreciated. I am Ranjith Vutnoor , an AI/ML Software Engineer and IIT Bhilai alumnus working on production RAG systems, LLM evaluation, model fine-tuning, PyTorch, and GPU-accelerated machine learning.