{"slug": "gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension", "title": "GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension", "summary": "An M.Tech thesis at IIT Bhilai implemented GPU acceleration for the MSCRED multivariate time-series anomaly detection model, achieving a 12.68× encoder speed-up by using CUDA and an im2col plus GEMM approach. The work, conducted by a student in the Data Science and AI program, reduced encoder runtime from 300 seconds on CPU to 23.664 seconds on GPU.", "body_md": "During my M.Tech in Data Science and Artificial Intelligence at IIT Bhilai (2021–2023), I conducted thesis research on multivariate time-series anomaly detection and GPU acceleration using industrial process signals.\n\nMy thesis, titled **“Anomaly Detection of Multivariate Time Series Data and Acceleration Using GPUs,”** explored an MSCRED—Multi-Scale Convolutional Recurrent Encoder-Decoder—and ConvLSTM-based pipeline for modelling relationships between signals and detecting abnormal temporal patterns.\n\nAlthough the model produced useful anomaly-detection results, its convolution-heavy encoder created a significant CPU bottleneck. I therefore implemented GPU-accelerated operations using CUDA, transformed convolution into an `im2col`\n\nplus GEMM workflow, investigated global-memory and shared-memory implementations, and integrated the accelerated operations with PyTorch.\n\nIn the measured workload, encoder runtime decreased from **300 seconds on the CPU to 23.664 seconds on the GPU**, representing an approximately **12.68× encoder speed-up**.\n\nResearch attribution:MSCRED and ConvLSTM are established architectures introduced by the researchers cited in the References section. This article describes how I used, adapted, profiled, and accelerated selected operations as part of my M.Tech thesis.\n\nData confidentiality:The original research involved industrial process data. This article discusses the architecture, optimisation methodology, and aggregate benchmark results without publishing proprietary signals, internal process details, confidential datasets, or restricted source code.\n\nIndustrial systems often generate many related sensor and process-control signals at the same time. A single signal may appear normal when viewed independently, while the relationship between several signals may indicate an abnormal operating condition.\n\nThis makes multivariate time-series anomaly detection more difficult than analysing one signal at a time.\n\nThe model must learn:\n\nThe goal of the research was to detect unusual multivariate patterns while preserving the temporal and spatial relationships present in the data.\n\nMSCRED models relationships between multiple time-series signals by constructing **signature matrices** over different temporal windows.\n\nAt a high level, the pipeline contains:\n\nThe encoder processes the signature matrices through convolutional operations, while ConvLSTM captures how the encoded representations evolve over time.\n\n```\nMultivariate signals\n        ↓\nSignature matrices\n        ↓\nCNN encoder\n        ↓\nConvLSTM\n        ↓\nDecoder\n        ↓\nReconstruction error\n        ↓\nAnomaly score\n```\n\nA signature matrix represents pairwise relationships between signals over a selected time interval.\n\nBy generating these matrices at multiple scales, the model can observe both short-term and longer-term relationships. This helps identify anomalies that may not be visible in an individual signal.\n\nThe convolutional encoder extracts spatial patterns from the signature matrices. It reduces the raw matrix representation into compact feature maps that capture important relationships between signals.\n\nConvLSTM models how those spatial representations change over time.\n\nUnlike a standard fully connected LSTM, ConvLSTM preserves the spatial structure of feature maps while learning temporal dependencies.\n\nThe decoder reconstructs the expected signature matrices.\n\nWhen the reconstructed output differs significantly from the observed input, the reconstruction error can indicate an anomalous pattern.\n\nThe purpose of the MSCRED and ConvLSTM pipeline was to detect abnormal relationships and temporal behaviour across multiple industrial process signals.\n\nThe model learned normal multivariate patterns by reconstructing signature matrices. During evaluation, the difference between the observed signature matrices and the reconstructed outputs was used to calculate a reconstruction error.\n\n```\nObserved signature matrix\n            ↓\nMSCRED reconstruction\n            ↓\nReconstruction error\n            ↓\nThresholding\n            ↓\nNormal or anomalous\n```\n\nA larger reconstruction error indicated that the observed relationships differed from patterns learned during normal operation and could therefore represent an anomaly.\n\nThe thesis evaluated anomaly-detection quality using the F1 score. Under the reported experimental setup, the **best recorded F1 score was 0.941**.\n\nThis result represents model-quality evaluation and should be interpreted separately from the CUDA runtime measurements presented later. The GPU memory optimisations were designed to reduce computation time, not to improve predictive accuracy.\n\nThe encoder repeatedly applied convolutional operations to signature matrices.\n\nOn the CPU, these operations involved substantial nested computation across:\n\nThis made the encoder a strong candidate for GPU acceleration because many convolution calculations could be performed in parallel.\n\nThe objective was not to redesign the anomaly-detection model. Instead, it was to preserve its expected computational behaviour while accelerating selected expensive operations.\n\n`im2col`\n\nand GEMM\nA convolution repeatedly applies a kernel to overlapping regions of an input feature map.\n\nThe `im2col`\n\n, or image-to-column, transformation rearranges those local regions into columns of a matrix.\n\nAfter this transformation, convolution can be expressed as a general matrix multiplication:\n\n```\nOutput matrix = Weight matrix × im2col(Input)\n```\n\nThis conversion is useful because matrix multiplication is highly parallel and can be executed efficiently on a GPU.\n\n```\nInput feature map\n        ↓\nExtract overlapping kernel windows\n        ↓\nArrange windows as matrix columns\n        ↓\nMultiply by reshaped filter matrix\n        ↓\nReshape result into output feature map\n```\n\nGEMM stands for **General Matrix-Matrix Multiplication**.\n\nModern GPU architectures are designed to perform large numbers of multiply-and-accumulate operations in parallel. Expressing convolution as GEMM allows the workload to use that parallelism.\n\n`im2col`\n\nintroduces additional memory usage because overlapping input regions are copied into an intermediate matrix.\n\nThe method is most effective when the computational benefit of parallel matrix multiplication outweighs the cost of creating and storing the intermediate representation.\n\nThe CUDA implementation divided the computation across many GPU threads.\n\nEach thread was responsible for part of the output, while thread blocks grouped related work.\n\nImportant kernel-design considerations included:\n\nThe implementation required careful validation because incorrect indexing in GPU code can produce plausible-looking but incorrect outputs.\n\nThe first GPU implementation relied primarily on **global memory**.\n\nGlobal memory provides access to the complete input, but its latency is considerably higher than that of on-chip shared memory.\n\nThe optimised implementation staged reusable tiles inside **shared memory**. Threads within the same block could then reuse nearby values without repeatedly loading them from global memory.\n\nThe shared-memory optimisation was intended to:\n\n```\nGlobal memory\n      ↓\nLoad reusable tile\n      ↓\nShared memory\n      ↓\nThreads perform repeated calculations\n      ↓\nWrite result to global memory\n```\n\nShared memory is limited in capacity, so tile dimensions and thread-block organisation must be selected carefully.\n\nPoorly selected configurations can:\n\nTo use the accelerated operations inside the existing model pipeline, I connected the CUDA implementation to PyTorch through a custom extension.\n\nThe integration enabled PyTorch tensors to be passed to the CUDA implementation and the computed results to be returned to the model pipeline.\n\nImportant integration concerns included:\n\nCorrectness validation was essential. A faster kernel is not useful if it changes tensor shapes, produces incorrect indexing, introduces unacceptable numerical differences, or behaves inconsistently for the tested configurations.\n\nThe custom extension made it possible to retain PyTorch for model development while using lower-level CUDA operations for selected performance-critical components.\n\nThe benchmark used the following environment:\n\nThe timings reported here are measurements from my thesis experiments using the specified hardware and workload. The CPU and GPU implementations used the same input dimensions and batch configuration.\n\nThe results should be treated as comparative experimental measurements rather than universal library benchmarks. Performance may vary with implementation details, CUDA configuration, tensor dimensions, data-transfer costs, compiler settings, and hardware.\n\n| Component | CPU | GPU: global memory | GPU: shared memory |\n|---|---|---|---|\n| Encoder | 300 s | 27.492 s | 23.664 s |\n| ConvLSTM | 1,200 s | 909.208 s | 847.260 s |\n| Total runtime | 2,700 s | 1,594 s | 1,481 s |\n\nThe shared-memory implementation reduced encoder runtime from **300 seconds to 23.664 seconds**, producing an approximately **12.68× encoder speed-up**.\n\nFor the complete measured pipeline, runtime decreased from **2,700 seconds to 1,481 seconds**. The total-runtime measurement represents the complete measured pipeline rather than only the encoder and ConvLSTM components shown separately.\n\nThe convolutional encoder benefited strongly from GPU execution because its operations provided substantial data parallelism.\n\nConvLSTM achieved a smaller improvement because of:\n\nConsequently, the **12.68× encoder acceleration** did not translate into a **12.68× end-to-end speed-up**.\n\nAfter the encoder was accelerated, ConvLSTM became the dominant runtime bottleneck.\n\nThis is an important performance-engineering lesson: accelerating one component can move the bottleneck to another part of the system.\n\nThe CUDA implementation was compared with the reference implementation for the tested tensor shapes and configurations.\n\nThe optimisation work focused on reducing execution time while preserving the expected computational behaviour. Model-quality evaluation and runtime evaluation were treated as separate concerns.\n\nSeveral engineering challenges appeared during the implementation.\n\nThe CUDA kernel had to reproduce the same logical output as the reference convolution operation.\n\nA small indexing error could generate incorrect feature maps without necessarily causing a runtime failure.\n\nThe `im2col`\n\ntransformation created additional intermediate data.\n\nThis required careful allocation, reuse, and release of GPU memory.\n\nThreads sharing data through shared memory had to synchronise at appropriate points.\n\nMissing or unnecessary synchronisation could cause incorrect results or reduced performance.\n\nThread-block dimensions and tile sizes affected:\n\nThe extension had to account for tensor shapes, data types, device placement, memory layout, compilation, linking, and output comparison.\n\nImproving one component changed the relative cost of other components.\n\nProfiling therefore had to be repeated after major optimisations.\n\nOnce the encoder was accelerated, ConvLSTM represented a larger proportion of the remaining runtime.\n\n`im2col`\n\nMakes Convolution GPU-Friendly but Introduces Memory Overhead\nThe transformation enables GEMM-based execution while creating a larger intermediate representation.\n\nIts benefit depends on access patterns, tile sizes, synchronisation, and thread-block configuration.\n\nPerformance work should focus on measured bottlenecks rather than assumptions about which component is slow.\n\nGPU output should be compared against a trusted reference implementation before performance improvements are accepted.\n\nDevice handling, tensor shapes, data types, compilation, numerical validation, and fallback behaviour all require careful attention.\n\nMSCRED stands for **Multi-Scale Convolutional Recurrent Encoder-Decoder**.\n\nIt is an architecture designed to model relationships between multiple time-series signals and detect abnormal behaviour through reconstruction error.\n\n`im2col`\n\ndo?\n`im2col`\n\nrearranges overlapping convolution windows into matrix columns, allowing convolution to be expressed as matrix multiplication.\n\nShared memory allowed threads in the same block to reuse data without repeatedly loading it from slower global memory. This reduced memory-access overhead for reusable tiles.\n\nThat was not the purpose of the optimisation, and this article does not claim that it did.\n\nThe best recorded F1 score describes anomaly-detection quality in the thesis experiments. The CPU and GPU timing results describe computational performance. They should be interpreted separately.\n\nAfter the encoder became faster, ConvLSTM represented a larger share of total runtime. The overall system was therefore limited by the remaining bottleneck.\n\nNo.\n\nOptimised PyTorch and CUDA libraries are already highly efficient for many operations. A custom kernel is most useful when:\n\nNo.\n\nThe original dataset contains industrial process information. A public reproduction would require synthetic or appropriately anonymised multivariate time-series data.\n\nNo.\n\nPerformance depends on GPU architecture, CPU configuration, tensor dimensions, batch size, kernel implementation, memory layout, CUDA and PyTorch versions, compiler settings, and workload characteristics.\n\nThe reported numbers apply to the specific experimental setup described in this article.\n\nThe reported results were obtained on a specific hardware configuration and workload.\n\nDifferent GPUs, tensor dimensions, batch sizes, memory layouts, CUDA versions, and kernel configurations may produce different results.\n\nThe original dataset cannot be published because it contains industrial process information. A public reproduction would therefore require synthetic or appropriately anonymised multivariate time-series data.\n\nThis article presents aggregate benchmark measurements and technical methodology rather than the complete industrial dataset or restricted implementation.\n\nFuture work could focus on:\n\nThis work was completed as part of my M.Tech thesis research at the **Indian Institute of Technology Bhilai**.\n\nIt demonstrated how low-level GPU optimisation can be integrated into a deep-learning anomaly-detection pipeline.\n\nThe convolutional encoder benefited substantially from:\n\n`im2col`\n\n-based loweringEncoder runtime decreased from **300 seconds to 23.664 seconds** in the measured workload.\n\nThe broader result also illustrated an important systems principle: once one stage is accelerated, another stage may become the new performance bottleneck.\n\nIn this case, ConvLSTM limited the end-to-end improvement even after the encoder achieved a substantial speed-up.\n\nThe work reinforced that successful performance engineering requires both low-level optimisation and system-level measurement.\n\nZhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., and Chawla, N. V. (2019). **A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data.** *Proceedings of the AAAI Conference on Artificial Intelligence*, 33(01), 1409–1416. [https://doi.org/10.1609/aaai.v33i01.33011409](https://doi.org/10.1609/aaai.v33i01.33011409)\n\nShi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-C. (2015). **Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting.** *Advances in Neural Information Processing Systems 28*, 802–810. [https://proceedings.neurips.cc/paper_files/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html](https://proceedings.neurips.cc/paper_files/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html)\n\nNVIDIA. **CUDA C++ Programming Guide.** [https://docs.nvidia.com/cuda/cuda-c-programming-guide/](https://docs.nvidia.com/cuda/cuda-c-programming-guide/)\n\nPyTorch. **Custom C++ and CUDA Operators.** [https://docs.pytorch.org/tutorials/advanced/cpp_custom_ops.html](https://docs.pytorch.org/tutorials/advanced/cpp_custom_ops.html)\n\nIf this article informs academic or technical work, please cite the original MSCRED and ConvLSTM papers above.\n\nTo refer specifically to my thesis implementation, optimisation process, or reported benchmarks, you may cite this article as:\n\nVutnoor, R. (2026). “GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension.” DEV Community.\n\nIf you found the article useful, a reaction, comment, or constructive technical question on DEV would be appreciated.\n\nI am **Ranjith Vutnoor**, an AI/ML Software Engineer and IIT Bhilai alumnus working on production RAG systems, LLM evaluation, model fine-tuning, PyTorch, and GPU-accelerated machine learning.", "url": "https://wpnews.pro/news/gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension", "canonical_source": "https://dev.to/ranjithvutnoor/gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension-57b2", "published_at": "2026-08-05 09:00:08+00:00", "updated_at": "2026-08-05 09:48:19.692359+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence"], "entities": ["IIT Bhilai", "MSCRED", "ConvLSTM", "CUDA", "PyTorch"], "alternates": {"html": "https://wpnews.pro/news/gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension", "markdown": "https://wpnews.pro/news/gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension.md", "text": "https://wpnews.pro/news/gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension.txt", "jsonld": "https://wpnews.pro/news/gpu-accelerating-mscred-with-cuda-im2col-gemm-and-a-custom-pytorch-extension.jsonld"}}