Building Road Infrastructure Digital Twins with Computer Vision & Deep Learning

By Dr. Mohammad Amin Talaghat (Ph.D. in Construction Management | AI & Digital Twin Researcher)

Modern transportation infrastructure demands proactive, data-driven pavement management systems (PMS). Traditional manual condition surveys are not only labor-intensive, costly, and hazardous, but also suffer from human subjectivity and low spatial frequency.

In recent research collaborations with the DigiPave project at the University of Oulu (Finland), we have been developing and optimizing end-to-end digital twin architectures for continuous road surface inspection, automated defect recognition, and life-cycle maintenance optimization.

In this article, I share the core architectural pillars, deep learning models, and practical integration workflows that bridge Computer Vision (YOLO/SAM), Geographic Information Systems (GIS), and Building Information Modeling (BIM) to create a true Digital Twin for civil infrastructure.

An effective infrastructure digital twin is not just a 3D visualization—it is a closed-loop cyber-physical system composed of four foundational layers:

Accurate pavement distress assessment requires distinguishing between various distress classes:

Using Ultralytics YOLOv8 segmentation in Python, we extract both spatial bounding boxes and exact polygon contours to calculate surface defect area:

import cv2
import numpy as np
from ultralytics import YOLO

model = YOLO('best_road_defect_segmentation.pt')

def process_inspection_frame(frame, conf_threshold=0.35):
    results = model(frame, conf=conf_threshold)[0]
    defect_summary = []

    if results.masks is not None:
        for idx, (box, mask) in enumerate(zip(results.boxes, results.masks.xy)):
            cls_id = int(box.cls[0].item())
            class_name = model.names[cls_id]
            confidence = float(box.conf[0].item())

            contour = np.array(mask, dtype=np.int32)
            pixel_area = cv2.contourArea(contour)

            defect_summary.append({
                'class': class_name,
                'confidence': round(confidence, 3),
                'contour_points': len(contour),
                'pixel_area': pixel_area
            })

    return results.plot(), defect_summary

Under ASTM D6433, raw detections must be converted to quantifiable engineering indices. For each inspection sample unit:

$$\text{PCI} = 100 - \text{CDV}$$

Automating this pipeline with Python transforms raw mobile video recordings into interactive engineering dashboards within minutes.

I have built several open-source tools, scientific articles, and interactive demos bridging artificial intelligence with structural engineering, contracts, and digital twins:

Feel free to leave your thoughts, questions, or collaboration ideas below!