cd /news/machine-learning/sensitivity-analysis-of-gru-lstm-and… · home topics machine-learning article
[ARTICLE · art-84165] src=arxiv.org ↗ pub= topic=machine-learning verified=true sentiment=· neutral

Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

A study by researchers evaluating GRU, LSTM, and Transformer encoder models for classifying Level 2 automated driving systems (Comma Openpilot, Tesla Autopilot, Cadillac Super Cruise) from vehicle telematics data found that all three achieve strong clean-data performance with macro F1-scores of 0.92, 0.90, and 0.93, respectively, but temporal jitter collapses macro-F1 to 0.44-0.50 across all models, revealing a critical vulnerability in real-world telematics degradation.

read1 min views1 publishedAug 3, 2026

arXiv:2607.28665v1 Announce Type: new Abstract: Automated driving systems (ADSs) are becoming ubiquitous. Future Software Defined Vehicles (SDVs) may be able to run multiple ADSs, both native and aftermarket such as Comma.ai's Openpilot. Monitoring systems to independently verify which automated driving system is active are important for safety monitoring, regulatory compliance, insurance assessment, and anomaly detection. In this paper, we first evaluate the effectiveness of three sequence-based classification models: Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) networks, and a Transformer encoder model for identifying Level 2 automated driving systems using vehicle telematics data alone: Comma Openpilot, Tesla Autopilot, and Cadillac Super Cruise, along with manual driving. All three models achieve strong clean-data performance with macro F1-scores of 0.92 (GRU), 0.90 (LSTM), and 0.93 (Transformer encoder model) when trained on clean data; threat-matched training yields 0.904-0.916 macro F1 with only a modest clean-data penalty. Second, we introduce a modular robustness evaluation framework that simulates realistic telematics degradation through five corruption families at five severity levels (L1-L5). Continuous channels are perturbed using additive white Gaussian noise with cumulative drift, correlated cross-channel noise, and temporal jitter. Binary event signals are subjected to burst loss, delayed transitions, spurious toggles and cross-feature inconsistencies inspired by communication errors. Robustness is measured using macro-F1, which gives equal weight to each class and is suitable for imbalanced multiclass evaluation. Our evaluation reveals a sharp failure-mode split: event-level corruptions reduce macro-F1 only slightly (greater than equal to 0.87 at L5), while temporal jitter collapses macro-F1 to 0.44-0.50 across GRU, LSTM, and Transformer encoder model.

── more in #machine-learning 4 stories · sorted by recency
── more on @comma.ai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/sensitivity-analysis…] indexed:0 read:1min 2026-08-03 ·