{"slug": "can-you-steal-a-robot-s-next-move-by-watching-its-clock-journal-of-our-on-timing", "title": "Can you steal a robot's next move by watching its clock? Journal of our experiments on timing side channels in multi agent RL", "summary": "An independent researcher demonstrated that timing side channels in multi-agent reinforcement learning (MARL) policies deployed on microcontrollers can leak information about an agent's next action. By measuring inference durations on an ESP32-S3, the researcher was able to predict actions without seeing the input observations, highlighting a potential vulnerability for edge AI systems like autonomous drones and warehouse robots.", "body_md": "I have been keenly interested in the intersection of multi-agent reinforcement learning (MARL) and hardware security. When you deploy a trained RL policy onto a microcontroller, the model runs inference to decide what action to take. But here's the thing: different actions can take different amounts of *time* to compute. If an adversary can measure that timing, can they figure out what the agent is about to do without ever seeing the input? This post is about the findings I have found so far in this independent research endeavor.\n\nThe core question is straightforward. A MARL policy, say two cooperative agents navigating a grid, runs on an ESP32-S3 microcontroller. The attacker sits on the outside. They can measure:\n\nThey **cannot** see the raw observations fed to the policy, the internal activations, or the weights. Pure black-box timing. The goal: predict the agent's **action** from timing alone, without ever seeing the observation input.\n\n```\n┌─────────────────────────────────────────────────┐\n│                   ATTACKER                      │\n│  Can observe:                                   │\n│    • Inference duration (total + per-layer)      │\n│    • Number of inferences per timestep           │\n│    • Network packet timing (if WiFi used)        │\n│  Cannot observe:                                │\n│    • Raw observations fed to the policy          │\n│    • Internal activations or weights             │\n│    • Source code (black box timing only)           │\n│                                                 │\n│  Goal: Predict the agent's ACTION from timing   │\n│        without seeing the observation input      │\n└─────────────────────────────────────────────────┘\n```\n\nThis threat model matters because MARL is increasingly being deployed on edge devices like autonomous drones, warehouse robots, and cooperative IoT networks where an adversary with physical proximity could realistically tap into timing signals. If the leakage is real, it's a serious vulnerability: you can predict what an agent will do before it does it.\n\nFor the environments, I set up three test beds with increasing complexity:\n\n**1. Cooperative Grid Navigation (Custom, Primary):** Two agents on a 5×5 grid with swapped goals. They communicate their position and intended direction. There's a wall with a single gap in the middle, so they need to coordinate to avoid collision. The policy is a small MLP: `MLP(9 → 32 → 32 → 5)`\n\nwith 5 discrete actions (stay, up, down, left, right).\n\n```\nA . . . .     2 agents, 5×5 grid, swapped goals\n. . . . .     Comm: position + intended direction\n# # . # #     Shared reward, collision penalty\n. . . . .     \n. . . . B     Policy: MLP(9 → 32 → 32 → 5)\n```\n\n**2. CartPole (Gymnasium Baseline):** The classic single-agent balance task. This serves as a sanity check. It establishes whether timing leakage is a general phenomenon of NN inference on MCUs or specific to multi-agent setups. Policy: `MLP(4 → 32 → 32 → 2)`\n\n.\n\n**3. MPE Simple Spread (PettingZoo):** 3 agents cooperatively covering 3 landmarks. A standard MARL benchmark with a larger policy: `MLP(18 → 64 → 64 → 5)`\n\n. This one hasn't been fully analyzed yet.\n\nFor both grid_nav and CartPole, the policies are trained using PPO (and IPPO for multi-agent), then exported to TFLite (both FP32 and INT8 quantized variants) for deployment. The export pipeline goes PyTorch → Keras → TFLite, and also generates a `.h`\n\nC header for embedding directly into the ESP32 firmware.\n\nThe full analysis pipeline I built goes from training all the way through to leakage quantification:\n\n```\nTrain Policy → Export TFLite → Flash ESP32-S3 → Collect Timing → Analyze Leakage → Generate Report\n```\n\nThe analysis module is the core of the project. For each model configuration, it computes:\n\n`MI(timing; action)`\n\n: how many bits of action information leak through timing, with bootstrap confidence intervalsIf the classifiers beat random baseline by a significant margin, the system is operationally vulnerable: an attacker can build the same model.\n\nBefore touching real hardware, I validated the pipeline with simulated timing. The `SimulatedCollector`\n\nruns real TFLite inference on the laptop but generates synthetic cycle counts that are intentionally action dependent: `base_cycles + action × bias + noise`\n\n. This confirms that when there *is* leakage baked in, the analysis correctly detects it.\n\nThe results across 2000 simulated traces per configuration:\n\n| Config | MI (bits) | RF Accuracy | KW p-value | Max Cohen's d |\n|---|---|---|---|---|\n| cartpole_fp32 | 0.011 | 99.5% | 1.98e-9 | 0.274 |\n| cartpole_int8 | 0.022 | 99.8% | 1.71e-9 | 0.276 |\n| grid_nav_fp32 | 0.976 | 99.8% | 1.46e-281 | 8.879 |\n| grid_nav_int8 | 1.005 | 100.0% | 1.15e-286 | 8.838 |\n\nA few observations jump out immediately:\n\n**The grid_nav environment leaks significantly more than CartPole.** Grid navigation has 5 actions with MI near 1 bit (out of a max of ~2.32 bits for 5 actions), while CartPole's MI hovers near 0. This makes sense: the simulated timing injects `action × 200`\n\ncycles of bias, and with 5 spread out actions, there's more room for distinct timing signatures than CartPole's 2 actions.\n\n**Classifiers hit near perfect accuracy.** Both RF and MLP classifiers achieve 99.5%+ accuracy on the simulated data, far exceeding the 20% random baseline for 5 actions and 50% for 2 actions. This demonstrates that even simple models can operationally exploit timing leakage.\n\n**The Kruskal-Wallis p values are absurdly small.** Values like 1.46e-281 are effectively zero. The null hypothesis (timing distributions are the same across actions) is annihilated. Combined with Cohen's d values above 8 for grid_nav (anything above 0.8 is conventionally \"large\"), this signals massive, unmistakable effect sizes.\n\n**INT8 quantization doesn't suppress leakage.** In fact, the INT8 variants show slightly *higher* MI and comparable classifier accuracy. This is notable because quantization changes the computational profile of each operator, and we had hypothesized it might reduce timing variance by simplifying the computation paths. At least in simulation, it doesn't.\n\nBeyond total inference time, I also measured per operator cycle counts (5 simulated operators: two FullyConnected, two activation, one Softmax). The per operator MI analysis revealed that the leakage isn't uniform. Specific operators (the FullyConnected layers and the final Softmax) carry most of the timing signal, while activation functions contribute very little.\n\nThis has practical implications: if you wanted to *defend* against timing side channels, you'd focus your constant time countermeasures on the matmul and softmax operators specifically, rather than trying to make the entire inference pipeline constant time (which is much harder).\n\nUsing Spearman correlation between individual observation dimensions and total timing, I found statistically significant correlations in multiple dimensions. This means timing doesn't *only* leak the output action, it also leaks information about the *input* observations. In a MARL context, this is arguably worse: an adversary could potentially reconstruct features of the agent's perceived state, not just its decision.\n\nThe ESP32-S3 firmware is written in C/C++ with ESP-IDF and TFLite Micro. The key features:\n\n`esp_cpu_get_cycle_count()`\n\nat 240MHz (4.17ns resolution)`MicroProfiler`\n\nto attribute timing to individual operators (FullyConnected, ReLU, Softmax)`(action, total_cycles, per_op_cycles[])`\n\nThis is fully built and compiles, but the actual on device data collection is the next step. This will be covered in the next part.\n\nThis is an ongoing experiment. This blog serves as a journal of my progress so far and will continually be updated once new findings are found. A few concrete next steps:\n\nHopefully we'll be able to get a paper through this independent endeavor.", "url": "https://wpnews.pro/news/can-you-steal-a-robot-s-next-move-by-watching-its-clock-journal-of-our-on-timing", "canonical_source": "https://dev.to/skondho_kata/can-you-steal-a-robots-next-move-by-watching-its-clock-journal-of-our-experiments-on-timing-side-1k5f", "published_at": "2026-08-10 02:46:47+00:00", "updated_at": "2026-08-10 03:15:03.458938+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-safety", "ai-ethics"], "entities": ["ESP32-S3", "PPO", "IPPO", "TFLite", "Gymnasium", "PettingZoo"], "alternates": {"html": "https://wpnews.pro/news/can-you-steal-a-robot-s-next-move-by-watching-its-clock-journal-of-our-on-timing", "markdown": "https://wpnews.pro/news/can-you-steal-a-robot-s-next-move-by-watching-its-clock-journal-of-our-on-timing.md", "text": "https://wpnews.pro/news/can-you-steal-a-robot-s-next-move-by-watching-its-clock-journal-of-our-on-timing.txt", "jsonld": "https://wpnews.pro/news/can-you-steal-a-robot-s-next-move-by-watching-its-clock-journal-of-our-on-timing.jsonld"}}