{"slug": "the-security-ai-that-learns-the-language-of-movement", "title": "The Security AI That Learns the Language of Movement", "summary": "Researchers at the University of North Carolina at Charlotte, led by associate professor Hamed Tabkhi, are developing an AI system that learns normal human movement patterns to detect anomalies in public safety settings, using existing CCTV infrastructure and edge computing on Nvidia Jetson platforms. The system, which converts human poses into motion tokens and uses transformer architectures like SPARTA and Shopformer, aims to flag unusual behavior without relying on predefined threat classes or identifying individuals. The work is supported by the National Science Foundation and is being piloted in Charlotte's city center.", "body_md": "Most security AI is trained to recognize something specific—an intrusion, a weapon, a fall, a fight, or another predefined event. But public safety threats do not always arrive in familiar forms, and collecting enough labeled examples of every dangerous behavior is impractical.\n\nHamed Tabkhi, an associate professor at the University of North Carolina at Charlotte (UNC Charlotte), is taking a different approach. Rather than teaching AI to recognize every possible threat, his team is teaching it what normal human movement looks like in a particular environment. The system can then learn how those movements are likely to continue and flag behavior that departs sufficiently from what it expects.\n\n“We are building an LLM for movements,” Tabkhi told EE Times.\n\nThe analogy is not literal, as the system is not a large language model, but the underlying idea is similar: A language model learns relationships between tokens and predicts what’s likely to come next. Across several research projects, Tabkhi’s group has developed ways to turn human poses and movements into compact representations that AI models can use to learn patterns over time.\n\n[View All](https://www.eetimes.com/category/sponsored-content/)\n\n### **Edge AI on existing CCTV**\n\nTabkhi’s system is designed to work with existing CCTV installations rather than requiring organizations to replace their cameras. A processing device sits inside the closed network and taps video streams from selected IP cameras.\n\nThe incoming pixels are analyzed on the fly, but the AI does not need faces, clothing, skin color, or other identifying information to analyze behavior. Instead, the research converts human activity into representations of motion over time and space.\n\n“We just look at the motions and movements,” Tabkhi said. He described the resulting information as motion heat maps or motion tokens.\n\n[A 2024 paper from Tabkhi and colleagues](https://arxiv.org/abs/2408.15185) introduced a transformer architecture called SPARTA that tokenizes pose information across space and time, allowing the model to capture both the configuration of the body and how that configuration changes. Subsequent work included [Shopformer](https://arxiv.org/abs/2504.19970), a transformer-based AI framework developed by Tabkhi’s group for detecting shoplifting from human pose sequences rather than raw video.\n\nTheir system does not need to know who a person is to reason about how that person is moving.\n\nThe software runs locally on off-the-shelf Nvidia Jetson edge GPU platforms installed inside the CCTV network. Tabkhi said a Jetson-based processing device can connect to multiple cameras without requiring fundamental changes to the existing video infrastructure.\n\nThat makes the architecture a form of edge AI. Instead of continuously sending camera streams to a remote cloud service for analysis, the system can process movement locally and generate anomaly alerts close to where the video originates.\n\nFor Tabkhi, retrofitting existing infrastructure is important. Cameras are already widespread in campuses, city centers, retailers, and other public environments. “Our AI is a complementary layer,” he said.\n\nThe work is supported by the National Science Foundation (NSF), including through the ongoing NSF Civic Innovation Challenge, which enables the team to deploy and evaluate the technology in multiple real-world pilots across Charlotte’s city center.\n\n### **Learning what’s ‘normal’**\n\nThe more radical departure is in how the system decides something is suspicious. Traditional supervised AI starts with predefined classes. If developers want a model to recognize shoplifting, for example, they typically need examples of shoplifting behavior and labels telling the system what those examples represent.\n\nTabkhi argued that this model is poorly suited to many public-safety problems because abnormal events are rare, diverse, and dependent on context. His group instead treats [video anomaly detection](https://arxiv.org/abs/2408.14329) as an open-ended problem in which the AI learns normal behavior and searches for departures from it.\n\nAfter deployment, a model can adapt to the environment in which it operates. A coffee shop, university campus, retail store, or intersection each have different patterns of normal behavior.\n\n“The system learns by itself, so it can adjust itself to the unique environment,” Tabkhi said. “You can move this system to any environment, and it’s going to learn the normal behavior of that environment.”\n\nPrediction is one way the researchers have explored that problem. Their [TSGAD model](https://arxiv.org/abs/2406.15395)—short for two-stream graph-improved anomaly detection—combines learned representations of human movement with trajectory prediction, using differences between expected and observed motion to help identify unusual behavior.\n\nTabkhi put the principle more simply: “We always try to predict the next motion.”\n\nRetail security provides practical proving ground. Tabkhi’s group has increasingly treated shoplifting as an anomaly-detection problem rather than simply another image-classification task. Its [PoseLift work](https://arxiv.org/abs/2501.06591) created a privacy-preserving dataset from real retail environments, representing shoppers through pose information while preserving behavioral signals relevant to shoplifting.\n\nShopformer then applied a transformer specifically to pose sequences for shoplifting detection. More recent research pushes the idea toward continuous real-world deployment.\n\nIn a 2026 paper, the researchers describe a dataset collected across multiple cameras in an operating U.S. retail store, supplemented with actual and staged shoplifting incidents. They also address a major deployment problem: A store does not remain static. Customer patterns, layouts, camera conditions, and other factors change over time.\n\nTo cope with that drift, the researchers developed a [periodic adaptation framework](https://arxiv.org/abs/2603.04723) that learns from new, unlabeled data collected at the site rather than leaving the model frozen after initial training. Earlier work from the group similarly explored continual learning for anomaly-detection systems operating in changing environments.\n\nThat matters because “normal” is not permanent.\n\nThe approach also changes what the analytics layer needs to know about the people being observed. “We don’t do any facial recognition, and we don’t use any personally identifiable information,” Tabkhi said. “We are blind with respect to those details.”\n\nPose-based anomaly detection replaces much of a person’s visual appearance with information about body position and movement. The researchers argue that this can both reduce privacy exposure and limit appearance-related biases that can enter pixel-based systems.\n\nFor example, a person wearing different clothing from everyone around them should not be considered anomalous if that person is moving normally. Removing clothing, facial features, and other appearance information allows the model to concentrate more directly on behavior.\n\n### **AI as an attention mechanism**\n\nAn anomaly does not necessarily mean a crime has occurred. A group suddenly dancing in a coffee shop, for example, would probably violate the system’s expectations. That makes it unusual, not dangerous. Tabkhi therefore does not want the AI to make the final judgment.\n\n“AI is not a decision-maker,” he said. “AI rather is an attention mechanism to enhance and scale human cognition across many cameras.”\n\nWhen the system flags behavior, a human operator can examine the original CCTV footage and decide whether anything requires action. Tabkhi acknowledged that false positives and false negatives cannot be eliminated, making fully automated security decisions inappropriate.\n\nTabkhi’s group also developed [ALFred](https://arxiv.org/abs/2508.09058), an active-learning framework for semi-supervised video anomaly detection that brings human feedback into the learning loop. The system selects uncertain or informative examples for human review, then uses that feedback to refine anomaly labels and adjust detection thresholds as conditions change.\n\nThe objective is not to replace the security operator but to reduce the impossible task of expecting people to watch large numbers of camera feeds continuously.\n\nThat changes the fundamental question that security AI asks. Instead of always trying to determine, “What threat am I looking at?”, Tabkhi’s work increasingly asks something closer to: “Given what has happened so far, what should happen next?”\n\nIf reality breaks the learned pattern, a human gets a chance to look. That may be the most useful language for intelligent cameras to learn.\n\n##### Also read:\n\n[Google’s Marvell Deal Shows Custom Silicon Spreading Beyond the TPU](https://www.eetimes.com/googles-marvell-deal-shows-custom-silicon-spreading-beyond-the-tpu/)\n\n[Qualcomm Bets Open-Source AI Software Can Break Nvidia’s Lock-In](https://www.eetimes.com/qualcomm-bets-open-source-ai-software-can-break-nvidias-lock-in/)\n\n[IBM Makes Quantum Cryogenics Modular, but Scaling Problems Remain](https://www.eetimes.com/ibm-makes-quantum-cryogenics-modular-but-scaling-problems-remain/)\n\n[AI Hardware’s Next Frontier Is Integration](https://www.eetimes.com/ai-hardwares-next-frontier-is-integration/)", "url": "https://wpnews.pro/news/the-security-ai-that-learns-the-language-of-movement", "canonical_source": "https://www.eetimes.com/the-security-ai-that-learns-the-language-of-movement/", "published_at": "2026-09-08 13:15:17+00:00", "updated_at": "2026-09-08 13:29:17.724213+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "ai-research", "ai-products"], "entities": ["University of North Carolina at Charlotte", "Hamed Tabkhi", "SPARTA", "Shopformer", "Nvidia Jetson", "National Science Foundation", "EE Times"], "alternates": {"html": "https://wpnews.pro/news/the-security-ai-that-learns-the-language-of-movement", "markdown": "https://wpnews.pro/news/the-security-ai-that-learns-the-language-of-movement.md", "text": "https://wpnews.pro/news/the-security-ai-that-learns-the-language-of-movement.txt", "jsonld": "https://wpnews.pro/news/the-security-ai-that-learns-the-language-of-movement.jsonld"}}