# Semgrep Multimodal Goes Beyond Authentication: What we learned from Comparing it with Mythos > Source: > Published: 2026-08-27 00:00:00+00:00 Semgrep Multimodal Goes Beyond Authentication: What we learned from Comparing it with Mythos Semgrep Multimodal found 63 manually reviewed IDOR vulnerabilities that Mythos missed. Brenden Noblitt Jaweed Metz Seth Jaksik August 27th, 2026 We ran Semgrep Multimodal and Mythos in Claude Security against the same four repositories at identical source revisions. Semgrep Multimodal reported 63 IDOR vulnerabilities that Mythos did not identify, and manual review confirmed every one. Forty of those findings appeared in all three Semgrep Multimodal runs, indicating that most of the difference achieved was reproducible rather than a lucky pass. These were not unauthenticated routes; they were cases where the application recognized the caller but failed to verify that the caller was authorized for the specific object being accessed or modified. Benchmarking Semgrep Multimodal, Mythos, and Codex Security We evaluated the systems against 275 manually reviewed positive and negative IDOR labels to measure performance using precision, recall, and F1. Precision measures how often a reported finding is a genuine vulnerability rather than a false positive. Recall measures how many of the benchmark’s known vulnerabilities a system identifies. F1 balances precision and recall in a single score. Both precision and recall matter in security analysis. High precision helps you keep your backlog manageable and noise down, whereas high recall reduces the likelihood of real vulnerabilities remaining undetected. Semgrep Multimodal achieved 59.9% recall, compared with 13.9% for Claude Security with Mythos and 11.3% for Codex Security. In other words, Semgrep Multimodal identified more than four times as many of the known IDORs as Mythos and more than five times as many as Codex Security. Conversely, Mythos achieved higher precision: 80.1% compared with 57.5% for Semgrep Multimodal. However, Semgrep Multimodal’s higher recall led to a substantially higher reported F1 score of 57.1%, compared with 23.7% for Mythos and 17.7% for Codex Security. The largest difference was recall: Mythos returned a smaller, more selective set of findings, whereas Semgrep Multimodal identified more of the benchmark’s known vulnerabilities. These results suggest that Semgrep Multimodal provided materially deeper coverage, while Mythos favored precision over completeness. How Semgrep Multimodal Finds These: Rule-Based Analysis Plus Reasoning Multimodal runs two kinds of analysis against the same code. Rule-based dataflow tracks where a caller-controlled value travels: which parameter enters the endpoint, which service method receives it, which query it ends up in. AI reasoning handles the part a rule can't express, which is whether the authorization check sitting on that path actually covers the object the query touches. Neither mode gets there alone. Dataflow can follow a request parameter all the way into a database write, but it has no way to judge whether the authorization check it passed along the way was about that same object. A model can make that judgment by reading the code, but only if it is reading the right handful of functions out of a repository with thousands. The rules narrow the search. The reasoning judges what turns up. Focused benchmarks are useful because broad vulnerability coverage does not always show how well a system can reason deeply about a specific vulnerability class. We discuss this further in our model-groundedness blog post. IDORs Are Authorization Bugs, Not Authentication Bugs An Insecure Direct Object Reference, or IDOR, occurs when an application allows a caller to access or modify an object without verifying that the caller is authorized for that specific object. IDORs are often illustrated with a simple test: change an object identifier in a request and observe whether another user’s data is returned. In real applications, however, the same failure may be hidden inside authenticated endpoints, service layers, parent-child relationships, and state-changing workflows. Authentication tells an application who is making the request. It does not establish that the caller is allowed to access or modify every object named in that request. That distinction is what makes IDORs difficult to find. The endpoint may have an authorization check, but the check may apply to the wrong object. As an example: Each step may look reasonable in isolation. The application verifies that the caller can access the parent, then deletes a child by its identifier. However, unless the code also verifies that child_id belongs to parent_id, the authorization decision does not protect the object being deleted. The same pattern appears in read operations, updates, workflow transitions, and shared storage. A service may confirm that an object exists or that it is in the right state without checking who owns it. A controller may pass the current username through the service layer for auditing without using it to constrain the database query. A codebase may even contain both a user-aware service method and an unsafe raw-ID overload, with the endpoint calling the latter. The central question is: “Does the authorization decision protect the same object that reaches the sensitive operation?” Answering this question requires tracing the request from the endpoint, through the service layer, and into the final database operation. That is why these bugs are often endpoint-level problems rather than broad subsystem-level failures. The application may be secure in most of the surrounding subsystem while one specific route applies authorization to the wrong object, or fails to apply it at all. Why These IDORs Are Easy to Miss The hardest IDORs do not omit an authorization check entirely. They may contain a check that is incomplete, indirect, or applied to the wrong object. A typical request dataflow looks something like this: ``` caller-controlled identifier | v object lookup | v authorization decision | v sensitive operation ``` The vulnerability appears when those steps do not refer to the same object. A controller might authorize a parent while a service mutates a child. A service might check that a record exists without checking who owns it. A request might carry a task identifier into a state-changing workflow without verifying that the current user is assigned to that task. These bugs are also easy to hide behind ordinary application structure. The endpoint is authenticated. The service method is valid. The database query returns the requested record. The current username may even be passed through the call chain and recorded for auditing. But unless that identity is used to constrain the lookup or mutation, it does not provide authorization. Finding the bug requires following the specific request path far enough to answer four questions: Which value does the caller control? Which object does that value identify? Which object does the authorization check cover? Which object reaches the final read, write, delete, or state transition? The code may only span a few functions, but understanding the relationship requires deep semantic reasoning. The recurring Semgrep Multimodal findings came from exactly these mismatches. What the Comparison Revealed The manually reviewed Semgrep Multimodal findings were not random misses scattered across unrelated code. They clustered around a few recurring authorization patterns. Parent objects being authorized while child objects were mutated independently. Workflow actions operating on tasks or processes without checking ownership or assignment. In each case, the endpoint accepted a caller-controlled identifier and passed it through the application until it reached a database operation. The application performed some validation along the way, but never established that the caller was authorized for the specific object represented by that identifier. The first pattern looks like an ordinary authenticated list or lookup endpoint: ``` authenticated request | v group_id, object_id, or subobject_id | v controller and service layer | v database query filtered by the supplied identifier ``` The application confirms that the request is valid and that the referenced object exists. What it does not confirm is whether the current user belongs to the relevant group, owns the object, or has permission to view its contents. Several layers of valid code can obscure this failure mode. The controller may correctly require authentication. The service is called with a valid identifier. The database query returns exactly what was requested. The vulnerability is that the caller can select a record without the query constraining the selection to the caller’s authorized scope. The missing check is not “is this user logged in?” It is “does this object belong to, or fall within the authorized scope of, this user?” Parent Authorization with an Unscoped Child Mutation The second pattern occurs when the application authorized a parent object but performed the sensitive operation on a child object. ``` authorize(parent_id, current_user) | v load child by child_id | v update or delete child ``` The authorization check creates a reassuring impression: the caller is verified against a legitimate parent resource. But if the child lookup uses only child_id, the application never proves that the child belongs to the authorized parent. An attacker may be able to provide a valid parent identifier they are allowed to access, then substitute a child identifier belonging to another user or tenant. The result can be unauthorized data access or a state-changing operation against an object outside the caller’s scope. This is a relationship failure. The application checks whether the caller can access parent_id, but the operation ultimately trusts child_id. The relationship should be enforced when the child is retrieved or modified: ``` authorize(parent_id, current_user) | v load child where child_id = ? AND parent_id = ? | v update or delete child ``` This ensures that the authorization decision and sensitive operation refer to the same object relationship. Workflow Operations Without Ownership or Assignment Checks The third pattern appears in endpoints that change the state of a task, process, job, or other business object. ``` POST /startProcess/{id} | v load process by id | v change process state ``` The endpoint may require authentication and may validate that the process exists or is in a startable state. But those checks do not establish that the current user owns the process, is assigned to it, or has permission to trigger the transition. This turns a seemingly ordinary workflow operation into an IDOR. The attacker does not need to bypass the login requirement. They only need to provide an identifier for an object they should not control. Workflow IDORs are especially important because the impact is not limited to reading data. An unauthorized state transition can approve, start, cancel, delete, assign, or otherwise alter a business process on behalf of another user. Across all three patterns, the endpoint has a security boundary. The failure is that the boundary stops at authentication, a parent resource, or a workflow-state check before reaching the specific object being operated on. Why Broader Coverage Matters IDORs are often missed not because an application lacks authentication, but because authorization stops before reaching the specific object being accessed or modified. Across the examples in this evaluation, Semgrep Multimodal identified these relationship failures by tracing caller-controlled values through controllers, service layers, object lookups, and final operations. Its combination of program analysis and AI reasoning helped surface vulnerabilities that were distributed across multiple functions or hidden behind otherwise valid authorization checks. The benchmark results reinforce that pattern. Across 275 manually reviewed IDOR locations, Semgrep Multimodal achieved substantially higher recall and F1 than Mythos or Codex Security. Mythos produced a more selective set of findings with higher precision, but Semgrep Multimodal identified far more of the benchmarks known vulnerabilities. Precision remains essential in security: teams need findings that they can trust and review efficiently, but a clean findings list is not sufficient if it omits most of the vulnerabilities in scope. Effective security analysis should balance accuracy of reported findings with the ability to uncover vulnerabilities across a wide array of application paths. Semgrep Multimodal combines program analysis with AI-powered reasoning to uncover complex vulnerabilities that traditional approaches can miss. See what Semgrep Multimodal can find in your code. Dive deeper into Security Research or continue reading our featured posts.