DataFlowBench ranks eight static analyzers for CI, editors, security review, and low-noise workflows, including licensing trade-offs.
In the previous post, we looked at DataFlowBench's taint-analysis results one dimension at a time. Here, we ask what happens when you care about several at once: which trade-offs suit CI, an editor or agent loop, a security review, or a low-noise workflow?
A quick refresher: precision measures the share of reported flows that are real, recall measures the share of real flows found, and decisiveness measures how often the analyzer reaches a decision. Responsiveness captures the timings of the benchmark workloads.
From dimensions to rankings for your use case #
Different jobs need different rankings. Of course we want Bifrost to be #1 for everything, but your priorities get a vote too.
We use four profiles to reflect those priorities, weighting precision, recall, decisiveness, and responsiveness differently for each. Generalists and specialists are ranked separately.
Language coverage is excluded: these profiles use covered-kernel decisiveness — the share of assertions answered within each analyzer’s benchmarked kernels. Full-corpus decision coverage is not part of the weighted score. Responsiveness is log-normalized within each cohort, so scores should not be compared between generalists and specialists.
Among the generalists, CodeQL leads our CI and security-analyst profiles: its recall and covered-kernel decisiveness outweigh its latency. Joern is close behind, with Bifrost trailing CodeQL by 1.2 and 2.1 points, respectively, using the rounded scores shown.
Bifrost leads the in-editor profile on responsiveness and the precision-first profile with no false positives in this corpus. Semgrep CE places fourth across all four profiles; its speed is not enough to offset its other scores.
Within the specialist cohort, FlowDroid leads the CI, in-editor, and security-analyst profiles, balancing precision, recall, complete decisions, and the fastest specialist timings. Infer comes second in those profiles and leads precision-first with 98.4% precision.
OpenTaint's comparatively strong recall is most useful to the security-analyst profile, but lower precision and slower execution keep it from leading; Pysa shows the inverse tendency, pairing high precision with weaker recall. These rankings apply only within each tool's intended and benchmarked domain.
A specialist placing first here should (obviously) not be interpreted as having general-purpose language coverage.
What are the legal requirements for the tool that fits your use case? #
Not every tool on this list is free for commercial purposes. Some may require notices at a minimum, while others are free for research but need licenses for enterprise use.
Licensing also affects what we can benchmark. For example, Semgrep CE is fair game to run locally and benchmark, but the Pro edition (which offers interprocedural flows) requires permission for public benchmarks. So maybe it'll be included in future DataFlowBench results, maybe it won't — no promises.
Thankfully, seven of the eight tools we've shown here permit free private and commercial analysis, subject to their license terms. CodeQL requires a paid license for private-code analysis. Unfortunately for the budget, it's pretty good... but so are many of the others depending on what you care about.
Referring back to OpenTaint versus FlowDroid, if you're interested in redistribution or having closed mods when shipped, this may change your mind.
Finally, I should say that this is a practical license review, not legal advice. The linked license text and any applicable commercial agreement are up to you to review if you launch the next kick-ass SAST product with any of these at the core. If you do, take a look at the Bifrost extension template*!*
Reading the ranking responsibly #
There is no best taint analyzer, only the best one for the job. Query languages and day-to-day usability deserve a discussion of their own.
Our profiles favor CodeQL for CI, but for in-editor use, Bifrost leads the generalists by a wide margin. FlowDroid and Infer are worth considering as specialist alternatives where their language coverage fits your codebase. Legal obligations may shuffle this around slightly.
The benchmark is a starting point. Check language support, maintenance, licensing, and integration options. Check MCP support too if you want an analyzer in an agent loop. If false positives are a deal-breaker, look at precision alongside decision coverage. A tool can be confidently wrong.
The timing results describe small benchmark cases. They do not show how runtime or memory use scales with repository size, and a tool that is fast within its cohort may still be too slow for your interactive workflow. Treat these profiles as a shortlist to test on your own codebase.