The IRS needs better and more consistent data documentation for entries in its AI use case inventory, according to a Treasury Inspector General for Tax Administration report published Monday.
In interviews with IRS program managers and data scientists, TIGTA discovered that the agency has been completing data quality checks for its AI use cases, but documentation for each of those use cases in its AI impact assessments has varied.
“For example, 20 percent (1 of 5) of the cases had documents outlining numerous processes and procedures used in testing,” the report said. “The remaining 80 percent (4 of 5) did not have any testing documentation.”
The IRS is far from the only agency that has struggled to fully meet Office of Management and Budget requirements for documenting AI use cases. The handling of agency inventories has been marked for years by lists with missing information, inaccuracies and general inconsistencies.
At the IRS, the Research, Applied Analytics and Statistics unit is responsible for overseeing the compliance data warehouse and the production of the agency-wide AI use case inventory. The division reports directly to the chief data and analytics officer.
RAAS management told TIGTA that it has completed data checks for AI use cases and taken steps to determine whether data was the right “fit” — referring to “the quality and appropriateness of the data.”
The RAAS officials also acknowledged to TIGTA that better documentation of AI use cases — and the repeatability and consistency that comes with it — would likely improve risk management decisions.
“However, they did not prioritize documenting or standardizing baseline procedures or guidelines for evaluating data quality checks before their use in high-impact AI models,” the report noted.
Without better documentation, data quality for AI use cases could be poor, TIGTA added, leading to potential underperformance in AI projects.
“A lack of formal written procedures or guidelines for assessing the quality and appropriateness of the data used in high-impact AI use cases may result in inaccurate, biased, or unreliable AI outputs that can cause significant financial losses and reputational damage,” it said.
The watchdog recommended that the IRS develop and adopt standardized processes to assess data quality for AI use cases, in addition to documenting “the fitness of data.” Agency management said they agreed with the recommendation and have implemented the suggestions.
TIGTA also found improvements to be made in how the IRS implements risk management practices for high-impact AI use cases, and recommended that the agency follow OMB requirements. The IRS agreed and said it would “complete AI impact assessments for currently deployed high-impact AI use cases as of November 2026.”