{"slug": "california-discloses-use-of-six-high-risk-ai-systems", "title": "California Discloses Use of Six High-Risk AI Systems", "summary": "California state agencies are using six high-risk automated decision systems, including tools to predict recidivism, evaluate unemployment-fraud claims, and detect generative-AI use in student assignments, according to a California Department of Technology report. The disclosure comes after agencies reported zero such systems a year earlier, with the technology department attributing the increase to more thorough reviews. The systems have been in use for years, including COMPAS in corrections and an unemployment fraud tool that paused benefits for 600,000 Californians in 2020.", "body_md": "Photo: \ncdn.kqed.org\n \n· rights & takedowns\nAccording to a report from the California Department of Technology, state agencies are currently using \nsix\n automated decision systems the law classifies as \"high-risk,\" after reporting \nzero\n such systems a year earlier (CalMatters). The report lists systems used to predict recidivism, evaluate unemployment-fraud claims, remotely proctor California State University exams, and detect generative-AI use in student assignments (California Department of Technology report; CalMatters). CalMatters reports that some systems have been in use for years, including \nCOMPAS\n in the California Department of Corrections and Rehabilitation, and that an unemployment fraud tool previously paused benefits for \n600,000\n Californians in late 2020. The technology department says it found more systems this year by reviewing agency responses more thoroughly and meeting with agencies (CalMatters).\nWhat happened\nAccording to the California Department of Technology\u0002s annual report, state agencies are currently using \nsix\n automated decision systems that meet the state\u0002s statutory definition of \"high-risk\" (California Department of Technology report; CalMatters). A year earlier, agencies collectively reported \nzero\n high-risk automated decision systems to the state, CalMatters reports. The report lists systems used to predict whether incarcerated people will reoffend, evaluate unemployment-fraud claims, remotely administer exams for California State University students, and detect when students use generative AI in assignments (California Department of Technology report; CalMatters).\nTechnical details\nThe state law the report implements requires agencies to disclose systems \"used to assist or replace human discretionary decisions that have a legal or similarly significant effect,\" including decisions affecting \nhousing\n, \neducation\n, \nemployment\n, \nhealth care\n, and \ncriminal justice\n (CalMatters citing the statute). The technology department\u0002s report also notes that several systems named have been in use for multiple years; CalMatters cites \nCOMPAS\n as an example of a risk-assessment tool used by the California Department of Corrections and Rehabilitation. The report disclosed an additional \nsix\n systems that were initially flagged as high-risk but were later determined not to meet the statutory threshold (California Department of Technology report; CalMatters).\nEditorial analysis - technical context\nIndustry-pattern observations: public-sector inventories of automated decision tools frequently undercount systems on first pass, particularly when agencies use legacy procurement channels or vendor-classified features. Independent reporting and cross-agency review processes, like the meetings the technology department says it held with agencies, commonly raise inventories of known systems.\nContext and significance\nFor practitioners tracking governance and compliance, this disclosure matters because the law creates a formal reporting and oversight flow for systems that produce legally consequential outcomes. Civil rights, privacy, and civil liberties organizations advocated for the law, according to CalMatters, citing evidence that some automated decision tools produce disparate impacts in criminal justice, employment, and benefits adjudication. CalMatters highlights prior incidents linked to state systems, including an unemployment fraud detection process that paused benefits for \n600,000\n Californians around the 2020 holiday season, which underscores why advocates pressed for statutory oversight.\nWhat to watch\nObservational indicators: whether the technology department publicly releases detailed inventories or risk assessments for the six systems named; whether agencies publish impact assessments or mitigation plans as required by the statute; and whether auditors or civil-liberties groups seek access to model documentation and procurement records. Observers will also watch if the state\u0002s deeper review process yields additional retroactive disclosures about legacy systems and vendor contracts.\nLimitations of the record\nCalMatters reports the department\u0002s findings and summarizes the named system use cases; the sources do not include verbatim agency rationales for earlier nonreporting, and the technology department\u0002s full report is the primary document cited for the counts and classifications (CalMatters; California Department of Technology report).\nScoring Rationale\nThis is a notable policy and governance disclosure that affects compliance, procurement, and auditing for ML/AI in the public sector. It is not a frontier technical release but matters to practitioners building or evaluating high-stakes systems.\nPractice interview problems based on real data\n1,500+ SQL & Python problems across 15 industry datasets — the exact type of data you work with.\nTry 250 free problems", "url": "https://wpnews.pro/news/california-discloses-use-of-six-high-risk-ai-systems", "canonical_source": "https://letsdatascience.com/news/california-discloses-use-of-six-high-risk-ai-systems-21565445", "published_at": "2026-06-15 19:36:27.620199+00:00", "updated_at": "2026-06-15 19:36:29.950799+00:00", "lang": "en", "topics": ["ai-policy", "ai-ethics", "ai-safety"], "entities": ["California Department of Technology", "CalMatters", "California State University", "COMPAS", "California Department of Corrections and Rehabilitation"], "alternates": {"html": "https://wpnews.pro/news/california-discloses-use-of-six-high-risk-ai-systems", "markdown": "https://wpnews.pro/news/california-discloses-use-of-six-high-risk-ai-systems.md", "text": "https://wpnews.pro/news/california-discloses-use-of-six-high-risk-ai-systems.txt", "jsonld": "https://wpnews.pro/news/california-discloses-use-of-six-high-risk-ai-systems.jsonld"}}