Photo: cdn.kqed.org
· rights & takedowns According to a report from the California Department of Technology, state agencies are currently using six automated decision systems the law classifies as "high-risk," after reporting zero such systems a year earlier (CalMatters). The report lists systems used to predict recidivism, evaluate unemployment-fraud claims, remotely proctor California State University exams, and detect generative-AI use in student assignments (California Department of Technology report; CalMatters). CalMatters reports that some systems have been in use for years, including COMPAS in the California Department of Corrections and Rehabilitation, and that an unemployment fraud tool previously d benefits for 600,000 Californians in late 2020. The technology department says it found more systems this year by reviewing agency responses more thoroughly and meeting with agencies (CalMatters). What happened According to the California Department of Technologys annual report, state agencies are currently using six automated decision systems that meet the states statutory definition of "high-risk" (California Department of Technology report; CalMatters). A year earlier, agencies collectively reported zero high-risk automated decision systems to the state, CalMatters reports. The report lists systems used to predict whether incarcerated people will reoffend, evaluate unemployment-fraud claims, remotely administer exams for California State University students, and detect when students use generative AI in assignments (California Department of Technology report; CalMatters). Technical details The state law the report implements requires agencies to disclose systems "used to assist or replace human discretionary decisions that have a legal or similarly significant effect," including decisions affecting housing , education , employment , health care , and criminal justice (CalMatters citing the statute). The technology departments report also notes that several systems named have been in use for multiple years; CalMatters cites COMPAS as an example of a risk-assessment tool used by the California Department of Corrections and Rehabilitation. The report disclosed an additional six systems that were initially flagged as high-risk but were later determined not to meet the statutory threshold (California Department of Technology report; CalMatters). Editorial analysis - technical context Industry-pattern observations: public-sector inventories of automated decision tools frequently undercount systems on first pass, particularly when agencies use legacy procurement channels or vendor-classified features. Independent reporting and cross-agency review processes, like the meetings the technology department says it held with agencies, commonly raise inventories of known systems. Context and significance For practitioners tracking governance and compliance, this disclosure matters because the law creates a formal reporting and oversight flow for systems that produce legally consequential outcomes. Civil rights, privacy, and civil liberties organizations advocated for the law, according to CalMatters, citing evidence that some automated decision tools produce disparate impacts in criminal justice, employment, and benefits adjudication. CalMatters highlights prior incidents linked to state systems, including an unemployment fraud detection process that d benefits for 600,000 Californians around the 2020 holiday season, which underscores why advocates pressed for statutory oversight. What to watch Observational indicators: whether the technology department publicly releases detailed inventories or risk assessments for the six systems named; whether agencies publish impact assessments or mitigation plans as required by the statute; and whether auditors or civil-liberties groups seek access to model documentation and procurement records. Observers will also watch if the states deeper review process yields additional retroactive disclosures about legacy systems and vendor contracts. Limitations of the record CalMatters reports the departments findings and summarizes the named system use cases; the sources do not include verbatim agency rationales for earlier nonreporting, and the technology departments full report is the primary document cited for the counts and classifications (CalMatters; California Department of Technology report). Scoring Rationale This is a notable policy and governance disclosure that affects compliance, procurement, and auditing for ML/AI in the public sector. It is not a frontier technical release but matters to practitioners building or evaluating high-stakes systems. Practice interview problems based on real data 1,500+ SQL & Python problems across 15 industry datasets — the exact type of data you work with. Try 250 free problems