20 min read
Editor’s note: This is analysis of public legal developments, and is not legal advice. It does not create a lawyer-client relationship. We ask readers to verify every authority against the primary text, especially the EU AI Act dates, which has changed twice in twelve months.
TL;DR #
- The eight-year argument over whether the GDPR contains a right to explanation ended on 27 February 2025, when the CJEU decided(C-203/22).CK v Dun & Bradstreet Austria - The Court told controllers to describe the procedure and principles actually applied, and pointed atcounterfactual informationas a way to get there. Handing over the algorithm does not discharge the obligation. - EU AI Act Article 86 carries its own right to explanation, but it rides on the high-risk regime, and that regime was deferred. - The US has no general right to explanation. Instead, it has four narrower substitutes: ECOA adverse-action notices, employment-discrimination discovery, proposed Federal Rule of Evidence 707, and state disclosure statutes.
- In , a federal magistrate held in late May 2026 that a vendor’s AI bias-testing data was shielded by attorney-client privilege because counsel had curated it. Who commissions the explainability work now decides whether it is discoverable or protected.Mobley v. Workday
Table of Contents #
- TL;DR
- When AI Right to Explanation Stopped Being Academic
- Article 86 and the moving Calendar
- The Four American Substitutes
- Why Explainable AI May Not Satisfy Any of Them
- The Explanation Ladder
- What the Litigation Record Shows
- The Counterargument Worth Taking Seriously
- Where to Start
- Sources and method
- FAQ
- Related reading
When Right to Explanation Stopped Being Academic #
For eight years the literature argued about whether the GDPR contained a right to explanation. Article 15(1)(h) gives data subjects access to meaningful information about the logic involved in automated decision-making, but the phrase sat there without judicial content, and scholars split over whether it reached the reasoning behind a specific decision or only the general workings of a system. The facts that settled it were small. An Austrian customer, referred to in the papers as CK, was refused a mobile phone contract worth ten euros a month after an automated creditworthiness check by Dun & Bradstreet Austria came back negative. The Austrian Data Protection Authority ordered D&B to hand over meaningful information about the logic involved. D&B said that would expose a trade secret, and the enforcement question travelled up to Luxembourg.
On 27 February 2025 the First Chamber held that the controller must describe the procedure and principles actually applied, in a form the data subject can follow, so the person understands which of their own personal data were used and how. The judgment ties that access right back to recital 71 and Article 22(3), the right to contest a solely automated decision, which is the reason the information has to be intelligible rather than merely complete.
Then the Court said it could be appropriate to tell the data subject the extent to which a variation in the personal data taken into account would have produced a different result. That is, in substance, the counterfactual explanation Sandra Wachter, Brent Mittelstadt and Chris Russell proposed in the Harvard Journal of Law & Technology in 2017, under the title “Counterfactual explanations without opening the black box.” A model does not have to be transparent for a controller to say which input change would have flipped the outcome. Eight years after the proposal, a court reached for it as a compliance route.
What it refused to require
Communicating the algorithm itself does not satisfy the obligation. The referring court had an expert who thought the agency should produce the mathematical formula, the values attributed to CK and the intervals those values fell into. The CJEU declined to make that the standard, reasoning that a formula and a table of weights would not give the data subject an intelligible account of anything.
On trade secrets the Court took a procedural route. Where a controller says the required information is protected, it hands that information to the supervisory authority or the competent court, which balances the competing interests and decides what the data subject receives. Trade secrecy stops being an answer the applicant gets and becomes a question a tribunal settles through in camera review. The privilege-style objection that has shielded model internals for a decade now routes through a judge rather than blocking disclosure at the door.
Article 86 and the moving Calendar #
The EU AI Act carries its own right, at Article 86. Where a deployer makes a decision on the basis of output from an Annex III high-risk system, and that decision produces legal effects or similarly significant adverse effects on a person’s health, safety or fundamental rights, the person can obtain a clear and meaningful explanation of the role the AI system played and the main elements of the decision taken. Article 86(3) makes the right residual: it applies only to the extent an equivalent right is not already provided elsewhere in Union law.
Then the Commission published the Digital Omnibus on AI on 19 November 2025, proposing to defer the high-risk obligations because the harmonised standards and national authorities were not ready. Negotiators reached provisional agreement on 7 May 2026, Parliament adopted the text on 16 June 2026 and the Council on 29 June 2026, with signature on 8 July 2026 and entry into force following publication in the Official Journal.
| Obligation | Original date | Date after the Omnibus |
|---|---|---|
| Annex III standalone high-risk systems (employment, education, credit, law enforcement) | 2 August 2026 | 2 December 2027 |
| Annex I high-risk AI embedded in regulated products | 2 August 2027 | 2 August 2028 |
| National regulatory sandboxes | 2 August 2026 | 2 August 2027 |
Article 86 hangs off the Annex III regime, so the practical trigger for most deployers moved with it. The Omnibus staggered the deadlines rather than shifting them as a block, so confirm the exact date against the consolidated text before you advise anyone.
None of this touches the GDPR. Article 15(1)(h) and Article 22 apply now, they applied in 2018, and C-203/22 tells you what they mean. A client waiting for December 2027 to think about explanation is already exposed under a different instrument.
The Four American Substitutes #
There is no general right to explanation in US law. Four narrower mechanisms produce something similar in specific settings, and each one asks for a different artefact.
Credit: adverse-action notices
The Equal Credit Opportunity Act and Regulation B require a creditor taking adverse action to give a statement of specific reasons. The CFPB addressed algorithmic underwriting head-on in Circular 2022-03, taking the position that ECOA and Regulation B do not let creditors use complex algorithms when doing so means they cannot give specific and accurate reasons. Circular 2023-03, in September 2023, added that a creditor may not just pick the closest reason from the sample forms if that reason is not the real one.
Circulars are interpretive guidance, and enforcement priorities shift with administrations. The statute does not. A lender that cannot say why it declined an applicant has an ECOA problem whatever any agency chooses to pursue in a given year.
Employment: discovery does the work
Employment has no notice requirement equivalent to ECOA’s. Explanation arrives through discovery in discrimination litigation, which makes the scope of that discovery the whole game. Mobley v. Workday, Inc., No. 3:23-cv-00770-RFL (N.D. Cal.), is where the boundaries are being drawn, and it gets its own section below.
Evidence: proposed Rule 707
The Advisory Committee on Evidence Rules voted 8 to 1 in May 2025 to publish proposed Rule 707, and released it for comment that August. The draft says that when machine-generated evidence is offered without an expert witness, and would be subject to Rule 702 if a witness testified to it, the court may admit it only if it meets Rule 702(a) to (d). The effect is to run machine output through the Daubert reliability gate rather than letting it in on authentication alone.
The comment period closed on 16 February 2026 and drew real objections about scope. The American Association for Justice argued the draft would sweep in surveillance footage, geolocation data and electronic health records, and urged narrowing it to machine learning specifically. A final report was expected in June 2026. Even on an uninterrupted path the rule would not take effect before 1 December 2027, after Judicial Conference endorsement, Supreme Court approval and congressional review.
Litigators need not wait for it. The reliability arguments Rule 707 would formalise are already available under Rule 702 wherever a human expert leans on model output as their method.
State law: the Colorado reset
Colorado SB 24-205, signed in May 2024, was the first comprehensive US state AI statute, modelled in part on the EU approach: a duty of reasonable care against algorithmic discrimination, impact assessments, and a rebuttable presumption of compliance for deployers following recognised risk frameworks. It never took effect. What happened instead is the clearest warning available against building a compliance programme around a single state.
Colorado, two years and gone
MAY 2024
Governor Polis signs SB 24-205, the first comprehensive US state AI law. Effective date set at 1 February 2026
AUG 2025
A special-session bill pushes the effective date back to 30 June 2026
11 DEC 2025 FEDERAL
White House EO 14365 sets up a DOJ AI Litigation Task Force to challenge state AI laws
APR 2026 CHALLENGE
xAI sues to enjoin the Act. DOJ intervenes in support, the first federal intervention against a state AI law. The AG agrees to suspend enforcement
14 MAY 2026 RESET
SB 26-189 signed, repealing SB 24-205 and replacing it with a disclosure regime. Effective 1 January 2027
The replacement drops the duty of care, the mandatory impact assessments and the risk-framework presumption. In their place sit disclosure obligations, a plain-language explanation of an adverse outcome within thirty days, a right to meaningful human review, and record-keeping, enforced only by the Attorney General after a rulemaking that has to conclude by 1 January 2027. For the scale of the churn underneath all this, legislative trackers counted 1,561 AI-related bills introduced across 45 states in the first quarter of the 2026 session alone.
Why Explainable AI May Not Satisfy Any of Them #
Clients tend to hear “right to explanation” and reach for an XAI vendor. The mismatch is worth understanding before you sign that off as the compliance answer.
NIST set out four principles of explainable AI in NISTIR 8312: a system delivers an explanation; the explanation is meaningful to its audience; the explanation accurately reflects the system’s process; and the system operates only within the conditions it was built for. The second and third principles are separate on purpose. An explanation can be perfectly clear to a loan applicant and still be a poor description of what the model did.
That gap is where the exposure sits. The dominant tools, LIME and SHAP among them, are post-hoc: they build a simpler surrogate around a decision and report which features moved it. NIST’s own survey notes that post-hoc explanations often do not show precisely how a model works. Cynthia Rudin put the sharper version in Nature Machine Intelligence in 2019, arguing that for high-stakes decisions organisations should avoid black-box models altogether unless they can show no interpretable model reaches comparable accuracy.
Set a surrogate’s account against the CFPB’s demand for specific and accurate reasons, or against the CJEU’s demand to describe the procedure and principles actually applied. A surrogate model’s account of a decision is a hypothesis about the decision. Whether that clears “actually applied” has not been tested, and the party relying on it carries the risk.
Counterfactuals sit on firmer ground, which is presumably why the CJEU gestured at them. A statement that the applicant would have been approved at a different debt-to-income ratio can be checked against the deployed system by rerunning it. It makes a claim about the model’s actual behaviour rather than about a simplified stand-in.
The Explanation Ladder #
Five rungs, weakest to strongest. Each has an artefact, the regime that asks for it, and the failure mode that shows up when it is missing. Use it to work out which rung a client is actually standing on before you promise anyone a defence.
The Explanation Ladder
Weakest at the bottom. Most regimes want you three rungs up, and almost nobody keeps rung 5.
Existence
A record that an automated system was used, where, and with what human involvement
GDPR Arts 13 to 15, state disclosure statutesLogic
A plain-language account of the procedure and principles actually applied, and which data feed in
Article 15(1)(h) as read in C-203/22Individual reasons The specific factors that drove this decision for this person, stated accurately not approximately
ECOA and Regulation B, AI Act Article 86Counterfactual
What input change would have flipped the outcome, generated by rerunning the deployed system
The CJEU signposted it. Nobody mandates it yetContestability record
Model version, inputs, output, explanation and any human override, preserved well enough to reconstruct the decision years later
Article 22(3), Colorado’s replacement statute, proposed Rule 707### Rung 1. Existence
Artefact: a record that an automated system was used, at what point, and with what human involvement. Failure mode: the client cannot say whether a given decision was solely automated, which is what decides whether Article 22 applies at all. This is more common than it sounds, especially where a human nominally reviews the output but in practice ratifies it.
Rung 2. Logic
Artefact: a plain-language account of the procedure and principles actually applied, including which categories of personal data enter the system and how they are weighted or combined. Who asks for it: Article 15(1)(h) as construed in C-203/22. Failure mode: the client offers either a marketing description or the source code. The Court rejected the second as insufficient, and the first was never enough.
Rung 3. Individual reasons
Artefact: the specific factors that drove this decision for this person, expressed accurately rather than approximately. Who asks for it: ECOA and Regulation B, and Article 86’s reference to the main elements of the decision taken. Failure mode: a checkbox from a sample form that gestures at the reason. Circular 2023-03 addresses that one directly.
Rung 4. Counterfactual
Artefact: the input change that would have produced a different outcome, generated by rerunning the deployed system rather than a surrogate. Who asks for it: nobody explicitly, yet. The CJEU indicated it may be appropriate, which makes it the safest available way to demonstrate rungs 2 and 3 at once. Failure mode: counterfactuals generated against a stale model version, so the answer no longer describes the system that made the decision. Version pinning matters here.
Rung 5. Contestability record
Artefact: a preserved, time-stamped record of the model version, inputs, output, explanation given and any human override, enough to reconstruct the decision years later. Who asks for it: Article 22(3)’s right to contest presupposes it, Colorado’s replacement statute requires record-keeping, and proposed Rule 707 would need it in substance because reliability cannot be shown without it. Failure mode: the model has been retrained twice since the decision and nobody kept the artefacts, at which point the explanation obligation is not merely unmet but unmeetable.
Run the ladder against the Dun & Bradstreet facts and it maps cleanly. The agency cleared rung 1, since everyone agreed the score was automated. It failed rung 2, which is the reason the case existed at all, because it never gave CK an intelligible account of the procedure and principles applied, and the expert’s offer of the raw formula would have been rung 2 done wrong rather than rung 2 satisfied. The counterfactual the Court signposted is rung 4, and it would have carried rungs 2 and 3 with it.
What the Litigation Record Shows #
Mobley v. Workday is the most instructive US case, because it has run long enough to produce rulings on the questions that bite operationally.
Derek Mobley alleges that AI screening tools in the Workday platform disproportionately rejected applicants who are Black, over forty, or disabled, under Title VII, Section 1981, the ADEA and the ADA. The court declined to treat Workday as an employment agency but let claims proceed on an agency theory, reasoning that customers had plausibly delegated the function of advancing and rejecting candidates to the vendor. Conditional certification of an ADEA collective followed in May 2025, and notice to potential members went out in early 2026.
Then came the ruling that should change how counsel structure this work. In late May 2026, Magistrate Judge Laurel Beeler denied the plaintiffs’ motion to compel production of Workday’s bias-testing data, holding it protected by attorney-client privilege because Workday’s attorneys had curated the data and used the results in giving legal advice (2026 WL 1510537, N.D. Cal.). The same order made Workday’s own EEO-1 and OFCCP filings discoverable, since Workday used internally the tools it sold.
Two practical consequences follow. Bias testing commissioned and directed by counsel may be shielded in a way that engineering-led testing is not. And a vendor that runs its own product on itself can expect its internal employment data pulled into discovery about the product.
The tension underneath is not going away. Privilege protects the candour needed to do bias testing honestly, and it removes the very evidence a claimant would need to prove discrimination. Regulators demanding explanation and courts protecting privileged self-assessment are pulling in opposite directions, and nobody has reconciled them.
The Counterargument Worth Taking Seriously #
The strongest objection is that explanation is the wrong remedy, and that it absorbs effort that should go somewhere else.
An explanation tells a person why a decision was reached. It does not tell them whether the decision was correct, whether the model was accurate for people like them, or whether the outcome was lawful. A model can produce a perfectly explainable decision that is also systematically wrong about a protected group. Outcome testing catches that. Explanation does not, and this is roughly the same trap as mistaking what an AI tool claims it can prove for what it has actually proven.
There is a second-order problem. Explanations can be gamed. A deployer with a plausible-sounding account of a decision is in a stronger defensive position than one without, whether or not the account is faithful to what the model did, and the person on the receiving end usually cannot check. Colorado’s pivot from a duty of care and impact assessments toward disclosure arguably moves in exactly this direction, swapping a substantive standard for a procedural one.
The honest position is that explanation and outcome testing do different jobs. Explanation supports individual contestability, which is what Article 22(3) is for. Statistical validation supports systemic fairness, which is what discrimination law is for. A programme that delivers only the first has met a transparency obligation and left the substantive risk sitting where it was.
Where to Start #
Pick one live automated decision your client makes about individuals and try to answer, today, in writing: which model version produced it, which of that person’s data went in, what would have changed the outcome, and whether the artefacts still exist to prove all three. Most organisations find they can answer the first question and not the other three.
That exercise takes an afternoon and tells you which rung of the ladder the client is on. It is also the cheapest version of the discovery request they will eventually receive. For adjacent evaluations, we have written up a four-question framework for AI contract-review tools and the ownership questions to settle before publishing AI-generated content.
Related reading #
AI contract review for small business, a four-question framework to run before you buy a legal-AI tool.Who owns AI-generated content, the ownership questions to settle before you publish.Why AI-built tools collapse when real users arrive, for teams deploying models into live workflows.AI stock research versus trading claims, on separating what an automated system claims to prove from what it has actually proven, the same gap that makes explanation obligations matter.
FAQ #
Effectively yes, following the CJEU’s judgment of 27 February 2025 in CK v Dun & Bradstreet Austria (C-203/22). The Court held that Article 15(1)(h) requires a controller to describe the procedure and principles actually applied, in a form the data subject can understand, covering which of their personal data were used and how. Disclosure of the algorithm itself was held not to satisfy the obligation.
Article 86 depends on the Annex III high-risk regime, and those obligations were deferred from 2 August 2026 to 2 December 2027 by the Digital Omnibus on AI, adopted by Parliament on 16 June 2026 and the Council on 29 June 2026. High-risk AI embedded in regulated products moves to 2 August 2028. Because the Omnibus staggered rather than uniformly shifted the deadlines, confirm the specific date against the consolidated text.
Not as a general right. Four narrower mechanisms produce something comparable: adverse-action notices under ECOA and Regulation B in credit; discovery in employment discrimination litigation; the reliability gate that proposed Federal Rule of Evidence 707 would apply to machine-generated evidence; and state disclosure statutes, which are currently unstable following Colorado’s repeal and replacement of SB 24-205 in May 2026.
Not as a complete answer to the data subject. In C-203/22 the CJEU held that where a controller claims the required information is protected, it must supply that information to the supervisory authority or competent court, which then balances the competing interests and determines what the data subject receives. Trade secrecy becomes a matter for judicial assessment rather than a unilateral basis for refusal.
This is untested and carries risk. Both are post-hoc methods that approximate a model’s behaviour using a simpler surrogate, and NIST’s NISTIR 8312 treats explanation accuracy as a principle separate from whether an explanation is meaningful to its audience. Where a regime demands the reasons actually relied on, as ECOA does, a surrogate’s account may not qualify. Counterfactual explanations generated by rerunning the deployed system sit on firmer ground.
In practice it is an obligation to produce five artefacts: a record that an automated system was used, an intelligible account of the logic, the specific reasons for the individual decision, ideally a counterfactual showing what would have changed the outcome, and preserved records sufficient to reconstruct the decision later. Different regimes demand different combinations, but the artefacts are cumulative rather than alternative.