# Claude 3.5 Sonnet vs. GPT-4o: Legal Reasoning Benchmarks

> Source: <https://promptcube3.com/en/threads/2955/>
> Published: 2026-07-24 23:47:35+00:00

# Claude 3.5 Sonnet vs. GPT-4o: Legal Reasoning Benchmarks

[Claude](/en/tags/claude/)3.5 Sonnet and GPT-4o to the test against a complex inheritance dispute involving 19th-century housing cooperative statutes. The goal wasn't just to "write a letter," but to perform a precise cross-reference of contradictory legal provisions.

## The Technical Challenge: Dirty Data

The input was a nightmare: a "PDF" that was actually a zip of images with no text layer. To make this a real-world AI workflow, I had to preprocess the data:

1. Unzip the image archive.

2. Rasterize and run OCR via a German-specific model.

3. Feed the resulting imperfect text (where the section symbol § was often misread as $) into the LLM.

In 2019, this would have been a dead end. Today, the "noise" in the OCR didn't stop the reasoning, provided the model had a large enough context window to hold the entire set of bylaws and the correspondence.

## Model Performance: Claude 3.5 Sonnet vs. GPT-4o

I ran the same set of documents through both models to see which could actually spot the legal error. The error was subtle: the cooperative was applying rules for *inter vivos* transfers (between living people), which require board approval, rather than *mortis causa* transfers (inheritance), which happen automatically by operation of law.

**Reasoning Accuracy:** Claude 3.5 Sonnet identified the specific paragraph conflict immediately. It mapped the "transfer" clause against the "inheritance" clause and flagged the contradiction. GPT-4o tended to summarize the bylaws generally but missed the specific nuance of the "automatic" nature of the inheritance.**Citation Precision:** Claude provided a reference document mapping every legal provision in play. It cited four specific statutes and three bylaw sections with near-perfect accuracy despite the OCR errors.**Hallucination Rate:** GPT-4o occasionally "invented" a standard probate procedure that didn't exist in the provided bylaws, whereas Claude stuck strictly to the provided context.**Output Utility:** Claude generated a formal letter that didn't sound like "AI prose." It used the specific terminology of the bylaws to corner the administrator into a logical dead end.

## Practical Implementation: The Prompt Strategy

To get this result, you cannot use a simple "summarize this" prompt. You need a multi-step prompt engineering approach to force the model into a "legal auditor" persona.

```
# System Prompt
You are a senior legal analyst specializing in cooperative law. Your goal is to find contradictions between the provided Bylaws and the Correspondence.

# Task
1. Extract all mentions of "share transfer" and "inheritance/death" from the Bylaws.
2. Compare these provisions against the claims made in the Correspondence.
3. Identify if the Correspondence is applying a provision that is superseded by another.
4. Provide a list of citations (Section/Paragraph) to support your finding.

# Constraint
If the OCR text is corrupted (e.g., '$' instead of '§'), infer the meaning from context but flag the correction.
```

## The Verdict

For high-stakes reasoning where a single paragraph determines the outcome, Claude 3.5 Sonnet is currently superior to GPT-4o. The ability to handle "garbage" OCR text and perform a side-by-side comparison of conflicting rules makes it a legitimate tool for personal legal auditing.

The result of this specific deep dive was a two-page letter that proved the cooperative had applied the wrong paragraph. I didn't need a law degree; I just needed a model that could actually read.

[Next Jixp: Lisp DSL for Jax Neural Nets →](/en/threads/2937/)