{"slug": "llm-friendliness-as-a-metric-porting-20-languages-with-an-llm", "title": "LLM-friendliness as a metric: porting 20 languages with an LLM", "summary": "A developer ported a 51-test Crystal benchmark suite to 19 other programming languages using an LLM over roughly two months, publishing the performance and memory results at LangArena. The developer reports that an Expressiveness metric — where Crystal requires 44.8% less code than average and Zig requires 176.2% more — correlated almost perfectly with how easy each language was to port with an LLM, with Scala, Nim and Go among the easiest and Odin, C and Zig noticeably harder. Zig proved especially difficult because the LLM generated code for an older API version that looked correct but would not compile.", "body_md": "I recently completed a large experiment: I took a benchmark suite written in Crystal (51 tests covering sorting, parsing, algorithms, compression) and ported it to 19 other programming languages with the help of an LLM. The process took about two months. My original goal was to compare performance and memory usage across languages — and the results are available at [LangArena](https://kostya.github.io/LangArena/).\n\nAlong the way, I collected a lot of data — code size, compilation times, and so on. One of the tables I put together was an **Expressiveness** metric. After a while, I realized this metric was unexpectedly revealing about the experience of working with different languages.\n\n**Table legend**\n\nThis table compares how concisely different programming languages express the same program.\n\nAt first glance, Expressiveness looks like a simple measure of code size. The higher the percentage, the fewer lines you need to write. Crystal requires 44.8% less code than the average, while Zig requires 176.2% more.\n\nBut if you look closer, a question emerges: are we measuring brevity or expressiveness? Brevity is about the number of characters. Expressiveness is about how clearly an idea is conveyed. They are not the same thing.\n\nThe table includes a column called **Boilerplate** — the ratio of raw source code size to its gzipped size. Gzip compresses repeated patterns. If a language forces you to write the same structures over and over, gzip will squeeze them, but you still had to type them.\n\nHere are the numbers:\n\nThat's a 50% difference. This isn't abstract — it's real extra code that you (and the LLM) have to process.\n\nA critic might say: \"Rust writes more lines, but each line carries meaningful information — Result, Option, lifetimes. That's not boilerplate, it's safety.\"\n\nThat's fair. And here's the important part: **the metric already accounts for that**.\n\nGzip only compresses *repetition*. Unique constructs like Result, Option, and lifetimes remain in the compressed output. They increase the gzipped size, which *lowers* the boilerplate ratio.\n\nCompare Rust and Zig:\n\nRust's gzipped size is noticeably smaller than Zig's, and its source is smaller too. That's because Rust has fewer repetitive formalities and more unique, meaningful information.\n\nSo the metric doesn't penalize Rust for its safety features. It highlights languages with lots of repetitive ceremony, regardless of whether that ceremony serves a purpose.\n\nLook at the JVM family:\n\nThis matches every developer's intuition. The metric captures differences between languages on the same platform cleanly.\n\nYou can argue about whether to call it \"expressiveness.\" Maybe \"code density\" or \"writing efficiency\" is more accurate. The name doesn't matter.\n\nWhat matters is that this metric shows **how much code you have to write to solve a problem**. And the more code you write, the:\n\nAfter working with this data for a while, I noticed something surprising: the Expressiveness table correlates almost perfectly with my experience porting these languages using an LLM.\n\nThe higher a language sits in the table, the easier it was to work with.\n\n**Scala, Nim, Go** were among the easiest. The LLM would generate code, I'd run it, and it would just work. Sometimes the AI produced suboptimal solutions — I'd review, ask for tweaks, and things would quickly fall into place. Even when it took multiple iterations, the process was smooth.\n\nThe languages at the bottom of the table required more effort.\n\n**Odin, C, Zig** were noticeably more difficult. The codebases are large, and the LLM frequently lost context. A typical cycle: run the code — get a segfault, track down the cause — find a few more issues, fix them — new ones appear.\n\n**Zig** required more effort than others — partly due to its design philosophy, partly due to the API change. The LLM didn't know this — it generated code for the old version, which looked correct but wouldn't compile.\n\nTo understand why the difference is so stark, compare the same test — **Sort::Self** (array sort) — in two languages.\n\n```\ntype SortSelf struct {\n    BaseBenchmark\n    data   []int\n    result uint32\n}\n\nfunc (s *SortSelf) Prepare() {\n    size := int(s.ConfigVal(\"size\"))\n    s.data = make([]int, size)\n    for i := 0; i < size; i++ {\n        s.data[i] = NextInt(1_000_000)\n    }\n}\n\nfunc (s *SortSelf) Run(iteration_id int) {\n    s.result += uint32(s.data[NextInt(len(s.data))])\n    arr := make([]int, len(s.data))\n    copy(arr, s.data)\n    sort.Ints(arr)\n    s.result += uint32(arr[NextInt(len(arr))])\n}\n\nfunc (s *SortSelf) Checksum() uint32 {\n    return s.result\n}\n```\n\n27 lines. All the logic is right there. The LLM can easily hold this in context.\n\n``` js\nconst std = @import(\"std\");\nconst Benchmark = @import(\"benchmark.zig\").Benchmark;\nconst Helper = @import(\"helper.zig\").Helper;\n\npub const SortSelf = struct {\n    allocator: std.mem.Allocator,\n    helper: *Helper,\n    data: std.ArrayList(i32),\n    result_val: u32,\n\n    const vtable = Benchmark.VTable{\n        .prepare = prepareImpl,\n        .run = runImpl,\n        .checksum = checksumImpl,\n        .deinit = deinitImpl,\n    };\n\n    pub fn init(allocator: std.mem.Allocator, helper: *Helper) !*SortSelf {\n        const self = try allocator.create(SortSelf);\n        errdefer allocator.destroy(self);\n\n        self.* = SortSelf{\n            .allocator = allocator,\n            .helper = helper,\n            .data = .{},\n            .result_val = 0,\n        };\n\n        return self;\n    }\n\n    pub fn deinit(self: *SortSelf) void {\n        self.data.deinit(self.allocator);\n        self.allocator.destroy(self);\n    }\n\n    pub fn asBenchmark(self: *SortSelf) Benchmark {\n        return Benchmark.init(self, &vtable, self.helper, \"Sort::Self\");\n    }\n\n    fn prepareImpl(ptr: *anyopaque) void {\n        const self: *SortSelf = @ptrCast(@alignCast(ptr));\n        const allocator = self.allocator;\n\n        self.data.clearAndFree(allocator);\n        self.result_val = 0;\n\n        const size_val = self.helper.config_i64(\"Sort::Self\", \"size\");\n        const size = @as(usize, @intCast(size_val));\n\n        self.data.ensureTotalCapacity(allocator, size) catch return;\n        self.helper.reset();\n\n        for (0..size) |_| {\n            const val = self.helper.nextInt(1_000_000);\n            self.data.append(allocator, val) catch return;\n        }\n    }\n\n    fn testSort(self: *SortSelf, allocator: std.mem.Allocator) ![]i32 {\n        const arr = try allocator.alloc(i32, self.data.items.len);\n        @memcpy(arr, self.data.items);\n\n        if (arr.len > 0) {\n            std.sort.pdq(i32, arr, {}, std.sort.asc(i32));\n        }\n\n        return arr;\n    }\n\n    fn runImpl(ptr: *anyopaque, _: i64) void {\n        const self: *SortSelf = @ptrCast(@alignCast(ptr));\n        const allocator = self.allocator;\n        const data = self.data.items;\n\n        var arena = std.heap.ArenaAllocator.init(allocator);\n        defer arena.deinit();\n        const arena_allocator = arena.allocator();\n\n        if (data.len > 0) {\n            const idx1 = @as(usize, @intCast(self.helper.nextInt(@as(i32, @intCast(data.len)))));\n            self.result_val +%= @as(u32, @intCast(data[idx1]));\n        }\n\n        const sorted = self.testSort(arena_allocator) catch return;\n        if (sorted.len > 0) {\n            const idx2 = @as(usize, @intCast(self.helper.nextInt(@as(i32, @intCast(sorted.len)))));\n            self.result_val +%= @as(u32, @intCast(sorted[idx2]));\n        }\n    }\n\n    fn checksumImpl(ptr: *anyopaque) u32 {\n        const self: *SortSelf = @ptrCast(@alignCast(ptr));\n        return self.result_val;\n    }\n\n    fn deinitImpl(ptr: *anyopaque) void {\n        const self: *SortSelf = @ptrCast(@alignCast(ptr));\n        self.deinit();\n    }\n};\n```\n\n91 lines. Three times more. The Zig implementation includes manual memory management, virtual tables, type casting, error handling, arenas, and a full interface with callbacks are all part of the implementation. The sorting logic itself is a relatively small part of the code.\n\nWhen a neural network works with code like this:\n\n`defer` or a misused allocator.\nA large codebase isn't just hard for AI — it's presents challenges for humans as well. A new developer opens the code and drowns in details. You yourself open it six months later and struggle to remember what it does.\n\nIn Go, this doesn't happen. 27 lines — everything in plain sight. Clear now, clear in a year, clear to anyone.\n\nVerbosity isn't unique to Zig. C, Odin — all the languages at the bottom of the table share the same pattern. Lots of code, lots of ceremony, logic buried in details.\n\nYes, with perfect discipline you can keep such code readable. But the question is: **is it worth it?** In languages at the top, readability comes **for free**. In languages at the bottom, you have to **pay** for it.\n\nThe Expressiveness table turned out to be more than just a curiosity. It closely matches my experience porting these benchmarks with an LLM. The higher a language ranks, the easier it was to work with.\n\nThis metric has practical value: if you plan to use LLMs in your workflow, **languages from the top half of the table will require less effort**.\n\nIn 2026, with AI writing a growing share of our code, **LLM-friendliness is a useful consideration** when choosing a language. The Expressiveness metric offers one way to measure it.", "url": "https://wpnews.pro/news/llm-friendliness-as-a-metric-porting-20-languages-with-an-llm", "canonical_source": "https://kostya.github.io/LangArena/llm_friendliness.html", "published_at": "2026-09-24 10:15:08+00:00", "updated_at": "2026-09-24 10:32:13.968290+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "ai-research"], "entities": ["Crystal", "LangArena", "Zig", "Rust", "Scala", "Nim", "Go", "Odin"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llm-friendliness-as-a-metric-porting-20-languages-with-an-llm", "markdown": "https://wpnews.pro/news/llm-friendliness-as-a-metric-porting-20-languages-with-an-llm.md", "text": "https://wpnews.pro/news/llm-friendliness-as-a-metric-porting-20-languages-with-an-llm.txt", "jsonld": "https://wpnews.pro/news/llm-friendliness-as-a-metric-porting-20-languages-with-an-llm.jsonld"}}