Fuzzing the Gleam Compiler

A developer used structure-aware fuzzing to test the Gleam compiler, generating random Gleam programs and comparing outputs between its Erlang and JavaScript targets. The approach found one bug after an initial LLM-based attempt cost $20 in tokens and yielded a single issue (GitHub #5613). The fuzzing method aims to uncover edge cases in code generation and type inference across both compiler backends.

Fuzzing The Gleam Compiler Can you find bugs in a compiler by generating random programs? Published on: Tue Aug 25 2026 Introduction I regularly check on Gleam’s https://gleam.run/ changelog and issue tracker. I am very fond of this project and the people contributing to it. But every time I see an issue that relates to code generation or different outputs between the Erlang and JavaScript, it nags me that there was no way to basically “compute all the Gleam programs”, run them and see if there are any issues. I imagine it like a chessboard, where you have a quasi-infinite number of possible positions on the board. But we want the chessboard to contain Gleam programs and we want an infinitely big database of those programs to see if they uncover untested edge cases. My first attempt of doing something adjacent to this was actually prompting an LLM. I instructed it to read through loads of past Gleam issues and find more edge cases by “thinking hard about it”. It came up with all sorts of bit array combinations, nested anonymous functions, nested use patterns. Predictably, this approach did not yield many results. $20 bucks of tokens later, it found exactly one issue, which was reported and fixed right away: https://github.com/gleam-lang/gleam/issues/5613 https://github.com/gleam-lang/gleam/issues/5613 . One is definitely more than zero. But there are plenty of issues with “LLM fuzzing”: it’s pricey, not deterministic and a bit like pulling the lever on a slot machine. But there was another idea that I had avoided pursuing, because to be honest it just sounded like a lot of work: structure-aware fuzzing. Structure-aware Fuzzing Writing software is hard, and humans are not great at it. To help, we’ve built other software that can partially automate the search for bugs. One of these programs is a fuzzer. They generate randomized inputs to feed into our program. The premise is that on a large scale, these random inputs will distribute in such a way that edge cases will be surfaced that we haven’t thought of yet. Fuzzers can range from totally random scrambled bytes, to highly structured grammar-aware ASTs. Feeding totally random bytes to a program is usually done for use cases that are working with images, files, network requests, protocols, etc. There are plenty of examples where fuzzing found real security flaws and bugs in open source software. For example, this finding by zzuf in Firefox, where flipping some bits in an image file would result in a browser crash: https://nvd.nist.gov/vuln/detail/CVE-2007-6715 https://nvd.nist.gov/vuln/detail/CVE-2007-6715 . But fuzzers have also uncovered real exploitable security flaws via buffer overflows. There is a program by Google “OSS Fuzz” that continuously fuzzes a lot of important open source projects: https://google.github.io/oss-fuzz/ https://google.github.io/oss-fuzz/ In our case, we are not working on a browser or network protocol. We have a compiler. And that opens up the possibility for structure-aware fuzzing. That means that we do not generate a stream of random bytes, but rather a stream of code in the form of source code or an AST. Enter Gleam There are a few things about Gleam that make it a particularly interesting candidate for fuzzing. It generates code for two targets: JavaScript and Erlang. We can compare the output of the same program for both targets and flag any differences. Gleam has a minimalistic syntax. At least compared to most other popular programming languages. We can generate valid programs that cover almost all concepts provided by the language with relatively little code. Static types. Needless to say, this is an amazing feature that lets us ensure that a program will not crash at runtime. That doesn’t mean there can’t be any bugs in the type system. There have been issues related to type inference in the past. But as we will learn later on, each aspect of the language will require its own testing approach. The functional nature and the fact that everything is an expression makes composing and structuring the programs very convenient. Rust. This might be easy to overlook, but the fact that the Gleam compiler itself is written in Rust makes it very easy to integrate existing fuzzing tooling. We can test parts of the compiler without having to run a single .gleam file. Resources I Used We are going to dive into more technical aspects of the fuzzer. But I am not going to go into a lot of code or detail. If you would like to read more about that, do check out this post and blog by Nick Fitzgerald. It served as the main inspiration for this project: https://fitzgen.com/2020/08/24/writing-a-test-case-generator.html https://fitzgen.com/2020/08/24/writing-a-test-case-generator.html Our fuzzer is going to be generation-based, not mutation-based. If you would like to understand the difference better, I recommend reading this article: https://fitzgen.com/2026/06/01/structure-aware-fuzzing-experiment.html https://fitzgen.com/2026/06/01/structure-aware-fuzzing-experiment.html In the article, the author comes to the conclusion that, at least for wasm, the mutation-based approach found a lot more issues than the generation-based approach. So it is probably worth implementing for this project in the future For an even more in-depth dive into the topic, check out this resource: https://www.fuzzingbook.org/ https://www.fuzzingbook.org/ . You can find the full code for the Gleam fuzzer in this branch of my Gleam fork: https://github.com/daniellionel01/gleam/tree/fuzzing https://github.com/daniellionel01/gleam/tree/fuzzing Phase 1: The Parser An important design choice for the fuzzer: use the public compiler API. Even though there might not be any stability guarantees for the compiler API, this makes it easy to stay compatible with future versions of Gleam. It also avoids fiddling with implementation details, which is a good way to ensure we’re not creating any false positives or negatives. To see some examples of how our parser catches and categorizes the outputs: php $ cargo run -p fuzzing-core --example classify "pub fn main { 1 }" - compiled js: 39B, ts: 32B, erl: 238B "pub fn main { let f = fn x { x + 1 }; f 41 }" - parse error "pub fn main { 1 +. \"x\" }" - analysis rejected javascript "pub fn main {" - parse error Using the fuzz crate https://github.com/rust-fuzz/cargo-fuzz and some wrapper code, we can very quickly spam the Gleam compiler with randomly generated inputs not structured yet , to see if we can crash the compiler instead of giving us an error message with more context. We’re only going to run it for 1 second, because the output is quite large: bash $ cargo +nightly fuzz run parse only --fuzz-dir fuzzing-harness -- -max total time=1 -timeout=10 INFO: Running with entropic power schedule 0xFF, 100 . INFO: Seed: 302379076 INFO: Loaded 1 modules 740945 inline 8-bit counters : 740945 0x105eeac70, 0x105f9fac1 , INFO: Loaded 1 PC tables 740945 PCs : 740945 0x105f9fac8,0x106aedfd8 , INFO: 2466 files found in fuzzing-harness/corpus/parse only INFO: -max len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: seed corpus: files: 2466 min: 1b max: 4046b total: 425422b rss: 62Mb 2467 INITED cov: 2434 ft: 8563 corp: 1249/171Kb exec/s: 0 rss: 108Mb 2513 REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 108Mb L: 48/3753 MS: 1 EraseBytes- 2645 REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 108Mb L: 8/3753 MS: 2 ChangeBit-EraseBytes- 2656 REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 109Mb L: 2/3753 MS: 1 EraseBytes- 2937 REDUCE cov: 2434 ft: 8563 corp: 1249/171Kb lim: 3764 exec/s: 0 rss: 109Mb L: 314/3753 MS: 1 EraseBytes- 3183 NEW cov: 2434 ft: 8578 corp: 1250/172Kb lim: 3764 exec/s: 0 rss: 110Mb L: 1054/3753 MS: 1 CopyPart- 3591 REDUCE cov: 2434 ft: 8578 corp: 1250/172Kb lim: 3764 exec/s: 0 rss: 111Mb L: 99/3753 MS: 3 ShuffleBytes-CrossOver-EraseBytes- 3934 NEW cov: 2434 ft: 8585 corp: 1251/173Kb lim: 3764 exec/s: 0 rss: 112Mb L: 399/3753 MS: 3 CMP-CopyPart-CopyPart- DE: "\010\000\000\000\000\000\000\000"- ... NEW FUNC 1/7 : 0x0001031cfb88 in RINvNtCs3kGMwX4aip8 4core3ptr9drop glueINtNtCshX1O598ANu2 5alloc3vec3VecINtNtNtCs846PmCUGaYz 10gleam core3ast8constant8ConstantuEEEB1f +0x0 parse only:arm64+0x1002abb88 NEW FUNC 2/7 : 0x00010323bff4 in RINvNtCs3kGMwX4aip8 4core3ptr9drop glueINtNtNtCs846PmCUGaYz 10gleam core3ast8constant8ConstantuEEBI +0x0 parse only:arm64+0x100317ff4 16785 NEW cov: 2483 ft: 8694 corp: 1262/177Kb lim: 3786 exec/s: 16785 rss: 144Mb L: 85/3753 MS: 1 CrossOver- 18061 REDUCE cov: 2483 ft: 8694 corp: 1262/177Kb lim: 3797 exec/s: 18061 rss: 149Mb L: 403/3753 MS: 1 EraseBytes- NEW FUNC 1/4 : 0x00010312c3b0 in RINvMs NtCs846PmCUGaYz 10gleam core5parseINtB5 6ParserINtNtB5 5lexer5LexerINtBT 14NewlineHandlerINtNtNtNtCs3kGMwX4aip8 4core4iter8adapters3map3MapNtNtNtB1F 3str4iter11CharIndicesNCNvBT 14make tokenizer0EEEE23parse bit array segmentNtNtNtB7 3ast7untyped11UntypedExprNCNCNvB2 21parse expression units6 00NvB2 17expect expressionNvB5 24bit array expression intEB7 +0x0 parse only:arm64+0x1002083b0 NEW FUNC 2/4 : 0x00010318893c in RINvNtCs3kGMwX4aip8 4core3ptr9drop glueINtNtCs846PmCUGaYz 10gleam core3ast15BitArraySegmentNtNtBE 7untyped11UntypedExpruEEBG +0x0 parse only:arm64+0x10026493c ... Recommended dictionary. "\010\000\000\000\000\000\000\000" Uses: 879 "\201\000" Uses: 941 End of recommended dictionary. Done 24887 runs in 2 second s Sweet. Looking at some of the artifacts it produces, you can see what kind of inputs are generated: fn ar n,n,n,///A o ఌఌ「彸䕅䕅ⅅ䕅+� " \u{000000000000000000.%\f0 fn ar n ar:rn a The nice thing about this is that it can test everything without running the gleam binary at all. It runs in-memory with the compiler pipeline in Rust. And guess what When I let this fuzzer run for quite a while, it actually found a regression on nightly, which did not happen on v1.18.1 which was the latest version of Gleam at the time of writing this : bash $ cargo +nightly fuzz run --fuzz-dir fuzzing-harness parse only fuzzing-harness/artifacts/parse only/crash-8b14db5e4bf152924501e0818787026e9f5ea229 =fuzzing-harness/artifacts/parse only/ fuzzing-harness/artifacts/parse only/crash-8b14db5e4bf152924501e0818787026e9f5ea229 INFO: Running with entropic power schedule 0xFF, 100 . INFO: Seed: 3949856390 INFO: Loaded 1 modules 750122 inline 8-bit counters : 750122 0x107773860, 0x10782aa8a , INFO: Loaded 1 PC tables 750122 PCs : 750122 0x10782aa90,0x10839cd30 , fuzzing-harness/target/aarch64-apple-darwin/release/parse only: Running 1 inputs 1 time s each. Running: fuzzing-harness/artifacts/parse only/crash-8b14db5e4bf152924501e0818787026e9f5ea229 thread '<unnamed ' 23346909 panicked at /gleam/compiler-core/src/parse.rs:5226:52: Token could not be converted to binop. note: run with RUST BACKTRACE=1 environment variable to display a backtrace ==41428== ERROR: libFuzzer: deadly signal 0 0x000109a27654 in sanitizer print stack trace+0x28 librustc-nightly rt.asan.dylib:arm64+0x87654 1 0x000106bcdf0c in fuzzer::PrintStackTrace +0x30 parse only:arm64+0x1024a9f0c 2 0x000106bc1f48 in fuzzer::Fuzzer::CrashCallback +0x54 parse only:arm64+0x10249df48 3 0x000181edb740 in sigtramp+0x34 libsystem platform.dylib:arm64e+0x3740 4 0x000181ed18d4 in pthread kill+0x124 libsystem pthread.dylib:arm64e+0x68d4 ... shortened ... NOTE: libFuzzer has rudimentary signal handlers. Combine libFuzzer with AddressSanitizer or similar for better crash reports. SUMMARY: libFuzzer: deadly signal ──────────────────────────────────────────────────────────────────────────────── Error: Fuzz target exited with exit status: 77 The key input was a pipeline | in a const expression, like so: const b = 1 | 2 . Once we find an issue, we also use git bisect on the issue. That way we can differentiate between a regression on nightly and an issue that is currently with the latest version of Gleam. Of course we ideally not only let this run for 1 second, but many hours. To be fair, the bugs found in this step are going to be nice to catch, but you are probably not going to find a code generation bug with this. And at the time of posting this article, the libFuzzer package has been removed from the branch to keep the focus on type-safe programs. I have more targeted and efficient compiler crash tests in mind for a later stage of this project namely tree-splicer . Phase 2: Type-safe Programs To generate type-safe Gleam programs we are going to build a “smith”. We create our own simplified version of the Gleam AST that we then generate programs for in a probabilistic manner. That way, once the smith decides “we need an expression that resolves to an Int ”, it can either provide a literal value like 3 or an anonymous function fn { 3 } or a variable, and so on and so forth. This is how we reliably create a lot of variety between programs to eventually discover new edge cases. This is what one of those programs looks like: js pub const k seed: Bool = False pub const k e: Int = 5 pub const k golden: String = "data" pub type V0 { Number value: String, inner: List Int } fn walk xs: List Int , acc: Int - Int { case xs { - acc x, ..rest - walk rest, acc + x } } fn f0 arguments: Float, String , l: Int, item: Float, Bool - List Int { } fn f1 class: Int, acc: String - Int { 100 } fn f2 pair: Int - Float { case "" < "abc" { "data" < rest as whole - fn v1 { { let whole = 0.0 } } 0.0 inner | "ab" < inner - { let self = 3.14 fn v2, v3 { self } "x", 4 } "b" < b - case fn v4 { 42 } 2.0 { 9 - { 1.0 } . { 2.0 } item - 3.14 } } } pub fn main { let v = walk 5 , k e - 100 let z = f2 v echo 2 echo { case "res" < "data", f0 1.0, "data" , v, 0.5, False { , - fn v5, v6 { 10.0 } False, "x" "b" < , - z +. { 2.0 } k seed, 2, h, .. as whole - 10.0 , - f2 5 } } /. { 0.5 } echo { fn v7 { 100.0 } False } . { case "b", 2 { "ab", 1 - { let rest = k seed z } "x", 5 - 2.0 , v8 - z +. { 0.25 } } } } That is a whole lot of gibberish. And that is the point We generate programs that combine together with randomly selected valid expressions to hopefully uncover combinations that cause incorrect behaviour on one or both targets. But how do we know if one of our Gleam programs produces incorrect behaviour if it compiles successfully? This is where the two compilation targets come in. For example, if there is a bug in a program that contains a case expression, we can uncover it if the logic is correctly implemented in Erlang, because we would get a different output on the JavaScript target if we provide different values in each branch. This is not 100% foolproof, but a solid approach to start with. We can still have bugs in the compiler that happen on both targets, so we can get false positives from this approach. As always: there is no silver bullet https://en.wikipedia.org/wiki/No Silver Bullet . Before we look at our Gleam smith, I need to go on a slight tangent. The echo Problem Here is the thing. JavaScript and Erlang have different ideas about how to represent values at runtime. For example, you don’t have dedicated integer and float types in JavaScript. You have Number . Some values are represented differently when you convert them to a string and use Gleam’s built-in echo keyword. Take this program: pub type Wibble { Wibble wobble: Int } pub fn main { echo "hello" echo <<1, 2, 3 echo <<"a":utf8, "b":utf8, "c":utf8 echo 1, 2, 3 echo Wibble 3 echo 1.2 echo 1.0 } Here is the output of that program on Erlang vs JavaScript: bash $ gleam run --target erlang src/app.gleam:6 "hello" src/app.gleam:7 "\u{0001}\u{0002}\u{0003}" src/app.gleam:8 "abc" src/app.gleam:9 1, 2, 3 src/app.gleam:10 Wibble 3 src/app.gleam:11 1.2 src/app.gleam:12 1.0 bash $ gleam run --target javascript src/app.gleam:6 "hello" src/app.gleam:7 <<1, 2, 3 src/app.gleam:8 <<97, 98, 99 src/app.gleam:9 1, 2, 3 src/app.gleam:10 Wibble wobble: 3 src/app.gleam:11 1.2 src/app.gleam:12 1 In JavaScript 1.0 is printed as 1 , whereas in Erlang the bit array <<1, 2, 3 is printed as "\u{0001}\u{0002}\u{0003}" . The record Wibble does not contain any labels in Erlang. Not ideal if we want to compare the outputs directly in our fuzzer. There are a few options I can think of to circumvent this issue: - Intentionally not test the values we know to diverge in their echo output. - Build a custom echo function in Gleam and inject it into the generated modules. - Build a custom “parse this output from Gleam” in Rust, since we know from the generated Module AST which values are going to be echo ’d in Gleam. For this version I went with the third option. Since this was a very well constrained problem, I generated loads of test cases and the appropriate code to parse any echo output coming from a Gleam program and parse it into a Rust enum to predictably compare the outputs. This is definitely far from perfect, but it seems to work well for now. Alright, now that we have everything in place, it’s just time to blast the fuzzer and let it generate 100k programs right? Well, this is where I need to go on my other tangent. Duplicate Findings And Blocking Issues Once you get the fuzzer going, it will generate a lot of programs of a similar shape. Consequently, if you do discover a bug, you will keep generating programs that reproduce that bug. We again have a few ways to deal with this: - Modify the Gleam smith code so that certain combinations of expressions will not be generated, until the issue is fixed. - If there is a fix available either by yourself or as a PR , provide and apply a patch to the Gleam fork running the fuzzer. That way you can keep running the fuzzer until a PR that fixes the issue is available on nightly and can be merged back into the fork. - If your issue is an error message not a value mismatch , you can usually find a string.contains phrase to skip this in your analysis. At the time of writing this, I went with the third approach. I did try the patch approach, but this assumes that the PR or your fix is definitely correct and won’t introduce any new bugs. It’s better to rely on the exact state of Gleam that the official repository is in. For example, I uncovered 2 code generation issues with JavaScript, so at the moment the fuzzer is completely ignoring any issues that generate a similar signature to the ones reported in the official repository. This does mean that I am potentially skipping over other code generation issues that have a similar error signature. I could introduce more heuristics and analysis of the program AST if it contains the exact expression that causes these issues. For a larger scale approach, this would definitely be a good idea, since right now I am running the fuzzer in batches of 100 programs and manually verify any findings it skips or flags as “new bug”. Out of 100 programs that is usually just a handful, so it’s still manageable for one person. // https://github.com/gleam-lang/gleam/issues/6182 fn is gleam issue 6182 raw: &str - bool { raw.contains "SyntaxError: Unexpected token '&&'" || raw.contains "SyntaxError: Unexpected token ' '" } // https://github.com/gleam-lang/gleam/issues/6212 fn is gleam issue 6212 raw: &str - bool { raw.contains "TypeError:" && raw.contains "is not a function" } Using The Fuzzer If you would like to play around with this yourself in the repository: bash $ cargo run -p fuzzing-cli -- run 948 === fuzz === gleam: /daniellionel01/gleam/target/release/gleam seed: 948 erlang exit: 0 nodejs exit: 0 match: OK --- values --- 0 == erl=Bool false js=Bool false 1 == erl=String "bcx" js=String "bcx" 2 == erl=Bool false js=Bool false bash A mismatch in the second echo 'd value $ cargo run -p fuzzing-cli -- run 947 === fuzz === gleam: /daniellionel01/gleam/target/release/gleam seed: 947 erlang exit: 0 nodejs exit: 0 match: NO --- values --- 0 == erl=Int 2 js=Int 2 1 = erl=Int 42 js=Int 103 2 == erl=Bool true js=Bool true You can also print the program it is running to stdout : js $ cargo run -p fuzzing-cli -- print 947 pub const golden value: String = "constructor" pub const limit value: Float = 2.0 pub const seed value: Int = 42 fn yield constructor: Float, prototype: Int, default: Float - List Int { 7 } ... lots more code ... And fuzz batches of programs like so: seed 947 is a known bug, but there is no good heuristic to detect it right now. $ cargo run -p fuzzing-cli -- batch 900 100 fuzzing-cli gleam: /daniellionel01/gleam/target/release/gleam fuzzing-cli seeds 900..999 fuzzing-cli 10/100 run, 0 mismatches, 0 skipped fuzzing-cli 20/100 run, 0 mismatches, 0 skipped fuzzing-cli 30/100 run, 0 mismatches, 0 skipped fuzzing-cli 40/100 run, 0 mismatches, 0 skipped fuzzing-cli DIVERGENCE seed 947 - erlang:0 nodejs:0 fuzzing-cli 50/100 run, 1 mismatches, 0 skipped fuzzing-cli 60/100 run, 1 mismatches, 0 skipped fuzzing-cli 70/100 run, 1 mismatches, 0 skipped fuzzing-cli 80/100 run, 1 mismatches, 0 skipped fuzzing-cli 90/100 run, 1 mismatches, 0 skipped fuzzing-cli 100/100 run, 1 mismatches, 0 skipped fuzzing-cli done: 100 programs, 1 mismatch es , 0 skipped And that’s it I go through batches of 100 at a time, take a look at any crashes or differences in the Erlang vs JavaScript output and see if it is a known program shape, or something novel. Findings By The Fuzzer So far the fuzzer has already found 9 issues One of them was closed and reported upstream to Erlang/OTP That’s a really cool finding: https://github.com/erlang/otp/issues/11494 https://github.com/erlang/otp/issues/11494 6179 — Unreachable branches cause Erlang codegen to panic https://github.com/gleam-lang/gleam/issues/6179 6180 — A type called https://github.com/gleam-lang/gleam/issues/6180 Record gives a warning from Erlang 6181 — Matching on https://github.com/gleam-lang/gleam/issues/6181 << :utf8 takes the wrong branch on JavaScript 6182 — Matching on https://github.com/gleam-lang/gleam/issues/6182 <<"":utf8 generates incorrect JavaScript 6187 — Erlang throws on two consecutive https://github.com/gleam-lang/gleam/issues/6187 echo s with a list and empty string 6192 — Compiler crash on nightly when using https://github.com/gleam-lang/gleam/issues/6192 | in a const 6212 — Function called as https://github.com/gleam-lang/gleam/issues/6212 wibble shadows a local on JavaScript 6213 — JS codegen uses the wrong variable when a https://github.com/gleam-lang/gleam/issues/6213 let shadows an outer one The fun thing is that we’re not even covering all of the potential expressions in Gleam with the current state of the fuzzer. It is one approach to fuzzing with a limited subset of Gleam programs. When we expand it to all possible expressions and statements in Gleam and introduce even more ways to test program correctness, this will become an incredibly useful tool for the Gleam community to harden the tool even more. It is also worth noting that larger projects like esbuild, Babel and Elm, that compile to JavaScript, are still getting issues related to JavaScript code generation, even after having been around many many years evanw/esbuild 4520 https://github.com/evanw/esbuild/issues/4520 , evanw/esbuild 4517 https://github.com/evanw/esbuild/issues/4517 , evanw/esbuild 4495 https://github.com/evanw/esbuild/issues/4495 , babel/babel 16009 https://github.com/babel/babel/issues/16009 , babel/babel 15111 https://github.com/babel/babel/issues/15111 , babel/babel 15145 https://github.com/babel/babel/issues/15145 , elm/compiler 2360 https://github.com/elm/compiler/issues/2360 , elm/compiler 1223 https://github.com/elm/compiler/issues/1223 , elm/compiler 1636 https://github.com/elm/compiler/issues/1636 . Code generation, especially for JavaScript, is a really really hard problem. Especially when you consider that Gleam also compiles to Erlang, which could not be more different from a language like JavaScript, while still juggling both targets incredibly well Future Work & Emerging Problems This project is in its infancy and there are a lot of things we can work on in the future: - Something I wonder is how you would do fuzzing at scale. I’m sure other companies have dedicated servers and hardware for this. But parts of the fuzzer can also be run in GitHub Actions. I think a good workflow would be for this to be available for the Gleam core maintainers and be able to run a good amount of fuzzed programs when they are reviewing a PR and before they release a new RC. - Right now, reviewing the findings of the fuzzer can be done manually and with the help of an LLM, depending on the quantity of programs. Because when you’re generating hundreds or even thousands of programs, a lot of them will run into the same issue once you find a new one. Finding a reliable approach to stop producing the same program shapes for known and not yet fixed issues seems like something that would yield a very high ROI the investment being time in this case . - I think implementing the fuzzer for the language server would uncover a lot of new edge cases, since it’s very susceptible to actions being triggered in unforeseen states. Metamorphic testing https://en.wikipedia.org/wiki/Metamorphic testing would be a good way to test the output of a program without having to compare both targets.- There have been issues related to generics and type inference in the Gleam compiler that are definitely worth fuzzing for. This will require a more complex program smith than right now that takes into account those concepts. - You can fuzz lots of inputs for official Gleam packages. Especially the standard library . There have been inconsistencies between behaviour on both targets before. gleam stdlib - I would even give the LLM fuzzing approach another shot, if I got my hands on physical hardware, like an outdated but powerful enough GPU to run a “good-enough” open weight model in a loop and just let it auto-discover new approaches 24/7 in a well-constructed harness. - I think this paper would be incredibly cool to implement for this use case and might also help for deduplicating issue reports: https://www.cs.purdue.edu/homes/xyzhang/fall07/Papers/delta-debugging.pdf https://www.cs.purdue.edu/homes/xyzhang/fall07/Papers/delta-debugging.pdf Conclusion I have a lot of fun writing code when I am working in a codebase where I can work on something without being anxious about the integration and testing of those changes. And the best thing about a test suite is that it is usually additive. Every new edge case and bug fix can provide more safety in the future. I am super excited for the future of Gleam, and contributing to it has been very rewarding. Join the Community especially Discord if you haven’t yet Cheers