{"slug": "making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic", "title": "Making foreach on an IEnumerable allocation-free using reflection and dynamic methods", "summary": "The article explains that when using `foreach` on an `IEnumerable<T>` variable instead of a concrete type like `List<T>`, the C# compiler generates code that allocates heap memory for the enumerator, causing slower performance and memory allocation. This occurs because the compiler uses pattern matching to find a `GetEnumerator()` method; with a concrete type it can use a value-type enumerator (allocation-free), but with `IEnumerable<T>` it must use the reference-type `IEnumerator<T>` interface, which requires a heap allocation. The author describes a technique using reflection and dynamic methods to eliminate this allocation when iterating over `IEnumerable<T>`.", "body_md": "In this post I describe a technique for reducing the allocation associated with calling foreach\non an IEnumerable<T>\n. This has been described and used previously by others, but I was recently optimizing some code in my day job working on the .NET SDK at Datadog and used the technique, so decided to explain it in more detail.\nBackground: when foreach\nallocates\nforeach\nis one of the most commonly used patterns in C#; it's literally used all over the place. A quick, crude, search of the dotnet/runtime repository reveals 3.9 thousand instances! The vast majority of those cases are enumerating built-in types from the .NET base class library, such as List<T>\nand arrays, but you can easily foreach\nover your own custom types too.\nInterestingly, the way that most people likely think or are taught about foreach\nis that you need to implement IEnumerable\n(or IEnumerable<T>\n), and then you can enumerate the collection. This is correct, but there's actually an interesting subtlety. Technically the compiler uses pattern matching, and looks for a GetEnumerator()\nmethod that returns an Enumerator\n-like type with a Current\nproperty and MoveNext\nmethod. That pattern requirement is the same as what IEnumerable\ndefines, so what's the difference?\nBefore we dig into that, it's worth taking a look at a quick benchmark which demonstrates the difference.\nCreating a benchmark to compare foreach\nI started by creating a new BenchmarkDotNet project using their templates by running\ndotnet new benchmark\nI then updated the Benchmarks\nfile as shown below. This simple benchmark just calls foreach\non a List<int>\ninstance, and then runs the same foreach\nloop on the same List<int>\n, but this time stored as an IEnumerable<int>\nvariable:\nusing System.Collections.Generic;\nusing System.Linq;\nusing BenchmarkDotNet.Attributes;\n[MemoryDiagnoser]\npublic class Benchmarks\n{\nprivate List<int> _list;\nprivate IEnumerable<int> _enumerable;\n[GlobalSetup]\npublic void GlobalSetup()\n{\n_list = Enumerable.Range(0, 10_000).ToList();\n_enumerable = _list;\n}\n[Benchmark]\npublic long List()\n{\nvar value = 0;\nforeach (int i in _list)\n{\nvalue += i;\n}\nreturn value;\n}\n[Benchmark]\npublic long IEnumerable()\n{\nvar value = 0;\nforeach (int i in _enumerable)\n{\nvalue += i;\n}\nreturn value;\n}\n}\nYou might think that both these benchmarks would give the same results. Afterall, they're running the same foreach\nloop on the same List<T>\ninstance. The only difference is whether the variable is stored as a List<int>\nor an IEumerable<int>\n, that can't make much difference, right?\nIf we run the benchmarks (I ran them against both .NET Framework and .NET 9), then we can see there actually is a difference; the IEnumerable\nversion is both slower and it allocates:\nSo the question is, why?\nforeach\nas lowered C#\nIt helps initially to understand exactly what the foreach\nconstruct looks like in \"lowered\" C#. This is effectively what the compiler converts the foreach\nloop into before converting it to IL. If we take the EnumerateList()\nmethod above and run it through sharplab.io, you get the following:\nprivate List<int> _list;\npublic long EnumerateList()\n{\nint num = 0;\nList<int>.Enumerator enumerator = _list.GetEnumerator();\ntry\n{\nwhile (enumerator.MoveNext())\n{\nint current = enumerator.Current;\nnum += current;\n}\n}\nfinally\n{\n((IDisposable)enumerator).Dispose();\n}\nreturn num;\n}\nAs you can see, in this example, the GetEnumerator()\nmethod returns a List<int>.Enumerator\ninstance, which exposes a MoveNext()\nmethod, a Current\nproperty, and implements IDisposable\n. If we compare that to EnumerateIEnumerable()\nwe get almost the same code:\nprivate IEnumerable<int> _enumerable;\npublic long EnumerateIEnumerable()\n{\nint num = 0;\nIEnumerator<int> enumerator = _enumerable.GetEnumerator();\ntry\n{\nwhile (enumerator.MoveNext())\n{\nint current = enumerator.Current;\nnum += current;\n}\n}\nfinally\n{\nif (enumerator != null)\n{\nenumerator.Dispose();\n}\n}\nreturn num;\n}\nThe main difference in the code above is that the GetEnumerator()\nmethod returns an IEnumerator<int>\ninstance instead of a concrete List<int>.Enumerator\ninstance. If we look at the implementation details of List<T>\n's enumeration methods, we can see there actually 3 different implementations, but they all ultimately delegate to the GetEnumerator()\nmethod that returns an List<T>.Enumerator\ninstance.\npublic class List<T>\n{\npublic Enumerator GetEnumerator() => new Enumerator(this);\nIEnumerator<T> IEnumerable<T>.GetEnumerator() => GetEnumerator();\nIEnumerator IEnumerable.GetEnumerator() => ((IEnumerable<T>)this).GetEnumerator();\npublic struct Enumerator : IEnumerator<T>, IEnumerator\n{\n// details hidden for brevity\n}\n}\nAnd importantly, the List<T>.Enumerator\nis defined as a struct\ntype.\nStruct enumerators\nThe struct\nenumerator is the key to the difference in allocation. By returning a mutable struct\nimplementation of the Enumerator\ninstead of a class\n, the List<T>.Enumerator\ntype can be allocated on the stack, avoid any allocation on the heap, and so avoid adding pressure on the garbage collector. That's as long as the compiler can call the GetEnumerator()\nmethod directly…\nHowever, when calling foreach\non the IEnumerable\nvariable, we need to return an IEnumerator\n(or IEnumerator<T>\n) to satisfy the interface. The only way to do that is for the List<T>.Enumerator\nobject to be boxed onto the heap. This is the source of the allocation we saw in the benchmark for the IEnumerable\nvariable.\nIn general, this limitation is all a little unfortunate and kind of annoying. Returning basic interface types like IEnumerable<T>\nor ICollection<T>\nrather than their concrete types is a standard method of encapsulation, which allows for later evolution without disrupting the public API, and is generally, rightly, encouraged. It's just a shame that results in allocation. Unless, that is, you're using .NET 10…\nA .NET 10 caveat: deabstraction\nIf I run the same benchmark above on .NET 10, I get some interesting results:\nBoth benchmarks are essentially the same. There's no allocation, and the execution time is essentially the same! So what's going on here? the short answer is that .NET 10 introduced a bunch of techniques to make this sort of pattern faster. There's devertualization, so the runtime can see that it's always a List<T>\nand call the struct\nenumerator, and there's also Object Stack Allocation, where objects which would otherwise be allocated to the heap are actually allocated to the stack if the compiler can prove the object won't \"escape\". Add to that additional work to fix the List<T>.Enumerator\n, and you get the glorious results above!\nWhich is all great if you're using .NET 10. Unfortunately, in my work on the Datadog .NET SDK, we have customers that run on all sorts of older versions of .NET (including .NET Framework), and as we are often in the hot path for apps, we need to be as efficient as possible. And all those 40 byte allocations add up!\nAvoiding foreach\nallocation for known return types\nThese days, most collection types that are exposed by the BCL or by popular libraries will use the same pattern of a stack\n-based enumerator. But you lose these performance benefits when the collection is exposed as an IEnumerable\ncollection.\nOne way to avoid this regression (if you know what the return type of an API will be) is to simply cast to that type, so the compiler can \"find\" the better GetEnumerator()\nmethod:\nIEnumerable<int> someCollection = SomeApiThatReturnsAList();\n// If we know that someCollection always returns List<T>, we can \"help\" the compiler\nif(someCollection is List<int> list)\n{\n// The compiler can call `List<T>.GetEnumerator()`, allocate\n// on the stack, and avoid the boxing allocation\nforeach(var value in list)\n{\n}\n}\nelse\n{\n// Optionally Keep a fallback case for safety, in case our assumptions are wrong\n// or it changes in the future\nforeach(var value in someCollection)\n{\n}\n}\nIt feels a bit clumsy but it works to avoid the allocations, and when you're trying to be efficient, every byte counts!\nAvoiding foreach\nallocation when you can't reference the return type\nThe above approach is easy and works well if\n- You know what type is going to be returned by an API. Obviously this may change (that's the whole point of using\nIEnumerable\nafter all!) so you must make sure to handle this scenario. - That type is public, so you can reference it.\nThat second point is often a problem for us in the Datadog SDK, because we instrument many different libraries, and can't reference them at compile time. So if we want to avoid allocation from enumerators, we need to do something else.\nTake for example the Activity.TagObjects\nproperty. This API returns an IEnumerable<KeyValuePair<string, object>>\n, but the concrete type is TagsLinkedList\n, which is an internal type, with a struct\nenumerator. We can't use the is\ntrick above because TagsLinkedList\nisn't public (and we can't use the EnumerateTagObjects()\nmethod, because that's not available in all runtimes we support). So how can we avoid the allocation?\nThe answer was to use an approach that we use in various other places: use Reflection.Emit capabilities to create a DynamicMethod\nthat explicitly uses the struct enumerator.\nAs I mentioned at the start of this post, this approach isn't novel, and has been described and used previously by others. I mostly took that prior art and tweaked it for my purposes, so kudos to them for doing the hard work!\nDesigning our Reflection.Emit DynamicMethod\nReflection.Emit refers to the System.Reflection.Emit namespace, which contains various methods for creating new intermediate language (IL) in your application. IL instructions are the \"assembly code\" that the compiler outputs when you compile your application. The JIT in the .NET runtime converts these IL instructions into real assembly code when your application runs.\nReflection.Emit is primarily used by libraries and frameworks that are either trying to wild things or are trying to eek out performance wherever they can, so it's definitely an \"advanced\" API. If you haven't used it before, or you find it confusing, don't worry about it!\nIn the implementation coming below, we're basically going to \"manually\" construct a method that contains a \"lowered\" foreach\nloop, but making sure we call the struct\n-based GetEnumerator()\non the object. Something like this:\n// This is effectively the method we're going to create\npublic static void AllocationFreeForEach(\nTagsLinkedList list, // The object to enumerate\nref SomeState state, // A state object the callback object can use\nFunc<SomeState, KeyValuePair<string, object>, bool> callback) // The callback to execute\n{\n// We create a lowered version of this code:\n// foreach(var item in list)\n// {\n// if (!callback(ref state, item))\n// break;\n// }\nusing (TagsLinkedList.Enumerator enumerator = list.GetEnumerator())\n{\nwhile (enumerator.MoveNext())\n{\nif (!callback(ref state, enumerator.Current))\nbreak;\n}\n}\n}\nWe have to create the \"lowered\" version of the code when constructing our Dynamic Method, which means we also need to lower the using\nblock, so we're actually looking at something more like this instead:\npublic static void AllocationFreeForEach(\nTagsLinkedList list,\nref SomeState state,\nFunc<SomeState, KeyValuePair<string, object>, bool> callback)\n{\nTagsLinkedList.Enumerator enumerator = list.GetEnumerator();\ntry\n{\nwhile (enumerator.MoveNext())\n{\nif (!callback(ref state, enumerator.Current))\nbreak;\n}\n}\nfinally\n{\nenumerator.Dispose();\n}\n}\nThat covers pretty much what we want to emit, all we need to do now is to generate our DynamicMethod\n.\nGenerating the DynamicMethod\nWe'll emit a method similar to the code above, but as a generalized version that can be called with many different enumeration types, and with many different item types.\ninternal static class AllocationFreeEnumerator<TEnumerable, TItem, TState>\nwhere TEnumerable : IEnumerable<TItem>\nwhere TState : struct\n{\n// Use reflection to references to the methods we need to call\nprivate static readonly MethodInfo GenericGetEnume", "url": "https://wpnews.pro/news/making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic", "canonical_source": "https://andrewlock.net/making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic-methods/", "published_at": "2026-01-20 10:00:00+00:00", "updated_at": "2026-05-23 21:39:25.607894+00:00", "lang": "en", "topics": ["developer-tools"], "entities": [".NET", "C#", "Datadog", "BenchmarkDotNet"], "alternates": {"html": "https://wpnews.pro/news/making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic", "markdown": "https://wpnews.pro/news/making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic.md", "text": "https://wpnews.pro/news/making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic.txt", "jsonld": "https://wpnews.pro/news/making-foreach-on-an-ienumerable-allocation-free-using-reflection-and-dynamic.jsonld"}}