Creating a .NET CLR profiler using C# and NativeAOT with Silhouette

This article summarizes an exploration of Kevin Gosse's Silhouette library, which enables developers to build .NET CLR profilers using C# and NativeAOT instead of traditional C/C++ code. The author creates a basic profiler that simply logs when an assembly is loaded, focusing on the ease of getting started with the library rather than delving into the complexities of the underlying unmanaged profiling APIs. The post also explains that NativeAOT allows the profiler to run as a self-contained native binary with its own separate .NET runtime within the same process as the application being profiled.

In this post I take Kevin Gosse's https://minidump.net/ Silhouette library https://github.com/kevingosse/Silhouette for a spin, to see how easy it is to build a basic .NET CLR profiler. And when I say basic, I mean basic —in this post we'll simply log when an assembly is loaded and that's it. I was mostly interested in seeing how easy it was to get up and running. Kevin already has a 5 part series https://minidump.net/writing-a-net-profiler-in-c-part-5/ on writing a .NET profiler in C in which he describes his Silhouette https://github.com/kevingosse/Silhouette library, as well as follow up posts using the library to do real work, like measuring UI responsiveness in Resharper https://minidump.net/measuring-ui-responsiveness/ and using profiler function hooks https://minidump.net/using-function-hooks-with-silhouette/ . This post is going to be much more basic than that, just a demonstration of the library in action. I'm also not going to go into great details about the profiling APIs themselves. Kevin covers some of this in his above posts, or alternatively you can see Christophe Nasarre https://chnasarre.medium.com/ 's series of posts on the profiling APIs https://chnasarre.medium.com/start-a-journey-into-the-net-profiling-apis-40c76e2e36cc . In this post we're really not diving in deep like that, we're just going to dip a toe in and see what's possible So before we get started, we should first recap: what are the .NET profiling APIs? What are the .NET profiling APIs? what-are-the-net-profiling-apis- For 99% of people, working with .NET means staying in a nice managed runtime and never having to worry about native code, other than perhaps the occasional P/Invoke https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke . However, behind the scenes, the .NET base class libraries often interact with native libraries, and the .NET runtime itself is a native application /exploring-the-dotnet-boot-process-via-host-tracing/ . What's more, both .NET Core and .NET Framework expose a whole suite of unmanaged APIs https://learn.microsoft.com/en-us/dotnet/core/unmanaged-api/ that you can invoke from native code. .NET Core documents three main categories of unmanaged APIs: Debugging APIs https://learn.microsoft.com/en-us/dotnet/core/unmanaged-api/debugging/ for debugging code that runs in the common language runtime CLR environment. Metadata APIs https://learn.microsoft.com/en-us/dotnet/core/unmanaged-api/debugging/ for reading or generating details about modules and types without loading them in the CLR. Profiling APIs https://learn.microsoft.com/en-us/dotnet/core/unmanaged-api/profiling/ for monitoring a program's execution by the CLR. In this post we're primarily looking at the final category, the profiling APIs, and will use the metadata APIs in a supporting role. When you think of "profiling", you probably think about the profiling tools built into Visual Studio https://learn.microsoft.com/en-us/dotnet/core/unmanaged-api/profiling/ , JetBrain's dotTrace https://www.jetbrains.com/profiler/ , or viewing dotnet-trace traces https://learn.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-trace in PerfView https://github.com/microsoft/perfview . These "traditional" profilers can use the profiling APIs for their functionality though there are other approaches too , but the profiling APIs are far more general than that. For example, we use them in the Datadog .NET client library https://github.com/DataDog/dd-trace-dotnet to rewrite methods to include our instrumentation. The profiling APIs are very powerful, but the real difficulty is that they're unmanaged APIs, which typically means writing C/C++ code to use them. Yuk. Well, that's where Silhouette https://github.com/kevingosse/Silhouette comes in. Who needs C when you have NativeAOT? who-needs-c-when-you-have-nativeaot- Microsoft have been working on NativeAOT for many releases, and with each new version of .NET it gets a little bit better. With NativeAOT, you can compile your .NET application to a native , standalone binary. And a native standalone binary is all you need to write a .NET profiler The key thing with a NativeAOT binary is that it's fully self-contained. That means that when your profiling binary is loaded, it's running a completely separate.NET runtime from the application being profiled. Yes, that technically means there's two .NET runtimes loaded in the process Of course, just compiling .NET as a native binary isn't the only requirement. You also need to make sure your native binary exposes all the correct entrypoints and interfaces such that the .NET runtime can load your library as though it was built using C++. To quote from Kevin's article on Silhouette https://minidump.net/writing-a-net-profiler-in-c-part-1-d3978aae9b12/ : In a nutshell, we need to expose a DllGetClassObject method that will return an instance of IClassFactory . The .NET runtime will call the CreateInstance method on the class factory, which will return an instance of ICorProfilerCallback or ICorProfilerCallback2 , ICorProfilerCallback3 , …, depending on which version of the profiling API we want to support . Last but not least, the runtime will call the Initialize method on that instance with an IUnknown parameter that we can use to fetch an instance of ICorProfilerInfo or ICorProfilerInfo2 , ICorProfilerInfo3 , … that we will need to query the profiling API. If that went right over your head, that's normal😅 These are APIs that very few .NET engineers ever need to work with, and seeing as they're C++ APIs, that's even worse But that's the point; with Native AOT and the Silhouette library, you don't need to understand all these interactions. The Silhouette library handles the messy work of setting up your entrypoint and exposing .NET types as C++ interfaces https://minidump.net/writing-a-net-profiler-in-c-part-1-d3978aae9b12/ exposing-a-c-interface-kind-of . Of course, Silhouette doesn't let you completely off the hook—you still need to know what the unmanaged APIs are for , how to use them, and how to chain them together. But Silhouette makes it easier to get started, and means you can write your logic in C , the language you know best, instead of having to wrestle with C++. Writing a .NET profiler in C writing-a-net-profiler-in-c- As an example of how easy Silhouette makes getting started, for the rest of the post we're going to write a simple profiler using .NET that simply prints out the assemblies that are loaded to the console. Creating the profiler and test projects creating-the-profiler-and-test-projects We'll start by creating a simple solution. This will consist of two projects: a class library which is our profiler, and a "hello world" test app, which we will profile: Create the two projects dotnet new classlib -o SilhouetteProf dotnet new console -o TestApp Add the projects to a sln file dotnet new sln dotnet sln add .\SilhouetteProf\ dotnet sln add .\TestApp\ This gives us our basic project structure. Now we'll add the Silhouette https://github.com/kevingosse/Silhouette library to our profiler project: dotnet add package Silhouette --project SilhouetteProf Next we need to ensure we publish our application using NativeAOT and allow unsafe code. We're not actually going to write any unsafe code ourselves in this test, but Silhouette includes a source generator which does use unsafe . Open up SilhoutteProj.csproj , and add the two properties <Project Sdk="Microsoft.NET.Sdk" <PropertyGroup <TargetFramework net10.0</TargetFramework <RootNamespace SilhouetteProf</RootNamespace <ImplicitUsings enable</ImplicitUsings <Nullable enable</Nullable < -- 👇 Add these two -- <PublishAot true</PublishAot <AllowUnsafeBlocks true</AllowUnsafeBlocks </PropertyGroup <ItemGroup <PackageReference Include="Silhouette" Version="3.2.0" / </ItemGroup </Project Now we have our prerequisites, we can start creating our profiler. Creating the basic profiler creating-the-basic-profiler To create a .NET profiler with Silhouette, you create a class that derives from the Silhouette-provided CorProfilerCallbackBase or CorProfilerCallback2Base , CorProfilerCallback3Base etc, depending on which functionality you need . You then decorate this class with a Profiler attribute and provide a unique Guid : using Silhouette; namespace SilhouetteProf; // 👇 Use a new random Guid, don't just use this one Profiler "9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6" internal partial class MyCorProfilerCallback : CorProfilerCallback5Base { } In the example above I chose a random Guid for my profiler, and derived from CorProfilerCallback5Base . Nothing in this post actually needs the "5" from ICorProfilerInfo5 , I'm just including it here to demonstrate the pattern. The Profiler attribute above drives a source generator included with Silhouette which generates the boilerplate necessary required by the .NET runtime for creating an IClassFactory . You don't have to use this generated code if, for example, you need additional logic in your DllGetClassObject method ; if you don't want this code, just omit the Profiler attribute. The generated code looks like this: namespace Silhouette. Generated { using System; using System.Runtime.InteropServices; file static class DllMain { UnmanagedCallersOnly EntryPoint = "DllGetClassObject" public static unsafe HResult DllGetClassObject Guid rclsid, Guid riid, nint ppv { if rclsid = new Guid "9fd62131-bf21-47c1-a4d4-3aef5d7c75c6" { return HResult.CORPROF E PROFILER CANCEL ACTIVATION; } ppv = ClassFactory.For new global::SilhouetteProf.MyCorProfilerCallback ; return HResult.S OK; } } } We have the skeleton of our profiler, but before we can compile it, we need to implement the Initialize method: using Silhouette; namespace SilhouetteProf; Profiler "9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6" internal partial class MyCorProfilerCallback : CorProfilerCallback5Base { protected override HResult Initialize int iCorProfilerInfoVersion { Console.WriteLine " SilhouetteProf Initialize" ; if iCorProfilerInfoVersion < 5 { // we need at least ICorProfilerInfo5 and we got < 5 return HResult.E FAIL; } // Call SetEventMask to tell the .NET runtime which events we're interested in return ICorProfilerInfo5.SetEventMask COR PRF MONITOR.COR PRF MONITOR ALL ; } } The above code is about the most simple version Initialize method we can write. Silhouette takes care of working out which version of ICorProfilerInfo is available, and passes this as an int to the method. In the above code, we're making sure that we have at least ICorProfilerInfo5 available, so we can call any methods exposed by ICorProfilerInfo5 , ICorProfilerInfo4 , ICorProfilerInfo3 etc. You'll notice a lot of HResult values used as return values. Returning error codes is the predominant error handling approach with the native APIs, so you'll be dealing with these a lot. Luckily, Silhouette exposes this as a handy enum for you to use. Once we've confirmed that the current .NET runtime supports the features we need, we need to tell the runtime which events we're interested in using the COR PRF MONITOR enum https://learn.microsoft.com/en-us/dotnet/framework/unmanaged-api/profiling/cor-prf-monitor-enumeration and the or https://learn.microsoft.com/en-us/dotnet/framework/unmanaged-api/profiling/icorprofilerinfo-seteventmask-method SetEventMask methods. For simplicity I used https://learn.microsoft.com/en-us/dotnet/framework/unmanaged-api/profiling/icorprofilerinfo5-seteventmask2-method SetEventMask2 ICorProfilerInfo5.SetEventMask and just enabled all the features.The ICorProfilerInfo5 field is initialized prior to Initialize being called, based on the available interface version. For example, if iCorProfilerInfoVersion is 7 , then all the ICorProfilerInfo fields up to ICorProfilerInfo7 will be initialized. It'sveryimportant you only call interface versions that have been initialized. So if iCorProfilerInfoVersion is 7 ,don'tcall ICorProfilerInfo8 or higher At this point we could test our profiler, but I'm going to continue with the implementation a bit before we come to testing. Adding functionality to our profiler adding-functionality-to-our-profiler Responding to events with a Silhouette profiler is as easy as overriding a method in the base class. For example we could override the Shutdown method, called when the runtime is shutting down: protected override HResult Shutdown { Console.WriteLine " SilhouetteProf Shutdown" ; return HResult.S OK; } To add a bit of interest, we're going to override the AssemblyLoadFinished method which is called when an assembly has finished loading shocking, I know : protected override HResult AssemblyLoadFinished AssemblyId assemblyId, HResult hrStatus { // ... } The AssemblyLoadFinished method provides an AssemblyId which we can use to retrieve the name of the assembly by calling another method on ICorProfilerInfo5 , GetAssemblyInfo AssemblyId . The AssemblyId type is a very thin wrapper around an IntPtr , acting as strongly-typed wrappers around the otherwise-ubiquitous IntPtr s used in the profiling APIs. I'm a big fan of this approach as it eliminates a whole class of mistakes that you could otherwise make of passing a "Class ID" IntPtr to a method expecting an "Assembly ID" IntPtr for example . We can call GetAssemblyInfo easily enough using the ICorProfilerInfo5 field, but this is a good opportunity to look at a common pattern in the Silhouette library, the use of HResult<T : protected override HResult AssemblyLoadFinished AssemblyId assemblyId, HResult hrStatus { HResult<AssemblyInfoWithName assemblyInfo = ICorProfilerInfo5.GetAssemblyInfo assemblyId // ... } HResult<T is effectively a simple result pattern https://andrewlock.net/series/working-with-the-result-pattern/ discriminated union. It contains both an HResult and, if the HResult represents success, an object T . This approach is a way of avoiding the common pattern in the profiling APIs of having multiple " out " parameters, and an HRESULT to indicate whether it's valid to use those values, e.g. HRESULT GetAssemblyInfo in AssemblyID assemblyId, in ULONG cchName, out ULONG pcchName, out, size is cchName , length is pcchName WCHAR szName , out AppDomainID pAppDomainId, out ModuleID pModuleId ; The typical pattern when working with the profiling APIs directly is to make the call, check the return value, and then decide whether to continue or not. HResult<T allows that pattern too, but you can also go YOLO mode. Instead of doing all the checks yourself, you can instead call HResult<T .ThrowIfFailed which returns the T if the call was successful, and throws a Win32Exception otherwise. This can make for some dramatically simpler code to read and write, so it's a real win. Of course, whether you would want to do this with a productiongrade profiler is a whole other thing. But then, should you really be using anything from this post for production? Probably not 😉 Using the ThrowIfFailed approach gives us the code below. We try to get the assembly name, and if it's available, print it: protected override HResult AssemblyLoadFinished AssemblyId assemblyId, HResult hrStatus { try { // Try to get the AssemblyInfoWithName, and if the HResult returns non-success, throw AssemblyInfoWithName assemblyInfo = ICorProfilerInfo5.GetAssemblyInfo assemblyId .ThrowIfFailed ; Console.WriteLine $" SilhouetteProf AssemblyLoadFinished: {assemblyInfo.AssemblyName}" ; return HResult.S OK; } catch Win32Exception ex { // GetAssemblyInfo failed for some reason, weird. Console.WriteLine $" SilhouetteProf AssemblyLoadFinished failed: {ex}" ; return ex.NativeErrorCode; } } The gains of ThrowIfFailed aren't particularly obvious if you just have a single call. Where it really shines is when you want to chain multiple calls. For example, if we wanted to implement ClassLoadStarted , we would need to chain multiple calls, and that's where ThrowIfFailed comes into its own: protected override HResult ClassLoadStarted ClassId classId { try { ClassIdInfo classIdInfo = ICorProfilerInfo.GetClassIdInfo classId .ThrowIfFailed ; using ComPtr<IMetaDataImport ? metaDataImport = ICorProfilerInfo2 .GetModuleMetaDataImport classIdInfo.ModuleId, CorOpenFlags.ofRead .ThrowIfFailed .Wrap ; TypeDefPropsWithName classProps = metaDataImport.Value.GetTypeDefProps classIdInfo.TypeDef .ThrowIfFailed ; Console.WriteLine $" SilhouetteProf ClassLoadStarted: {classProps.TypeName}" ; return HResult.S OK; } catch Win32Exception ex { Console.WriteLine $" SilhouetteProf ClassLoadStarted failed: {ex}" ; return ex.NativeErrorCode; } } We have three ThrowIfFailed calls in the above code, which keeps the nice, procedural, flow. We could instead have added three additional if result = HResult.S OK in the code, but that's harder to follow, particularly if you're writing something similar or just prototyping. OK, we now have enough functionality to take our profiler for a spin Testing our new profiler testing-our-new-profiler To test our profiler, we need to do three things - Publish our test app. - Publish our profiler. - Set the required profiling environment variables. Publish the test app publish-the-test-app We'll startup by publishing the test app. This isn't technically required, we could just run the app using dotnet run for example. The difficulty is that this invokes the .NET SDK, which is itself a .NET app /exploring-the-dotnet-boot-process-via-host-tracing/ finding-the-sdk , which means we'd end up profiling that too. Which is fine, it's just not what we're trying to do. We can publish our hello world app using a simple dotnet publish : ❯ dotnet publish .\TestApp\ -c Release Restore complete 0.6s TestApp net10.0 succeeded 0.9s → TestApp\bin\Release\net10.0\publish\ Publish our profiler publish-our-profiler Publishing our profiler is similar, but as we're using NativeAOT, we also need to provide a runtime ID. In .NET 10, you can also use the --use-current-runtime option to publish for "whatever runtime you're currently using". As you can see below, the SDK used win-x64 as I'm running on Windows: ❯ dotnet publish .\SilhouetteProf\ -c Release --use-current-runtime Restore complete 0.6s SilhouetteProf net10.0 win-x64 succeeded 4.2s → SilhouetteProf\bin\Release\net10.0\win-x64\publish\ Build succeeded in 5.5s As we're using NativeAOT, the result is a single, self contained dll plus separate debug symbols . This is our .NET app, compiled as a NativeAOT .NET profiler Setting the profiling environment variables setting-the-profiling-environment-variables To attach a profiler to the .NET runtime, you need to set some environment variables. These are different depending on whether you're profiling a .NET Framework or .NET Core app. There are three different variables to set: For profiling a .NET Framework app: COR ENABLE PROFILING=1 —Enable profiling COR PROFILER={9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6} —Set to the value of the GUID from the Profiler attribute. COR PROFILER PATH=c:\path\to\profiler —Path to the profiler dll For profiling a .NET Core/.NET 5+ app: CORECLR ENABLE PROFILING=1 —Enable profiling CORECLR PROFILER={9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6} —Set to the value of the GUID from the Profiler attribute. CORECLR PROFILER PATH=c:\path\to\profiler —Path to the profiler dll There are additional platform-specific versions of the path variable you can set if you need to support multiple platforms. After publishing the profiler and the app, I copied the absolute path to the profiler dll, and set the required environment variables using powershell. You technically don't haveto use an absolute path for the dll, you can use a relative path, but is that relative to the target app? To the working directory? I prefer to use absolute paths as they're are unambiguous $env:CORECLR ENABLE PROFILING=1 $env:CORECLR PROFILER="{9FD62131-BF21-47C1-A4D4-3AEF5D7C75C6}" $env:CORECLR PROFILER PATH="D:\repos\temp\silouette-prof\SilhouetteProf\bin\Release\net10.0\win-x64\publish\SilhouetteProf.dll" Note that the GUID variable does include the {} surrounding braces. Once the variables are set, we can take our profiler for a spin Testing our app with our NativeAOT profiler testing-our-app-with-our-nativeaot-profiler When we run our app, the .NET runtime checks the CORECLR variables, and loads our NativeAOT profiler, emitting events as the application executes. As each event is raised, we write to the console, and we can see all the assemblies being loaded as the "Hello World " application runs ❯ .\TestApp.exe SilhouetteProf Initialize SilhouetteProf AssemblyLoadFinished: System.Private.CoreLib SilhouetteProf AssemblyLoadFinished: TestApp SilhouetteProf AssemblyLoadFinished: System.Runtime SilhouetteProf AssemblyLoadFinished: System.Console SilhouetteProf AssemblyLoadFinished: System.Threading SilhouetteProf AssemblyLoadFinished: System.Text.Encoding.Extensions SilhouetteProf AssemblyLoadFinished: System.Runtime.InteropServices Hello, World SilhouetteProf Shutdown And there we have it, our .NET profiler, written in .NET, works as expected 🎉 Now, this was obviously a very simple implementation, but it showed me how easy it is to use the Silhouette library to get something up and running vastly quicker than if I had to mess with C++. One thing to bear in mind is that while Silhouette helps with the mechanics of listening to events and interoperating with the C++ interfaces, you still need to know how to use the native APIs. Silhouette helps with the learning curve there, but you'll likely still need to do research for how to achieve what you want. From my point of view, Silhouette is clearly a handy tool for fulfilling a specific need. You won't necessarily want to use it to produce a production-grade profiler, but for proof of concept or development work, it seems invaluable. Especially if Kevin continues posting practical examples https://minidump.net/measuring-ui-responsiveness/ of using Silhouette himself Summary summary In this post I gave a brief introduction to the unmanaged .NET profiling APIs, and how you would typically interact with these APIs using C++. I then described how you can use .NET to produce a binary that can interact with these APIs instead, giving all the benefits of working in .NET, while still being able to call native APIs. I then introduced Kevin Gosse's https://minidump.net/ Silhouette library https://github.com/kevingosse/Silhouette , and showed how this library makes producing a profiler with NativeAOT simple, by deriving from a base class, and overriding the methods you're interested in. I produced a simple profiler, published it, and used it to show all the assemblies loaded by a hello world console application. Overall I was impressed with how simple it was to Silhouette and will likely explore it much more in the future too