Botan bindings devlog

Technical challenges of importing Botan's `BOTAN_HAS_<feature>` support macros for use in Haskell with CPP and conditional compilation. The main obstacles include an indented `#define` in `build.h` that violates strict C parsing rules, and the inability to import `botan/version.h` for version macros. The developer ultimately solves the problem by creating a C++ shim header with a `.hpp` extension, which allows proper macro import and includes backwards-compatible definitions for older Botan versions.

The last update ended with a brief foray into importing botan’s BOTAN HAS <feature support macros, so we can eg determine what algorithms are available in a given installation - this update continues that. A quick recap, adding this support isn’t trivial - there are a few issues that combine and really get in the way of what should have been an easy import-some-C-macros-and-use-them. The first, and core, issue is that we can’t actually import any botan macros for use with CPP and conditional compilation, which, as the intended use, would have been really nice : define , which is according to GHC’s strict C parser not legal, and so if we use eg {- LANGUAGE CPP, CApiFFI -} and then import <botan/build.h , it throws an error because of the indented definebuild.h is actually technically a C++ header, but it gets parsed as a C header because of the .h extension.import <botan/ffi.h , which genuinely IS a C header, except…ffi.h imports build.h , which was written to be usable as a C header and this works when compiling C with a C++ compiler and C++ compilers do allow indented defines, but we are compiling with a strict C compiler, so we can’t even import <botan/ffi.h to get access to BOTAN VERSION MAJOR/MINOR/PATCH macros for conditional compilation that way.The second issue is how botan intends for you to use the macros: BOTAN HAS <feature macro is defined20260521 .botan/version.h , which… is not a C header, and it also throws errors if importedforeign import capi unsafe itforeign import capi unsafe it will… throw an error because of a missing symbol 0 if its missing, right? 0 The third issue is retconning missing defines in older versions of botan: 3.0-3.12 , every new feature has come with a BOTAN HAS <feature macro to indicate support.BOTAN HAS macro can now mean one thing for one version of botan, and different thing for another version3.0 , but has since had a new BOTAN HAS KDF macro added, in 3.9 .3.0-3.8 , the BOTAN HAS KDF macro is always missing, because the feature is always present, because it cannot be turned off, but for 3.9+ , if BOTAN HAS KDF is missing, it means that the feature is missing because it has been turned off.Basically, we can’t use botan’s support macros how they are intended to be used, because it fails the swiss cheese model because these few holes lined up perfectly, to make fixing this one of the most annoying things I’ve had to deal with in a very long time. None of this would be a problem if it weren’t for that single indented define - crazy, right? We can actually handle this if we can get access to the version and support macros, except I’ve considered and / or tried a lot of different things to various levels build.h file myself to generate a fixed header with just the support macros defined but that sounds fragile.h extension? This also sounds like a big hammerbotan --version just to get the version code? This might not detect the right installation, eg when testing against a local install vs a global installI am not that much of a C++ guy, I prefer my C. So it took me a while to realize: .hpp extension?And, fingers crossed, initial testing shows that it works I’m still finalizing precisely how it goes about defining macros for pre-existing features, but roughly something like this ifndef BOTAN MACRO SHIM H define BOTAN MACRO SHIM H include <botan/ffi.h include <botan/build.h // Copied from <botan/version.h , which cannot be imported properly define BOTAN VERSION CODE FOR a, b, c a << 16 | b << 8 | c define BOTAN VERSION CODE BOTAN VERSION CODE FOR BOTAN VERSION MAJOR, BOTAN VERSION MINOR, BOTAN VERSION PATCH / Backwards-compatible macros for older versions / if BOTAN VERSION CODE < BOTAN VERSION CODE FOR 3,9,0 // Macro is always missing <3.9, so we just define it as the current version code define BOTAN HAS KDF BOTAN VERSION CODE // Alternative: Set it if if any algorithm is supported // However, the KDF feature still technically exists and is supported, // even if no algorithms are present if defined BOTAN HAS KDF1 || defined BOTAN HAS KDF2 || defined BOTAN HAS HKDF || ... define BOTAN HAS KDF BOTAN VERSION CODE // Or min code of all algorithms endif endif // End of < 3.9 / Convert undefined macros as 0 / // We have to do this, for every support macro ... ifndef BOTAN HAS KDF define BOTAN HAS KDF 0 endif // Alternative: // Additionally convert any 0 to 1 so we can import as Bool instead of as a CInt if defined BOTAN HAS KDF && BOTAN HAS KDF 0 define BOTAN HAS KDF 1 else define BOTAN HAS KDF 0 endif endif // END OF BOTAN MACRO SHIM H THIS seems to be working… instead of importing the support macros from build.h , I can import them from shim.hpp instead, where they have already been fixed to avoid any missing symbols Of course, now I have to do this for some 300-ish support macros - oooooh boy. It is going to take a little while because I can’t just auto this, I have to test each and every one. Oh, and its not documented when each macro was added If I have to run through a bunch of different versions testing things to see when they were added, I’m going to make it easier for everyone. You need 2 things to automate switching from a global botan install to a local one: botan installation of the requisite version to point tocabal.project.local to do the pointingHitch in the plans - Botan requires that you set up local installs using ./configure.py , but also this is where and how you do the pointing. What to do, what to do… So anyway, I started scripting. Long story short, I have a glorious bash script that clones the randombit repo, checks out all official versions, builds all of them, and generates a cabal.project.local for each of them that can be automatically copied to the root. Now I can test any of the 12 versions almost painlessly - it still requires exiting GHC and re-building, but that’s way better than what it was like before. I’ll need to tidy up the script but this should make local builds easier for everyone, by allowing anyone to use any version or distribution of botan that they need - and the best part is, I needed to do this anyway to support WASM , which requires a local build, so building for web is that much closer Now I begin the tedious task of doing this for every support macro - all 300 of them. But it will be worth it to be able to build against any 3.x version, not just the default distribution but even ones that enable and disable various features which, is required for WASM . That’s it for today’s update