offset_of! slices

A Rust developer created a macro to compute the compile-time offset of slice fields in structs, bypassing the limitation of std::mem::offset_of! on stable Rust. The hack uses a fake instance in a large byte array and pointer arithmetic, with offsets capped at 64KB to avoid undefined behavior.

std::mem::offset of https://doc.rust-lang.org/stable/std/mem/macro.offset of.html is a helpful little macro that lets you compute the offset of a particular field of a struct at compile time. I haven’t had much use for it myself, but a friend https://terts.dev/ has been using it for a cool JIT’ed scripting language for Rust https://codeberg.org/NLnetLabs/roto , so it’s come up occasionally. It has a few quirks, though: you can’t use it for unsized fields e.g. b in Foo { a: u8, b: u8 } . This isn’t too common, but if you’re doing weird enough things that you need to compute byte offsets of fields, you are probably familiar with and happy to ab use dynamically sized types. Like all frustrating things, there’s a reason beneath the surface: in Rust, types that implement Sized have a fixed size and alignment , while others do not. With a type like Bar { a: u8, b: dyn Debug } , b could have an arbitrary type, thus an arbitrary alignment, thus an arbitrary offset. Slices are a bit of an exception here: they do not implement Sized , and their size is not fixed at compile time, but their alignment is . The alignment of T is just the alignment of T . So the offset of a slice field in a struct can, in theory, be computed. std::mem::offset of supports this under the unstable offset of slice feature … but what if you wanted this behavior on stable Rust? Behold also on the Rust playground https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=05491a0a69fb38c99d1dde6fdf0051ee : js macro rules fun offset of { $t:ty, $field:ident = { const { // An empty space we can play within. const EMPTY: & u8 = & 0u8; 65536 ; let empty: const u8 = &raw const EMPTY; // This cast only compiles if $t is Sized or has // slice metadata. let container = empty as const $t; // Now, extract the field. This is technically // unsafe , but since we're in a const block, // undefined behavior will be a compilation error. let field = unsafe { &raw const container .$field }; // Compute the offset between container and // field , in units of bytes. We satisfy the basic // requirements of offset from : both are derived // from the same allocation, EMPTY . let container = container.cast::<u8 ; let field = field.cast::<u8 ; unsafe { field.offset from container } } };} The idea is pretty simple: we can construct a fake instance of the containing type, get a pointer to the field within it, and return the offset of that pointer. All of these operations can be performed at compile-time Due to the interesting semantics of pointer casts https://doc.rust-lang.org/reference/expressions/operator-expr.html pointer-to-pointer-cast , the macro will only compile with containing types that are Sized or end with slice fields the exact case we want to support . Unfortunately, Rust requires for &raw const container .$field that the fake instance point to valid memory and the offset be in-bounds. We can’t just use a null pointer. It doesn’t care about the contents of that memory, so we do the simple thing and build a big byte array. The offset of the field must lie within this memory, so offsets larger than 64k won’t work, but I think that’s a manageable limitation. Undefined behavior at compile time will always be turned into a compilation error Before std::mem::offset of was stabilized, there were a few crates that provided similar functionality, e.g. offset https://docs.rs/offset , , https://docs.rs/memoffset memoffset , etc. As far as I can tell, they mostly just use https://docs.rs/repr offset repr offset std::mem::offset of now, and never supported getting the offset of a slice field. Maybe somebody else has invented a similar hack before and hidden it away in an unrelated crate, but I couldn’t find anything obvious.I didn’t need this functionality but it was fun to come up with. If this story needs a moral, it’s that you should have fun and commit Rust crimes :3