Generative AI Using Linuxulator and eGPU on FreeBSD

A FreeBSD user documented running generative AI workloads on FreeBSD by combining the operating system's Linux Binary Compatibility layer (Linuxulator) with an external NVIDIA eGPU, after native Python compilation errors and missing dependencies blocked a direct install. The setup installs FreeBSD NVIDIA drivers first, then adds matching NVIDIA Linux libraries into a Rocky Linux 9.7 userland at /compat/linux, where the author compiled and installed Python 3.10 and pulled dependencies via dnf packages from a temporary RL9 installation. The author framed the result as working "given an organized small amount of command line spells.

Generative AI using Linuxulator and eGPU on FreeBSD 3369 words, 16 minutes Not so long ago, I experimented on so-called Generative AI using an external eGPU and Slackware Linux https://www.tumfatig.net/2024/generative-ai-using-egpu-on-slackware-linux/ . Because I’m a BSD fanboy, I started looking at doing the same on FreeBSD. But I faced a lot of missing dependencies issues and Python compilation errors. As a non fluent Python person, I couldn’t solve all the errors I encountered and decided to see if the FreeBSD Linux Binary Compatibility https://docs.freebsd.org/en/books/handbook/linuxemu/ feature would be able to achieve the goal; after all, there are people using it to watch DRM stuff from the Clouds. Spoiler alert : it does work given an organized small amount of command line spells. AI opinion Feel free to jump to the next section if you don’t care about AI opinions ;- These days, AI is sold and forced everywhere. I am not the last to yell at Mozilla for pushing some shit into Firefox. I’m also pretty sure vibe coding https://en.wikipedia.org/wiki/Vibe coding is a bad idea. I have tested things like ChatGPT for a few things. Asking for help to debug error messages lead nowhere but to Stack Overflow and Unix & Linux Stack Exchange. Asking for pre-made code offered mostly non-working stuff but I must admit it gave me ideas by identifying features from software I didn’t know. Asking for product comparison was more or less an extract of Tom’s Hardware or Les Numériques. Asking for differences between technology A and B and better use cases felt approximate enough that I usually ended up sending words to SearXNG in order to get more informations. I also tested the Bing Image Generator; although I’m aware of the global harvesting of copyrighted data this implied. My opinion was that it was not as good as it was sold. It was about the same quality as child-me collecting free cliparts, colouring in Paint and organising in Designer… All in all, my take is that AI is not Artificial Intelligence. It’s not intelligence at all, yet. At best, it is Algorithm Induce. At worse, it is Aleatory Inference. And most of all, Machine Learning, Neural Networks and LLMs are not the same thing. I hate the Big Tech’s AI because they are thieves and liars. But I am still interested in the local software that can provide features other software don’t yet. And this is why I keep an eye on running stuff like Python Torch. I have not yet looked at LLaMA. To paraphrase some Monday meme, my take is that You don’t hate AI. You hate LLMs in the context of capitalism and patriarchy. Overall software overview There are quite a few bricks to assemble here. There may be smarter way but I have not found any all-in-one documentation. So I settled on choices based on my lack of knowledge. The OS is installed without particular requirements. I install software using binary packages as much as possible. The FreeBSD NVIDIA drivers are installed and configured first. When everything seems to be working, the Linux Binary Compatibility part can begin. There is a need to have NVIDIA Linux libraries available. Because it has to be the same version as the FreeBSD ones, I installed the dedicated package. From there, you get a /compat/linux directory with a fairly basic Rocky Linux 9.7 installation. It is bare enough to not ship with a package manager. As I have to install more software in the Linux userland, I’m using a temporary RL9 installation to grab dnf packages that will be installed in the /compat/linux directory. This allows installing some more software in an easy way - dealing with binary packages and dependencies. There is a dnf package in FreeBSD ports but I never understood how to use it with /compat/linux . Using the dnf package manager, I installed various required tools and libraries for later usage. I grabbed, compiled and installed Python 3.10 into the /compat/linux directory. Mostly because this version seem to be the one all the tools I tested need. The I installed a few additional NVIDIA / CUDA tools and librairies inside /compat/linux . There is also a specific NVIDIA Unified Memory UVM program to install in order to use the NVIDIA GPU from with Linuxulator. Finally, a bunch of Python virtual environments can be populated to use PyTorch based-software. Hardware, BIOS, OS As described in details here https://www.tumfatig.net/2024/generative-ai-using-egpu-on-slackware-linux/ , I am using an NVIDIA RTX 4060 Ti, connected via a Thunderbolt eGPU docking station to a Topton GM1 head-less machine. The BIOS is configured with “No Security” so that the hardware is recognised automatically. I have done all my trial & errors on a ThinkPad T480s running FreeBSD 14.3. It went the same way as my final configuration using FreeBSD 14.4 on the Topton. I didn’t go for 15.0 as I already had two bad experience with it on different projects and I see a lot of people on the Fediverse having issues with 15 too. Install and update FreeBSD 14.4/amd64. freebsd-update fetch freebsd-update install Everything else will be done remotely using SSH. FreeBSD NVIDIA drivers For some reasons, FreeBSD doesn’t support eGPU hotplug on the ThinkPad. The NVIDIA card has to be powered on and connected to the computer when it boots FreeBSD. This may change in the future. The Handbook explains how to install the NVIDIA FreeBSD drivers https://docs.freebsd.org/en/books/handbook/x11/ x-configuration-nvidia . pkg install -y nvidia-drm-kmod pkg info | grep nvidia nvidia-driver-580.119.02 1 NVidia graphics card binary drivers for hardware OpenGL rendering nvidia-drm-61-kmod-580.119.02.1403000 1 NVIDIA DRM Kernel Module nvidia-drm-kmod-580.119.02 NVIDIA DRM Kernel Module nvidia-kmod-580.119.02.1404000 1 kmod part of NVidia graphics card binary drivers for hardware OpenGL rendering sysrc kld list+=nvidia-drm During my testing period, the meta package installed incompatible versions of the serveral NVIDIA stuff. So I had to force installation of the proper version. For the record, this went like this: pkg -N install nvidia-drm-kmod ... nvidia-driver: 580.95.05 FreeBSD nvidia-drm-61-kmod: 580.95.05.1403000 FreeBSD nvidia-drm-kmod: 580.95.05 1 FreeBSD nvidia-kmod: 580.105.08.1403000 FreeBSD-kmods ... pkg install nvidia-drm-kmod-580.95.05 1 nvidia-kmod-580.95.05.1403000 ... nvidia-driver: 580.95.05 FreeBSD nvidia-drm-61-kmod: 580.95.05.1403000 FreeBSD nvidia-drm-kmod: 580.95.05 1 FreeBSD nvidia-kmod: 580.95.05.1403000 FreeBSD ... A reboot was performed to ensure I configured everything properly. The NVIDIA card was luckily identified and connected properly. pciconf -lv | grep -B3 display vgapci0@pci0:0:2:0: class=0x030000 rev=0x0c hdr=0x00 vendor=0x8086 device=0x4628 subvendor=0x8086 subdevice=0x2112 vendor = 'Intel Corporation' device = 'Alder Lake-UP3 GT2 UHD Graphics ' class = display -- vgapci1@pci0:5:0:0: class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x2805 subvendor=0x19da subdevice=0x7717 vendor = 'NVIDIA Corporation' device = 'AD106 GeForce RTX 4060 Ti 16GB ' class = display dmesg | grep -C 5 -i nvidia nvidia1: <NVIDIA GeForce RTX 4060 Ti on vgapci1 vgapci1: child nvidia1 requested pci enable io vgapci1: child nvidia1 requested pci enable io nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 580.119.02 Mon Dec 8 07:29:16 UTC 2025 drm nvidia-drm GPU ID 0x00000500 Loading driver sysctl add oid: can't re-use a leaf hw.dri.debug sysctl add oid: can't re-use a leaf hw.dri.vblank offdelay sysctl add oid: can't re-use a leaf hw.dri.timestamp precision drm Initialized nvidia-drm 0.0.0 20160202 for nvidia1 on minor 1 sysctl hw.nvidia hw.nvidia.gpus.1.type: PCIe hw.nvidia.gpus.1.firmware: hw.nvidia.gpus.1.vbios: ??.??.??.??.?? hw.nvidia.gpus.1.model: NVIDIA GeForce RTX 4060 Ti hw.nvidia.version: NVIDIA UNIX x86 64 Kernel Module 580.119.02 Mon Dec 8 08:42:31 UTC 2025 Side note for dual GPU configuration While trying to have all the following working, I used a laptop with an embedded Intel GPU. But when I booted with the NVIDIA eGPU connected, Xorg would detect it and use it as the primary display device. And because nothing was connected to the NVIDIA card output, I went blind on the laptop. To force Xorg to not use the NVIDIA GPU at all, I had to create a dedicated configuration snippet. This way, I was still able to run Xfce on the Intel GPU using the laptop monitor and the NVIDIA GPU for the Torch things. cat /usr/local/share/X11/xorg.conf.d/20-gpu.conf Section "Device" Identifier "intel0" Driver "modesetting" BusID "pci0:0:2:0" Option "PrimaryGPU" "true" EndSection Section "Device" Identifier "nvidia0" Driver "nvidia" BusID "pci0:9:0:0" EndSection Section "ServerFlags" Option "AutoAddGPU" "false" EndSection Linux Binary Compatibility Reading the Handbook Chapter 12. Linux Binary Compatibility https://docs.freebsd.org/en/books/handbook/linuxemu/ is probably a good idea. For the moment, I just needed to enable linux 4 binary compatibility: service linux enable service linux start This creates the /compat/linux directory and mount the required file systems. mount | grep linux linprocfs on /compat/linux/proc linprocfs, local linsysfs on /compat/linux/sys linsysfs, local devfs on /compat/linux/dev devfs fdescfs on /compat/linux/dev/fd fdescfs tmpfs on /compat/linux/dev/shm tmpfs, local Temporary Rocky Linux 9 A temporary RL9 instance is deployed on the server. I used the 9.7 version as this is the one available in the ports. It is based on an OCI image and is only used to easily grab all the packages required to install the dnf package manager in /compat/linux . Grab the RL97 container image and deploy in a temporary location: fetch https://dl.rockylinux.org/pub/rocky/9.7/images/x86 64/Rocky-9-Container-Base.latest.x86 64.tar.xz mkdir Rocky-9 /tmp/rl97 tar xf Rocky-9-Container-Base.latest.x86 64.tar.xz -C Rocky-9/ find Rocky-9/blobs/sha256 -type f -exec tar xpzf {} -C /tmp/rl97 \; 2 /dev/null cp -p /etc/resolv.conf /etc/hosts /tmp/rl97/etc/ chroot /tmp/rl97 /bin/bash -l Update the Rocky Linux system and grab the required dnf packages: update-ca-trust dnf --releasever 9.7 -y update dnf --releasever 9.7 -y install python3-dnf-plugin-modulesync dnf --releasever 9.7 -y download --resolve --alldeps --downloaddir ~/DNF dnf exit Back on the FreeBSD host, we have all the stuff available to manage packages in /compat/linux . Rocky Linux 9 Linuxulator When using the ports packages, several Linux distributions can be used. The NVIDIA Linux libraries will deploy a bare Rocky Linux 9.7 system inside /compat/linux . The rpm package is used to deploy the dnf packages we previously grabbed into /compat/linux . pkg install -y linux-nvidia-libs rpm4 cd /compat/linux for f in /tmp/rl97/root/DNF/ rpm; do rpm2cpio < "$f" | cpio -id; done cp -p /etc/resolv.conf /etc/hosts /compat/linux/etc/ chroot /compat/linux /bin/bash -l update-ca-trust dnf --releasever 9.7 -y update exit From there, the /compat/linux directory contains the NVIDIA librairies and a package manager suitable to add more things to the Linux instance. Don’t forget to remove the temporary instance as we should not need it anymore. Python 3.10 for Linux All the PyTorch software I tested recommand if not require Python 3.10. But because of RedHat policy, this version is not available as binary packages. AFAIK. Luckily, it can still be build from sources and installed in the Linux directory. chroot /compat/linux /bin/bash -l dnf --releasever 9.7 -y install gcc make tar wget \ bzip2-devel libffi-devel openssl-devel xz-devel zlib-devel cd /root wget https://www.python.org/ftp/python/3.10.19/Python-3.10.19.tgz tar xzf Python-3.10.19.tgz cd Python-3.10.19 ./configure --enable-optimizations make -j $ nproc make altinstall python3.10 -V Python 3.10.19 pip3.10 -V pip 23.0.1 from /usr/local/lib/python3.10/site-packages/pip python 3.10 exit Python 3.10 is now available inside /compat/linux and can be used to deploy virtual environments for PyTorch applications. Using the NVIDIA eGPU in Linuxulator A first quick test shows that the NVIDIA card is recognised inside the Linux emulation system. From /compat/linux , I can access the NVIDIA device. chroot /compat/linux /bin/bash -l uname -a Linux topton 5.15.0 FreeBSD 14.4-RELEASE releng/14.4-n273675-a456f852d145 GENERIC x86 64 x86 64 x86 64 GNU/Linux nvidia-smi Thu Mar 12 14:42:37 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 580.119.02 Driver Version: 580.119.02 CUDA Version: 13.0 | +-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 4060 Ti Off | 00000000:09:00.0 Off | N/A | | 33% 26C P0 21W / 165W | 0MiB / 16380MiB | 3% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ dnf --releasever 9.7 -y install epel-release dnf --releasever 9.7 -y install nvtop nvtop Let’s build a simple Python script that will be used to identify if the device is really accessible or not. cat test tensor.py python test tensor.py import torch device = torch.device "cuda" a = torch.ones 3, 3 .to device print f"Calculation Success \n{a + a}" ^D python3.10 -m venv venv source ./venv/bin/activate pip install torch python test tensor.py /root/venv/lib/python3.10/site-packages/torch/ subclasses/functional tensor.py:283: UserWarning: Failed to initialize NumPy: No module named 'numpy' Triggered internally at /pytorch/torch/csrc/utils/tensor numpy.cpp:84. cpu = conversion method template device=torch.device "cpu" Traceback most recent call last : File "/root/test tensor.py", line 4, in <module a = torch.ones 3, 3 .to device File "/root/venv/lib/python3.10/site-packages/torch/cuda/ init .py", line 424, in lazy init torch. C. cuda init RuntimeError: Unexpected error from cudaGetDeviceCount . Did you run some cuda functions before calling NumCudaDevices that might have already set an error? Error 304: OS call failed or operation not supported on this OS deactivate As detailed here https://github.com/isaponsoft/freebsd-ai-notes/blob/main/CUAD and llama-server.md 4-cuda-dummy-uvm , there is some need for UVM correction something. I have no idea what this is all about so I’m thankful to Isapon Soft for making their notes public. Lets build the magic thing and run the Torch test again wget https://gist.githubusercontent.com/shkhln/40ef290463e78fb2b0000c60f4ad797e/raw/0e1fd8e8ea52b7445c3d33f5e5975efd20388dcb/uvm ioctl override.c gcc -m64 -std=c99 -Wall -ldl -fPIC -shared -fno-lto -o dummy-uvm.so uvm ioctl override.c source ./venv/bin/activate LD PRELOAD="/root/dummy-uvm.so" python test tensor.py /root/venv/lib/python3.10/site-packages/torch/ subclasses/functional tensor.py:283: UserWarning: Failed to initialize NumPy: No module named 'numpy' Triggered internally at /pytorch/torch/csrc/utils/tensor numpy.cpp:84. cpu = conversion method template device=torch.device "cpu" Calculation Success tensor 2., 2., 2. , 2., 2., 2. , 2., 2., 2. , device='cuda:0' deactivate exit From there, we are now ready to run a bunch of PyTorch stuff inside our /compat/linux directory. Run upscaler I have a bunch of small family photos. And it is great to be able to get them a bit bigger. We are not in a Jason Bourne or NCIS movie so anything blurred will remain blurred. Upscaling and AI don’t guess/invent details. But visible stuff will be visible and a bit bigger. upscaler is an image upscaler using Real-ESRGAN with a web API and UI https://github.com/robertsLando/upscaler . There are maybe better software elsewhere but this one is really easy to use and provides a Web GUI that is perfect for using with a headless server. Once started, the interface is accessible using a Web browser targeting http://0.0.0.0:8000. chroot /compat/linux /bin/bash -l -x /bin/git && dnf --releasever 9.7 -y install git dnf --releasever 9.7 -y install libglvnd-glx git clone https://github.com/robertsLando/upscaler.git cd upscaler python3.10 -m venv pyenv source pyenv/bin/activate pip install uv pip install --upgrade pip uv sync LD PRELOAD="/root/dummy-uvm.so" make run auv run python -m upscaler INFO: Started server process 17878 INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 Press CTRL+C to quit ... INFO:upscaler.upscaler:Real-ESRGAN model initialized successfully INFO:upscaler.utils:Starting upscaling process... INFO:upscaler.utils:Upscaled to: 4316, 3392 INFO:upscaler.utils:Final image size: 3237, 2544 INFO: 10.15.5.61:42284 - "POST /upscale HTTP/1.1" 200 OK ^C INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process 17878 deactivate exit Using nvtop , one can see that the GPU is used during the upscaling process. Run Fooocus The installation is pretty well the same as the upscaler. The online documentation of the project https://github.com/lllyasviel/Fooocus?tab=readme-ov-file linux-using-python-venv works just well. But because I had a lot of trial & errors during my journey on FreeBSD, I wrote a script that would deploy, update and run the software in a more automatic way. cat /compat/linux/root/fooocus bash /bin/bash -d ~/sources && mkdir ~/sources -d ~/Applications && mkdir ~/Applications if -d ~/sources/Fooocus ; then cd ~/sources git clone https://github.com/lllyasviel/Fooocus.git fi if -d ~/Applications/Fooocus ; then python3.10 -m venv ~/Applications/Fooocus source ~/Applications/Fooocus/bin/activate cd ~/sources/Fooocus pip install -r requirements versions.txt pip install pip -U else source ~/Applications/Fooocus/bin/activate cd ~/sources/Fooocus fi if "$1" = "update" ; then git pull pip install -r requirements versions.txt fi LD PRELOAD="/root/dummy-uvm.so" python entry with update.py --listen deactivate exit 0 EOF This script makes it simpler to run Fooocus chrooted straight from a connexion on the FreeBSD host: chroot /compat/linux /root/fooocus ... Fooocus V2 Expansion: Vocab with 642 words. Fooocus Expansion engine loaded for cuda:0, use fp16 = True. Requested to load SDXLClipModel Requested to load GPT2LMHeadModel Loading 2 new models Fooocus Model Management Moving model s has taken 1.08 seconds Started worker with PID 18059 App started successful. Use the app with http://localhost:7865/ or 0.0.0.0:7865 Browsing to http://0.0.0.0:7865/ and looking at nvtop while generating ~~shit~~ stuff proves everything works as expected. Tips for other projects I will not detail all the GitHub projects that I tested. But some required extra stuff in the Linux environment. They were not always easy to understand. And they also explain why I wanted to have a working dnf environment. Need g++ If the project requires g++ to build Python stuff, the missing development tools can be installed using: dnf --releasever 9.7 -y groupinstall "Development Tools" Need ffmpeg If the project requires ffmpeg to be installed, this can be done this way: dnf --releasever 9.7 -y config-manager --enable crb dnf --releasever 9.7 -y install https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-9.noarch.rpm dnf --releasever 9.7 -y install https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-9.noarch.rpm dnf repolist repo id repo name appstream Rocky Linux 9 - AppStream baseos Rocky Linux 9 - BaseOS crb Rocky Linux 9 - CRB epel Extra Packages for Enterprise Linux 9 - x86 64 epel-cisco-openh264 Extra Packages for Enterprise Linux 9 openh264 From Cisco - x86 64 extras Rocky Linux 9 - Extras rpmfusion-free-updates RPM Fusion for EL 9 - Free - Updates rpmfusion-nonfree-updates RPM Fusion for EL 9 - Nonfree - Updates dnf --releasever 9.7 -y install ffmpeg ffmpeg -version ffmpeg version 5.1.8 Copyright c 2000-2025 the FFmpeg developers built with gcc 11 GCC configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86 64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D FORTIFY SOURCE=2 -Wp,-D GLIBCXX ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect libavutil 57. 28.100 / 57. 28.100 libavcodec 59. 37.100 / 59. 37.100 libavformat 59. 27.100 / 59. 27.100 libavdevice 59. 7.100 / 59. 7.100 libavfilter 8. 44.100 / 8. 44.100 libswscale 6. 7.100 / 6. 7.100 libswresample 4. 7.100 / 4. 7.100 libpostproc 56. 6.100 / 56. 6.100 Need NVIDIA CUDA If the project yells something like Failed to load library libonnxruntime providers cuda.so with error: libcudnn adv.so.9: cannot open shared object file: No such file or directory , it means you probably miss some NVIDIA CUDA Deep Neural Network cuDNN https://docs.nvidia.com/deeplearning/cudnn/installation/latest/linux.html rhel-and-rocky-network-installation librairies. dnf --releasever 9.7 -y config-manager --add-repo \ https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86 64/cuda-rhel9.repo dnf --releasever 9.7 -y install --allowerasing cudnn9-cuda-12 If the project yells something like Failed to load library libonnxruntime providers cuda.so with error: libnvrtc.so.12: cannot open shared object file: No such file or directory , it means you probably miss some CUDA toolkit libraries. dnf --releasever 9.7 -y install cuda-toolkit-12 Final thoughts It doesn’t look like FreeBSD did better than Slackware on the workloads I tested. Long processes still took hours. Short processes took from couple of tenth of seconds to minutes. CPU, memory and GPU resources were used about the same way and with the same amount. Nothing special regarding power consumption either: Topton used about 14W at IDLE and 33W while processing, the RTX eGPU idles at 26W and requires up to 140W while processing. Temperature never went above 48°C but I added an USB fan on the case and the room is about 19-20°C. All in all, it works great. Not better, not less. The objective me can’t go without thinking: what’s the point of running FreeBSD to do this if you need the Linux emulation to do the exact same thing than with a native Linux system. Just because you can doesn’t mean you should :D And given that the NVIDIA drivers have to carefully match both the FreeBSD host and the Linux layer, one could fear what will happen during the next upgrade… Still, this proves that BSDs are capable of running hype technology and that some more workload can be migrated from Linux. Because diversity matters And if you already have a FreeBSD desktop, and you 🎶 need some slop stuff baby tonight 🎶 , you don’t need a dedicated Linux machine.