{"slug": "generative-ai-using-linuxulator-and-egpu-on-freebsd", "title": "Generative AI Using Linuxulator and eGPU on FreeBSD", "summary": "A FreeBSD user documented running generative AI workloads on FreeBSD by combining the operating system's Linux Binary Compatibility layer (Linuxulator) with an external NVIDIA eGPU, after native Python compilation errors and missing dependencies blocked a direct install. The setup installs FreeBSD NVIDIA drivers first, then adds matching NVIDIA Linux libraries into a Rocky Linux 9.7 userland at /compat/linux, where the author compiled and installed Python 3.10 and pulled dependencies via dnf packages from a temporary RL9 installation. The author framed the result as working \"given an organized small amount of command line spells.", "body_md": "# Generative AI using Linuxulator and eGPU on FreeBSD\n\n#### 3369 words, 16 minutes\n\nNot so long ago, I experimented on so-called [Generative AI using an\nexternal eGPU and Slackware\nLinux](https://www.tumfatig.net/2024/generative-ai-using-egpu-on-slackware-linux/)\n.\n\nBecause I’m a BSD fanboy, I started looking at doing the same on FreeBSD.\nBut I faced a lot of missing dependencies issues and Python compilation errors.\nAs a non fluent Python person, I couldn’t solve all the errors I\nencountered and decided to see if the [FreeBSD Linux Binary\nCompatibility](https://docs.freebsd.org/en/books/handbook/linuxemu/)\nfeature would be able to achieve the goal; after all, there are people using\nit to watch DRM stuff from the Clouds.\n\n**Spoiler alert**: it does work given an organized small amount of\ncommand line spells.\n\n# AI opinion\n\nFeel free to jump to the next section if you don’t care about AI opinions ;-)\n\nThese days, AI is sold (and forced) everywhere. I am not the last to\nyell at Mozilla for pushing some shit into Firefox. I’m also pretty sure\n[vibe coding](https://en.wikipedia.org/wiki/Vibe_coding)\nis a bad idea.\n\nI have tested things like ChatGPT for a few things. Asking for help to debug error messages lead nowhere but to Stack Overflow and Unix & Linux Stack Exchange. Asking for pre-made code offered mostly non-working stuff but I must admit it gave me ideas by identifying features from software I didn’t know. Asking for product comparison was more or less an extract of Tom’s Hardware or Les Numériques. Asking for differences between technology A and B and better use cases felt approximate enough that I usually ended up sending words to SearXNG in order to get more informations.\n\nI also tested the Bing Image Generator; although I’m aware of the global harvesting of copyrighted data this implied. My opinion was that it was not as good as it was sold. It was about the same quality as child-me collecting free cliparts, colouring in Paint and organising in Designer…\n\nAll in all, my take is that AI is not Artificial Intelligence. It’s not intelligence at all, yet. At best, it is Algorithm Induce. At worse, it is Aleatory Inference. And most of all, Machine Learning, Neural Networks and LLMs are not the same thing.\n\nI hate the Big Tech’s AI because they are thieves and liars. But I am\nstill interested in the local software that can provide features other\nsoftware don’t yet. And this is why I keep an eye on running stuff like\nPython Torch. I have not yet looked at LLaMA. To paraphrase some Monday\nmeme, my take is that **You don’t hate AI. You hate LLMs in the context\nof capitalism and patriarchy.**\n\n# Overall software overview\n\nThere are quite a few bricks to assemble here. There may be smarter way but I have not found any all-in-one documentation. So I settled on choices based on my (lack of) knowledge.\n\nThe OS is installed without particular requirements. I install software using binary packages as much as possible.\n\nThe FreeBSD NVIDIA drivers are installed and configured first. When everything seems to be working, the Linux Binary Compatibility part can begin.\n\nThere is a need to have NVIDIA Linux libraries available. Because it has\nto be the same version as the FreeBSD ones, I installed the dedicated\npackage. From there, you get a `/compat/linux` directory with a fairly\nbasic Rocky Linux 9.7 installation. It is bare enough to not ship with a\npackage manager.\n\nAs I have to install more software in the Linux userland, I’m using a\ntemporary RL9 installation to grab `dnf` packages that will be installed\nin the `/compat/linux` directory. This allows installing some more\nsoftware in an easy way - dealing with binary packages and\ndependencies. There is a `dnf` package in FreeBSD ports but I never\nunderstood how to use it with `/compat/linux`.\n\nUsing the `dnf` package manager, I installed various required tools and\nlibraries for later usage.\n\nI grabbed, compiled and installed Python 3.10 into the `/compat/linux`\ndirectory. Mostly because this version seem to be the one all the tools\nI tested need.\n\nThe I installed a few additional NVIDIA / CUDA tools and librairies inside\n`/compat/linux`. There is also a specific NVIDIA Unified Memory (UVM)\nprogram to install in order to use the NVIDIA GPU from with Linuxulator.\n\nFinally, a bunch of Python virtual environments can be populated to use PyTorch based-software.\n\n# Hardware, BIOS, OS\n\nAs described in details\n[here](https://www.tumfatig.net/2024/generative-ai-using-egpu-on-slackware-linux/)\n,\nI am using an NVIDIA RTX 4060 Ti, connected via a Thunderbolt eGPU\ndocking station to a Topton GM1 head-less machine.\n\nThe BIOS is configured with “No Security” so that the hardware is recognised automatically.\n\nI have done all my trial & errors on a ThinkPad T480s running FreeBSD 14.3. It went the same way as my final configuration using FreeBSD 14.4 on the Topton. I didn’t go for 15.0 as I already had two bad experience with it on different projects and I see a lot of people on the Fediverse having issues with 15 too.\n\nInstall and update FreeBSD 14.4/amd64.\n\n```\n# freebsd-update fetch\n# freebsd-update install\n```\n\nEverything else will be done remotely using SSH.\n\n# FreeBSD NVIDIA drivers\n\nFor some reasons, FreeBSD doesn’t support eGPU hotplug on the ThinkPad. The NVIDIA card has to be powered on and connected to the computer when it boots FreeBSD. This may change in the future.\n\nThe Handbook explains how to [install the NVIDIA FreeBSD\ndrivers](https://docs.freebsd.org/en/books/handbook/x11/#x-configuration-nvidia)\n.\n\n```\n# pkg install -y nvidia-drm-kmod\n\n# pkg info | grep nvidia\nnvidia-driver-580.119.02_1     NVidia graphics card binary drivers for hardware OpenGL rendering\nnvidia-drm-61-kmod-580.119.02.1403000_1 NVIDIA DRM Kernel Module\nnvidia-drm-kmod-580.119.02     NVIDIA DRM Kernel Module\nnvidia-kmod-580.119.02.1404000_1 kmod part of NVidia graphics card binary drivers for hardware OpenGL rendering\n\n# sysrc kld_list+=nvidia-drm\n```\n\nDuring my testing period, the meta package installed incompatible versions of the serveral NVIDIA stuff. So I had to force installation of the proper version. For the record, this went like this:\n\n```\n# pkg -N install nvidia-drm-kmod\n(...)\n        nvidia-driver: 580.95.05 [FreeBSD]\n        nvidia-drm-61-kmod: 580.95.05.1403000 [FreeBSD]\n        nvidia-drm-kmod: 580.95.05_1 [FreeBSD]\n        nvidia-kmod: 580.105.08.1403000 [FreeBSD-kmods]\n(...)\n\n# pkg install nvidia-drm-kmod-580.95.05_1 nvidia-kmod-580.95.05.1403000\n(...)\n        nvidia-driver: 580.95.05 [FreeBSD]\n        nvidia-drm-61-kmod: 580.95.05.1403000 [FreeBSD]\n        nvidia-drm-kmod: 580.95.05_1 [FreeBSD]\n        nvidia-kmod: 580.95.05.1403000 [FreeBSD]\n(...)\n```\n\nA reboot was performed to ensure I configured everything properly. The NVIDIA card was luckily identified and connected properly.\n\n```\n# pciconf -lv | grep -B3 display\nvgapci0@pci0:0:2:0:     class=0x030000 rev=0x0c hdr=0x00 vendor=0x8086 device=0x4628 subvendor=0x8086 subdevice=0x2112\n    vendor     = 'Intel Corporation'\n    device     = 'Alder Lake-UP3 GT2 [UHD Graphics]'\n    class      = display\n--\nvgapci1@pci0:5:0:0:     class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x2805 subvendor=0x19da subdevice=0x7717\n    vendor     = 'NVIDIA Corporation'\n    device     = 'AD106 [GeForce RTX 4060 Ti 16GB]'\n    class      = display\n\n# dmesg | grep -C 5 -i nvidia\nnvidia1: <NVIDIA GeForce RTX 4060 Ti> on vgapci1\nvgapci1: child nvidia1 requested pci_enable_io\nvgapci1: child nvidia1 requested pci_enable_io\nnvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  580.119.02  Mon Dec  8 07:29:16 UTC 2025\n[drm] [nvidia-drm] [GPU ID 0x00000500] Loading driver\nsysctl_add_oid: can't re-use a leaf (hw.dri.debug)!\nsysctl_add_oid: can't re-use a leaf (hw.dri.vblank_offdelay)!\nsysctl_add_oid: can't re-use a leaf (hw.dri.timestamp_precision)!\n[drm] Initialized nvidia-drm 0.0.0 20160202 for nvidia1 on minor 1\n\n# sysctl hw.nvidia\nhw.nvidia.gpus.1.type: PCIe        \nhw.nvidia.gpus.1.firmware:             \nhw.nvidia.gpus.1.vbios: ??.??.??.??.??\nhw.nvidia.gpus.1.model: NVIDIA GeForce RTX 4060 Ti                                                        \nhw.nvidia.version: NVIDIA UNIX x86_64 Kernel Module  580.119.02  Mon Dec  8 08:42:31 UTC 2025\n```\n\n## Side note for dual GPU configuration\n\nWhile trying to have all the following working, I used a laptop with an embedded Intel GPU. But when I booted with the NVIDIA eGPU connected, Xorg would detect it and use it as the primary display device. And because nothing was connected to the NVIDIA card output, I went blind on the laptop. To force Xorg to not use the NVIDIA GPU at all, I had to create a dedicated configuration snippet. This way, I was still able to run Xfce on the Intel GPU using the laptop monitor and the NVIDIA GPU for the Torch things.\n\n```\n# cat /usr/local/share/X11/xorg.conf.d/20-gpu.conf \nSection \"Device\"\n        Identifier \"intel0\"\n        Driver \"modesetting\"\n        BusID \"pci0:0:2:0\"\n        Option \"PrimaryGPU\" \"true\"\nEndSection\n\nSection \"Device\"\n        Identifier \"nvidia0\"\n        Driver \"nvidia\"\n        BusID \"pci0:9:0:0\"\nEndSection\n\nSection \"ServerFlags\"\n        Option \"AutoAddGPU\" \"false\"\nEndSection\n```\n\n# Linux Binary Compatibility\n\nReading the Handbook [Chapter 12. Linux Binary\nCompatibility](https://docs.freebsd.org/en/books/handbook/linuxemu/)\nis\nprobably a good idea.\n\nFor the moment, I just needed to enable linux(4) binary compatibility:\n\n```\n# service linux enable\n# service linux start\n```\n\nThis creates the `/compat/linux` directory and mount the required file systems.\n\n```\n# mount | grep linux\nlinprocfs on /compat/linux/proc (linprocfs, local)\nlinsysfs on /compat/linux/sys (linsysfs, local)\ndevfs on /compat/linux/dev (devfs)\nfdescfs on /compat/linux/dev/fd (fdescfs)\ntmpfs on /compat/linux/dev/shm (tmpfs, local)\n```\n\n# Temporary Rocky Linux 9\n\nA temporary RL9 instance is deployed on the server. I used the 9.7\nversion as this is the one available in the ports. It is based on an OCI\nimage and is only used to easily grab all the packages required to\ninstall the `dnf` package manager in `/compat/linux`.\n\nGrab the RL97 container image and deploy in a temporary location:\n\n```\n# fetch https://dl.rockylinux.org/pub/rocky/9.7/images/x86_64/Rocky-9-Container-Base.latest.x86_64.tar.xz\n# mkdir Rocky-9 /tmp/rl97\n# tar xf Rocky-9-Container-Base.latest.x86_64.tar.xz -C Rocky-9/\n# find Rocky-9/blobs/sha256 -type f -exec tar xpzf {} -C /tmp/rl97 \\; 2>/dev/null\n# cp -p /etc/resolv.conf /etc/hosts /tmp/rl97/etc/\n# chroot /tmp/rl97 /bin/bash -l\n```\n\nUpdate the Rocky Linux system and grab the required `dnf` packages:\n\n```\n# update-ca-trust\n# dnf --releasever 9.7 -y update\n\n# dnf --releasever 9.7 -y install python3-dnf-plugin-modulesync\n# dnf --releasever 9.7 -y download --resolve --alldeps --downloaddir ~/DNF dnf\n\n# exit\n```\n\nBack on the FreeBSD host, we have all the stuff available to manage\npackages in `/compat/linux`.\n\n# Rocky Linux 9 Linuxulator\n\nWhen using the ports packages, several Linux distributions can be used.\nThe NVIDIA Linux libraries will deploy a bare Rocky Linux 9.7 system\ninside `/compat/linux`. The `rpm` package is used to deploy the `dnf`\npackages we previously grabbed into `/compat/linux`.\n\n```\n# pkg install -y linux-nvidia-libs rpm4\n\n# cd /compat/linux\n# for f in /tmp/rl97/root/DNF/*rpm; do rpm2cpio < \"$f\" | cpio -id; done\n# cp -p /etc/resolv.conf /etc/hosts /compat/linux/etc/\n\n# chroot /compat/linux /bin/bash -l\n\n# update-ca-trust\n# dnf --releasever 9.7 -y update\n\n# exit\n```\n\nFrom there, the `/compat/linux` directory contains the NVIDIA librairies\nand a package manager suitable to add more things to the Linux instance.\nDon’t forget to remove the temporary instance as we should not need it\nanymore.\n\n# Python 3.10 for Linux\n\nAll the PyTorch software I tested recommand (if not require) Python 3.10. But because of RedHat policy, this version is not available as binary packages. AFAIK.\n\nLuckily, it can still be build from sources and installed in the Linux directory.\n\n```\n# chroot /compat/linux /bin/bash -l\n\n# dnf --releasever 9.7 -y install gcc make tar wget \\\n  bzip2-devel libffi-devel openssl-devel xz-devel zlib-devel\n\n# cd /root\n# wget https://www.python.org/ftp/python/3.10.19/Python-3.10.19.tgz\n# tar xzf Python-3.10.19.tgz\n# cd Python-3.10.19\n# ./configure --enable-optimizations\n# make -j $(nproc)\n# make altinstall\n\n# python3.10 -V\nPython 3.10.19\n\n# pip3.10 -V\npip 23.0.1 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)\n\n# exit\n```\n\nPython 3.10 is now available inside `/compat/linux` and can be used to\ndeploy virtual environments for PyTorch applications.\n\n# Using the NVIDIA eGPU in Linuxulator\n\nA first quick test shows that the NVIDIA card is recognised inside the Linux emulation system.\n\nFrom `/compat/linux`, I can access the NVIDIA device.\n\n```\n# chroot /compat/linux /bin/bash -l\n\n# uname -a\nLinux topton 5.15.0 FreeBSD 14.4-RELEASE releng/14.4-n273675-a456f852d145 GENERIC x86_64 x86_64 x86_64 GNU/Linux\n\n# nvidia-smi \nThu Mar 12 14:42:37 2026       \n+-----------------------------------------------------------------------------------------+\n| NVIDIA-SMI 580.119.02             Driver Version: 580.119.02     CUDA Version: 13.0     |\n+-----------------------------------------+------------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |\n|                                         |                        |               MIG M. |\n|=========================================+========================+======================|\n|   0  NVIDIA GeForce RTX 4060 Ti     Off |   00000000:09:00.0 Off |                  N/A |\n| 33%   26C    P0             21W /  165W |       0MiB /  16380MiB |      3%      Default |\n|                                         |                        |                  N/A |\n+-----------------------------------------+------------------------+----------------------+\n\n+-----------------------------------------------------------------------------------------+\n| Processes:                                                                              |\n|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |\n|        ID   ID                                                               Usage      |\n|=========================================================================================|\n|  No running processes found                                                             |\n+-----------------------------------------------------------------------------------------+\n\n# dnf --releasever 9.7 -y install epel-release\n# dnf --releasever 9.7 -y install nvtop\n# nvtop\n```\n\nLet’s build a simple Python script that will be used to identify if the device is really accessible or not.\n\n```\n# cat > test_tensor.py\npython\n# test_tensor.py\nimport torch\ndevice = torch.device(\"cuda\")\na = torch.ones(3, 3).to(device)\nprint(f\"Calculation Success! \\n{a + a}\")\n^D\n\n# python3.10 -m venv venv\n# source ./venv/bin/activate\n# pip install torch\n\n# python test_tensor.py\n/root/venv/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:283: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)\n  cpu = _conversion_method_template(device=torch.device(\"cpu\"))\nTraceback (most recent call last):\n  File \"/root/test_tensor.py\", line 4, in <module>\n    a = torch.ones(3, 3).to(device)\n  File \"/root/venv/lib/python3.10/site-packages/torch/cuda/__init__.py\", line 424, in _lazy_init\n    torch._C._cuda_init()\nRuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 304: OS call failed or operation not supported on this OS\n\n# deactivate\n```\n\nAs [detailed\nhere](https://github.com/isaponsoft/freebsd-ai-notes/blob/main/CUAD_and_llama-server.md#4-cuda-dummy-uvm)\n,\nthere is some need for UVM correction something. I have no idea what\nthis is all about so I’m thankful to Isapon Soft for making their notes\npublic. Lets build the magic thing and run the Torch test again\n\n```\n# wget https://gist.githubusercontent.com/shkhln/40ef290463e78fb2b0000c60f4ad797e/raw/0e1fd8e8ea52b7445c3d33f5e5975efd20388dcb/uvm_ioctl_override.c\n# gcc -m64 -std=c99 -Wall -ldl -fPIC -shared -fno-lto -o dummy-uvm.so uvm_ioctl_override.c\n\n# source ./venv/bin/activate\n# LD_PRELOAD=\"/root/dummy-uvm.so\" python test_tensor.py\n/root/venv/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:283: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)\n  cpu = _conversion_method_template(device=torch.device(\"cpu\"))\nCalculation Success! \ntensor([[2., 2., 2.],\n        [2., 2., 2.],\n        [2., 2., 2.]], device='cuda:0')\n\n# deactivate\n# exit\n```\n\nFrom there, we are now ready to run a bunch of PyTorch stuff inside our\n`/compat/linux` directory.\n\n# Run upscaler\n\nI have a bunch of small family photos. And it is great to be able to get them a bit bigger. We are not in a Jason Bourne or NCIS movie so anything blurred will remain blurred. Upscaling and AI don’t guess/invent details. But visible stuff will be visible and a bit bigger.\n\n[upscaler is an image upscaler using Real-ESRGAN with a web API and\nUI](https://github.com/robertsLando/upscaler)\n. There are maybe better\nsoftware elsewhere but this one is really easy to use and provides a Web\nGUI that is perfect for using with a headless server.\n\nOnce started, the interface is accessible using a Web browser targeting http://0.0.0.0:8000.\n\n```\n# chroot /compat/linux /bin/bash -l\n\n# [ ! -x /bin/git ] && dnf --releasever 9.7 -y install git\n# dnf --releasever 9.7 -y install libglvnd-glx\n\n# git clone https://github.com/robertsLando/upscaler.git\n# cd upscaler\n\n# python3.10 -m venv pyenv\n# source pyenv/bin/activate\n\n# pip install uv\n# pip install --upgrade pip\n# uv sync\n\n# LD_PRELOAD=\"/root/dummy-uvm.so\" make run\nauv run python -m upscaler                            \nINFO:     Started server process [17878]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\nINFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)\n(...)\nINFO:upscaler.upscaler:Real-ESRGAN model initialized successfully\nINFO:upscaler.utils:Starting upscaling process...\nINFO:upscaler.utils:Upscaled to: (4316, 3392)\nINFO:upscaler.utils:Final image size: (3237, 2544)\nINFO:     10.15.5.61:42284 - \"POST /upscale HTTP/1.1\" 200 OK\n^C\nINFO:     Shutting down                            \nINFO:     Waiting for application shutdown.\nINFO:     Application shutdown complete.\nINFO:     Finished server process [17878]\n\n# deactivate\n# exit\n```\n\nUsing `nvtop`, one can see that the GPU is used during the upscaling\nprocess.\n\n# Run Fooocus\n\nThe installation is pretty well the same as the upscaler. [The online\ndocumentation of the\nproject](https://github.com/lllyasviel/Fooocus?tab=readme-ov-file#linux-using-python-venv)\nworks just well.\n\nBut because I had a lot of trial & errors during my journey on FreeBSD, I wrote a script that would deploy, update and run the software in a more automatic way.\n\n```\n# cat /compat/linux/root/fooocus\nbash\n#!/bin/bash\n\n[ ! -d ~/sources ]      && mkdir ~/sources\n[ ! -d ~/Applications ] && mkdir ~/Applications\n\nif [ ! -d ~/sources/Fooocus ]; then\n   cd ~/sources\n   git clone https://github.com/lllyasviel/Fooocus.git\nfi\n\nif [ ! -d ~/Applications/Fooocus ]; then\n   python3.10 -m venv ~/Applications/Fooocus\n   source ~/Applications/Fooocus/bin/activate\n   cd ~/sources/Fooocus\n   pip install -r requirements_versions.txt\n   pip install pip -U\nelse\n   source ~/Applications/Fooocus/bin/activate\n   cd ~/sources/Fooocus\nfi\n\nif [ \"$1\" = \"update\" ]; then\n   git pull\n   pip install -r requirements_versions.txt\nfi\n\nLD_PRELOAD=\"/root/dummy-uvm.so\" python entry_with_update.py --listen\n\ndeactivate\nexit 0\n#EOF\n```\n\nThis script makes it simpler to run Fooocus chrooted straight from a connexion on the FreeBSD host:\n\n```\n# chroot /compat/linux /root/fooocus\n(...)\nFooocus V2 Expansion: Vocab with 642 words.\nFooocus Expansion engine loaded for cuda:0, use_fp16 = True.\nRequested to load SDXLClipModel\nRequested to load GPT2LMHeadModel\nLoading 2 new models\n[Fooocus Model Management] Moving model(s) has taken 1.08 seconds\nStarted worker with PID 18059\nApp started successful. Use the app with http://localhost:7865/ or 0.0.0.0:7865\n```\n\nBrowsing to http://0.0.0.0:7865/ and looking at `nvtop` while generating\n~~shit~~ stuff proves everything works as expected.\n\n# Tips for other projects\n\nI will not detail all the GitHub projects that I tested. But some\nrequired extra stuff in the Linux environment. They were not always easy\nto understand. And they also explain why I wanted to have a working\n`dnf` environment.\n\n## Need g++\n\nIf the project requires `g++` to build Python stuff, the missing\ndevelopment tools can be installed using:\n\n```\n# dnf --releasever 9.7 -y groupinstall \"Development Tools\"\n```\n\n## Need ffmpeg\n\nIf the project requires `ffmpeg` to be installed, this can be done this\nway:\n\n```\n# dnf --releasever 9.7 -y config-manager --enable crb\n# dnf --releasever 9.7 -y install https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-9.noarch.rpm\n# dnf --releasever 9.7 -y install https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-9.noarch.rpm\n\n# dnf repolist\nrepo id                    repo name\nappstream                  Rocky Linux 9 - AppStream\nbaseos                     Rocky Linux 9 - BaseOS\ncrb                        Rocky Linux 9 - CRB\nepel                       Extra Packages for Enterprise Linux 9 - x86_64\nepel-cisco-openh264        Extra Packages for Enterprise Linux 9 openh264 (From Cisco) - x86_64\nextras                     Rocky Linux 9 - Extras\nrpmfusion-free-updates     RPM Fusion for EL 9 - Free - Updates\nrpmfusion-nonfree-updates  RPM Fusion for EL 9 - Nonfree - Updates\n\n# dnf --releasever 9.7 -y install ffmpeg\n\n# ffmpeg -version\nffmpeg version 5.1.8 Copyright (c) 2000-2025 the FFmpeg developers\nbuilt with gcc 11 (GCC)\nconfiguration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect\nlibavutil      57. 28.100 / 57. 28.100\nlibavcodec     59. 37.100 / 59. 37.100\nlibavformat    59. 27.100 / 59. 27.100\nlibavdevice    59.  7.100 / 59.  7.100\nlibavfilter     8. 44.100 /  8. 44.100\nlibswscale      6.  7.100 /  6.  7.100\nlibswresample   4.  7.100 /  4.  7.100\nlibpostproc    56.  6.100 / 56.  6.100\n```\n\n## Need NVIDIA CUDA\n\nIf the project yells something like`Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn_adv.so.9: cannot open shared object file: No such file or directory`, it means you probably miss some [NVIDIA CUDA Deep Neural\nNetwork\n(cuDNN)](https://docs.nvidia.com/deeplearning/cudnn/installation/latest/linux.html#rhel-and-rocky-network-installation)\nlibrairies.\n\n```\n# dnf --releasever 9.7 -y config-manager --add-repo \\\n  https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo\n\n# dnf --releasever 9.7 -y install --allowerasing cudnn9-cuda-12\n```\n\nIf the project yells something like`Failed to load library libonnxruntime_providers_cuda.so with error: libnvrtc.so.12: cannot open shared object file: No such file or directory`, it means you probably miss some CUDA toolkit libraries.\n\n```\n# dnf --releasever 9.7 -y install cuda-toolkit-12\n```\n\n# Final thoughts\n\nIt doesn’t look like FreeBSD did better than Slackware on the workloads I tested. Long processes still took hours. Short processes took from couple of tenth of seconds to minutes. CPU, memory and GPU resources were used about the same way and with the same amount.\n\nNothing special regarding power consumption either: Topton used about 14W at IDLE and 33W while processing, the RTX eGPU idles at 26W and requires up to 140W while processing. Temperature never went above 48°C but I added an USB fan on the case and the room is about 19-20°C.\n\nAll in all, it works great. Not better, not less.\n\nThe objective me can’t go without thinking: what’s the point of running FreeBSD to do this if you need the Linux emulation to do the exact same thing than with a native Linux system. Just because you can doesn’t mean you should :D And given that the NVIDIA drivers have to carefully match both the FreeBSD host and the Linux layer, one could fear what will happen during the next upgrade…\n\nStill, this proves that BSDs are capable of running hype technology and that some more workload can be migrated from Linux. Because diversity matters!!!\n\nAnd if you already have a FreeBSD desktop, and you *🎶 need some slop\nstuff baby tonight 🎶*, you don’t need a dedicated Linux machine.", "url": "https://wpnews.pro/news/generative-ai-using-linuxulator-and-egpu-on-freebsd", "canonical_source": "https://www.tumfatig.net/2026/generative-ai-using-linuxulator-and-egpu-on-freebsd/", "published_at": "2026-09-10 23:32:46+00:00", "updated_at": "2026-09-10 23:48:23.512096+00:00", "lang": "en", "topics": ["generative-ai", "ai-infrastructure", "ai-tools"], "entities": ["FreeBSD", "Linuxulator", "NVIDIA", "Rocky Linux 9.7", "Python 3.10", "dnf", "Slackware Linux", "ChatGPT"], "alternates": {"html": "https://wpnews.pro/news/generative-ai-using-linuxulator-and-egpu-on-freebsd", "markdown": "https://wpnews.pro/news/generative-ai-using-linuxulator-and-egpu-on-freebsd.md", "text": "https://wpnews.pro/news/generative-ai-using-linuxulator-and-egpu-on-freebsd.txt", "jsonld": "https://wpnews.pro/news/generative-ai-using-linuxulator-and-egpu-on-freebsd.jsonld"}}