Panduan Teknikal: Compile llama.cpp di Debian 12/13 dan Cross Compile ARM64

A developer published a technical guide for compiling llama.cpp on Debian 12/13, covering native builds for x86_64 and ARM64, as well as cross-compilation from x86_64 to ARM64. The guide includes steps for enabling OpenBLAS, checking binary architecture, and resolving dependency issues. It targets users running LLM inference on servers, workstations, and single-board computers like Raspberry Pi and Orange Pi.

1. Pengenalan llama.cpp ialah runtime inference LLM berasaskan C/C++ yang popular kerana ringan, pantas, dan sesuai untuk menjalankan model GGUF secara local. Ia boleh digunakan pada: Server x86 64 Workstation Linux Mini PC Raspberry Pi Orange Pi SBC ARM64 Container Linux Dalam deployment sebenar, terdapat dua pendekatan utama: Native build Compile terus pada mesin yang akan menjalankan llama.cpp. Cross compile Compile pada mesin lebih laju contohnya PC x86 64 , tetapi menghasilkan binary untuk platform lain contohnya ARM64 Orange Pi . Bahagian 1 — Persediaan Debian 12/13 1.1 Install dependency asas sudo apt update sudo apt install -y \ git \ build-essential \ cmake \ ninja-build \ pkg-config Komponen utama: Package Fungsi git Ambil source code build-essential GCC, G++, make cmake Build configuration ninja-build Build engine lebih pantas pkg-config Cari library dependency Bahagian 2 — Clone llama.cpp git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp Semak versi: git log -1 --oneline Bahagian 3 — Compile Native Mesin Sama Contoh: Debian 12/13 x86 64 Debian ARM64 Orange Pi Raspberry Pi 3.1 Configure CMake Build menggunakan Ninja: cmake -B build \ -G Ninja \ -DCMAKE BUILD TYPE=Release 3.2 Compile ninja -C build -j$ nproc atau: cmake --build build plaintext 3.3 Hasil build Semak: ls build/bin plaintext Contoh: llama-cli llama-server llama-bench llama-perplexity shell Bahagian 4 — Enable OpenBLAS Pilihan OpenBLAS boleh membantu operasi matrix CPU. Install: sudo apt install libopenblas-dev cmake Build: cmake -B build \ -G Ninja \ -DCMAKE BUILD TYPE=Release \ -DGGML BLAS=ON \ -DGGML BLAS VENDOR=OpenBLAS shell Kemudian: ninja -C build Nota Penting: CMake Cache Jika pernah configure dengan: -DGGML BLAS=ON kemudian buang option tersebut, CMake masih menyimpan konfigurasi lama. Contoh masalah: BLAS not found missing: BLAS LIBRARIES Penyelesaian: rm -rf build Kemudian configure semula. Sentiasa ingat: CMakeCache.txt menyimpan konfigurasi lama. Bahagian 5 — Cross Compile x86 64 → ARM64 Contoh: PC Debian 12 x86 64 | | v Orange Pi ARM64 Kelebihan: Compile lebih cepat Tidak membebankan SBC Sesuai untuk production image 5.1 Install ARM64 cross compiler sudo apt install -y \ gcc-12-aarch64-linux--gnu\ g++-12-aarch64-linux-gnu sudo apt install -y \ gcc-13-aarch64-linux--gnu\ g++-13-aarch64-linux-gnu Semak: aarch64-linux-gnu-gcc --version 5.2 Configure cross build Bersihkan dahulu: rm -rf build-arm Kemudian: cmake -B build-arm \ -G Ninja \ -DCMAKE BUILD TYPE=Release \ -DCMAKE SYSTEM NAME=Linux \ -DCMAKE SYSTEM PROCESSOR=aarch64 \ -DCMAKE C COMPILER=aarch64-linux-gnu-gcc \ -DCMAKE CXX COMPILER=aarch64-linux-gnu-g++ 5.3 Compile ninja -C build-arm -j$ nproc Hasil: ls build-arm/bin Bahagian 6 — Semak Architecture Binary Gunakan: file build-arm/bin/llama-server Contoh output berjaya: ELF 64-bit LSB pie executable, ARM aarch64, dynamically linked Maksud: Output Maksud ELF 64-bit Binary 64-bit ARM aarch64 Untuk ARM64 dynamically linked Perlukan shared library PIE executable Linux security hardening Bahagian 7 — Semak Dependency .so Jangan guna ldd untuk cross binary Jika compile ARM64 tetapi check pada PC x86: ldd llama-server boleh gagal: not a dynamic executable Sebab: PC: x86 64 loader Binary: ARM64 loader Gunakan readelf aarch64-linux-gnu-readelf \ -d build-arm/bin/llama-server | grep NEEDED Contoh: Shared library: libllama.so Shared library: libggml.so Shared library: libstdc++.so.6 Cari semua .so find build-arm -name " .so" Contoh: libllama.so libggml.so libggml-base.so libggml-cpu.so Semak architecture: file build-arm/bin/ .so Output: ARM aarch64 Bahagian 8 — Dynamic vs Static Binary Semak: file llama-server Contoh dynamic: dynamically linked Perlu: lib .so Contoh static: statically linked Tidak perlu .so. Bahagian 9 — Installation ke Linux Pilihan standard Binary: /usr/local/bin Library: /usr/local/lib Contoh: sudo cp llama-server /usr/local/bin/ sudo cp llama-cli /usr/local/bin/ sudo cp .so /usr/local/lib/ sudo ldconfig Pilihan appliance / embedded Untuk SBC: /opt/llama.cpp/ llama-server llama-cli libllama.so libggml.so Kemudian: export LD LIBRARY PATH=/opt/llama.cpp Sesuai untuk: Orange Pi kiosk AI edge inference node Bahagian 10 — Deploy ke Orange Pi Copy: scp build-arm/bin/llama-server \ orangepi:/usr/local/bin/ scp build-arm/bin/llama-cli \ orangepi:/usr/local/bin/ Jika perlu: scp build-arm/bin/ .so \ orangepi:/usr/local/lib/ Pada Orange Pi: sudo ldconfig Semak: uname -m Expected: aarch64 Bahagian 11 — Cadangan Production Architecture Untuk sistem AI agent: +----------------+ | Go Agent | | Tool Router | +-------+--------+ | | HTTP API | v +----------------+ | llama-server | | llama.cpp | +----------------+ | | GGUF Model Kelebihan: Go agent tidak perlu embed model Model boleh tukar tanpa rebuild llama.cpp boleh upgrade sendiri Mudah scale ke banyak node Kesimpulan Workflow yang stabil: Native cmake -B build -G Ninja -DCMAKE BUILD TYPE=Release ninja -C build Cross Compile ARM64 sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu rm -rf build-arm cmake -B build-arm \ -G Ninja \ -DCMAKE SYSTEM NAME=Linux \ -DCMAKE SYSTEM PROCESSOR=aarch64 \ -DCMAKE C COMPILER=aarch64-linux-gnu-gcc \ -DCMAKE CXX COMPILER=aarch64-linux-gnu-g++ ninja -C build-arm Verification file llama-server aarch64-linux-gnu-readelf -d llama-server | grep NEEDED find . -name " .so" Dengan proses ini, satu mesin Debian 12/13 boleh menjadi build server untuk menghasilkan node AI ARM64 seperti Orange Pi, Raspberry Pi, atau edge inference appliance.