The CENT architecture (PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference, Gu et al., ASPLOS 2025) proposes running Large Language Model (LLM) inference without GPUs, replacing HBM memory with GDDR6-AiM modules connected over CXL 3.0.
While the analytical and simulation gains reported in the paper are substantial, a practical gap separates academic results from software execution: there is no system layer capable of taking PyTorch tensors and dispatching them onto the architecture's execution model.
This technical report documents the architectural context, the physical bottlenecks motivating the project, the role of a hardware dispatcher, how the official simulator (aim_simulator / Ramulator 2.0) operates, and the implementation of cent_runtime β a C++17, C ABI, and Python library connecting linear layers of models such as Llama-2 7B to the CENT instruction set architecture. The seminar slide deck accompanying this work is available at the repository root in apresentacao_cent.pdf.
aim_simulator (Ramulator 2.0)cent_runtime
In classical computer architecture (Patterson & Hennessy, Ch. 5), the memory hierarchy balances speed, capacity, and cost across successive tiers:
CPU / GPU <---> L1 Cache <---> L2 Cache <---> L3 Cache <---> Main Memory (DRAM) <---> SSD / Disk
(fastest, smallest capacity) (slowest, largest capacity)
This organization relies on temporal and spatial locality: a block fetched from DRAM must be reused dozens or hundreds of times in caches before eviction. LLM inference breaks this assumption during its longest execution phase.
Inside a DRAM chip, data is organized into two-dimensional cell arrays. Activating a row loads an entire 8,192-bit block at once into the internal Row Buffer in roughly 1 internal access cycle β meaning high internal bandwidth exists within the memory die itself.
The bottleneck sits at the chip's output interface. Moving those 8,192 bits to the processor (CPU or GPU) requires crossing the external bus and physical I/O pins operating at roughly 64 bits per cycle (or 32 bits in specific pin configurations). Transferring a single row that was already sitting in the Row Buffer takes roughly 128 bus cycles. While data crosses the bus, GPU compute units remain in a stall state waiting on memory.
GEMM) vs. Decoding ( GEMV)
LLM inference splits into two phases with opposite physical profiles:
| Characteristic | Phase 1: Prefill (Initial Prompt) | Phase 2: Decoding (Token-by-Token Generation) |
|---|---|---|
| Share of total runtime | ~2% of inference time | ~98% of inference time |
| Mathematical operation | Matrix Γ Matrix ( GEMM , e.g.,4KΓ4K Γ4KΓ4K ) |
Vector Γ Matrix ( GEMV ,batch_size = 1 ) |
| Arithmetic intensity | High (~2,000 Ops/Byte) | Low (1 to 16 Ops/Byte) |
| Execution regime | Compute-Bound (saturates Tensor Cores) | Memory-Bound (limited by memory bandwidth) |
During decoding, generating each token requires reading entire weight matrices (tens of megabytes per layer, totaling tens of gigabytes across the model) only to compute a single vector-matrix product and discard the weights immediately after. On a 4Γ NVIDIA A100 node, this lack of reuse drops effective Tensor Core utilization to roughly 21% (with 79% of execution time spent stalled on HBM reads).
Processing-in-Memory (PIM) β specifically the GDDR6-AiM (Accelerator-in-Memory) design adopted in CENT β places multiply-accumulate execution units (PUs/ALUs) directly beside each DRAM bank, ahead of the narrow I/O pin interface.
Each GDDR6-AiM chip therefore consumes weights directly at the Row Buffer output without streaming weight matrices over the external bus:
Instead of dedicated GPUs and HBM interposers, CENT connects a Host CPU to 32 CXL 3.0 devices through a 48-port CXL Switch:
+------------+
| Host CPU |
+-----+------+
|
v
+--------------------+
| CXL 3.0 Switch |
+---------+----------+
|
+--------------------+--------------------+
| | |
v v v
+---------------+ +---------------+ +---------------+
| CXL Device #0 | | CXL Device #1 | .. | CXL Device #31|
+-------+-------+ +---------------+ +---------------+
|
+---> CXL Port + Inter-device Communication Controller
+---> Instruction Buffer (2 MB SRAM) -> PC -> Decoder
+---> PNM Units + Shared Buffer (64 KB)
+---> 16x PIM Ctrl ---> 16x GDDR6-PIM (32 channels per board)
Inside each CXL device, execution is divided across two tiers:
Softmax, RMSNorm, SiLU, and RoPE, transferring only compact partial vectors (<16 KB).
In the paper's evaluation against a 4Γ NVIDIA A100 server (80 GB per GPU):
At the same time, the architecture has three clear technical limitations:
This third gap is the focus of our work: building the system layer that connects high-level tensors to the architecture's instruction format and cycle-accurate simulator.
To validate the microarchitecture, the CENT authors released aim_simulator, built on top of Ramulator 2.0 (a cycle-accurate academic DRAM simulator).
aim_simulator compiles into a ramulator2 executable (paired with libramulator.so). It is not a neural network runtime, but a discrete-event hardware simulator:
[ configs/gddr6_aim.yaml ] βββ
(Physical parameters: β
32 channels, 16 banks, ββββΊ [ ramulator2 (aim_simulator) ] βββΊ Silicon Counters
GDDR6 timings, FRFCFS) β (e.g. CH31_AiM_ISR_WR_GB_cycles,
β memory_system_cycles)
[ file.trace ] βββββββββββββββ
(Static list of textual
CENT ISA commands)
When invoked from the command line (ramulator2 -f configs/gddr6_aim.yaml -t file.trace), it models Row Buffer accesses, bank conflicts, FRFCFS command scheduling, AllBank refresh, and AiM instructions cycle by cycle across 32 GDDR6-AiM channels, printing per-instruction cycle counters on completion.
.trace files:AiM WR_GB, AiM MAC_ABK, etc., already decomposed into physical rows and channel masks.y for the next model layer.
In systems software, a dispatcher is the intermediary layer sitting between a high-level framework and the accelerator controller. When PyTorch runs torch.matmul(x, W) on a GPU, the call passes through the internal dispatcher into native C/C++ libraries (libcudart.so, libcublas.so) that configure and submit work to the device.
For CENT, the hardware dispatcher fulfills four responsibilities:
batch_size = 1). Prefill batch_size > 1) are rejected so they remain on the host. DVid) hosts the layer and which of the 32 GDDR6 channels ( CHmask) are enabled for dimension K.[M Γ K] weight matrix into blocks aligned to the 16 DRAM banks per channel, writes the binary descriptors into the 2 MB SRAM buffer, and emits the corresponding cent_runtime structures this dispatcher into two main layers β Python (PyTorch/NumPy) and C++17 connected across a stable C ABI (libcent_runtime.so) β plus an optional integration module for ramulator2.
+-----------------------------------------------------------------------------+
| 1. PYTHON LAYER (python/cent_runtime/) |
| cent_runtime.matmul(x, W, layer_id=0, trace_path=None, simulate=False) |
| - Validates GEMV contract (batch=1) in ops.py |
| - Normalizes to contiguous float32 and extracts pointer via ctypes |
+--------------------------------------+--------------------------------------+
| ctypes (_bindings.py)
v
======================= [ C ABI BOUNDARY (extern "C") ] =======================
| 2. PUBLIC CONTRACT (include/cent/runtime_api.h -> build/libcent_runtime.so) |
| cent_dispatch_gemv(x, M, K, W, y, layer_id, trace_path, &telemetry) |
+--------------------------------------+--------------------------------------+
|
+----------------------------+----------------------------+
v v
+-----------------------------------+ +-----------------------------------+
| 3. C++17 ENGINE (src/) | | 4. NUMERICAL COMPUTATION |
| - topology.cpp -> DVid/CHmask | | - Row-major loop over W [M x K] |
| - sram_buffer.cpp -> 2 MB SRAM | | - FP64 accumulation -> FP32 out |
| - isa_emitter.cpp -> PIM blocks | | - Writes directly to pointer 'y' |
+-----------------+-----------------+ +-----------------------------------+
|
+---> Binary packets (14 B) in SRAM + Hex Dump
+---> .trace file (CENT ISA) ---> simulator.py -> ramulator2
python/cent_runtime/)
The Python package exposes cent_runtime.matmul, simulate_on_ramulator, and get_instruction_buffer_hex:
ops.py: torch.Tensor or numpy.ndarray (falling back to NumPy when PyTorch is not installed). Validates x as (K,) or (1, K) and W as (M, K). 3D+ tensors or ValueError. It then ensures a contiguous float32 layout on CPU (performing no extra copy when the tensor already satisfies this layout), allocates the output tensor ctypes._bindings.py: build/libcent_runtime.so (running make shared automatically if the library has not been built yet) and maps the C struct CentDispatchTelemetry (56 bytes on x86_64) into a Python dictionary.simulator.py: simulate=True (creating a temporary tempfile if trace_path=None), resolves the ramulator2 binary in configurable order (explicit ramulator_bin β CENT_RAMULATOR_BIN environment variable β external/ramulator2 β ../CENT/aim_simulator/build/ramulator2 β PATH), invokes the simulator via subprocess, and extracts wr_gb_cycles, mac_abk_cycles, and memory_system_cycles..h) and C++17 Core (.hpp)
In C++, namespaced or overloaded functions undergo name mangling (for instance, cent::compute_dvid(int) becomes _ZN4cent12compute_dvidEi in the symbol table), and the C++ ABI differs across compilers. To guarantee direct FFI interoperability with ctypes, the codebase separates:
isa.h, telemetry.h, runtime_api.h): extern "C", exporting unmangled symbols ( cent_init_runtime, cent_dispatch_gemv, cent_get_instruction_buffer_hex). topology.hpp, sram_buffer.hpp, isa_emitter.hpp): namespace cent, classes, RAII, and std::array.
The native library compiles with -std=c++17 -O3 -fPIC -Wall -Wextra -Werror and uses no C++ exceptions (throw / try-catch) across the boundary. All exported functions are marked noexcept and return explicit status codes via enum CentStatus (CENT_OK = 0, CENT_ERR_NULL_PTR = -1, CENT_ERR_INVALID_SHAPE = -2, CENT_ERR_SRAM_OVERFLOW = -3, CENT_ERR_TRACE_IO = -4).
CentInstruction)
include/cent/isa.h defines the controller parameters (CENT_SRAM_BUFFER_BYTES = 2 MB, 32 CXL devices, 32 channels per device, 16 banks per channel, and opsize = 64 bursts), the 5 CENT ISA opcodes, and the packed binary instruction layout:
enum CentOpcode : uint8_t {
CENT_OP_WR_GB = 0x02, // Writes activation vector to active channels' Global Buffer
CENT_OP_WR_BIAS = 0x03, // Initializes accumulator registers across DRAM banks
CENT_OP_RD_MAC = 0x05, // Reads accumulated result from banks to channel buffer
CENT_OP_MAC_ABK = 0x0F, // Parallel Multiply-Accumulate across all 16 banks (All-Bank)
CENT_OP_EOC = 0xFF // End of Computation (terminates the aim_simulator trace)
};
#pragma pack(push, 1)
typedef struct CentInstruction {
uint8_t opcode; // 8 bits : CENT ISA opcode
uint8_t dvid; // 8 bits : Target CXL device ID (0 to 31)
uint32_t channel_mask; // 32 bits : Bitmask of the 32 GDDR6 channels
uint16_t bank_index; // 16 bits : DRAM bank index (0 to 15)
uint32_t row_addr; // 32 bits : Physical row address (Row Buffer)
uint16_t opsize; // 16 bits : Operation size in bursts
} CentInstruction;
#pragma pack(pop)
static_assert(sizeof(CentInstruction) == 14,
"ABI Error: CentInstruction must be exactly 14 bytes packed");
The #pragma pack(push, 1) directive verified by static_assert eliminates alignment padding between fields, fixing each instruction at 14 contiguous bytes (1 + 1 + 4 + 2 + 4 + 2).
During cent_dispatch_gemv (src/runtime_api.cpp), four stages run in sequence:
src/topology.cpp): DVid = ((layer_id % 32) + 32) % 32 to distribute layers across the 32 CXL devices. The channel mask 0xFFFFFFFFu) when K >= 1024, 16 channels ( 0x0000FFFFu) when K >= 512, and 8 channels ( 0x000000FFu) for smaller dimensions. src/sram_buffer.cpp): SramBuffer manages a preallocated static 2 MB std::array<uint8_t, CENT_SRAM_BUFFER_BYTES>, avoiding dynamic allocation ( new/ delete) during dispatch. Instructions are copied via std::memcpy with bounds checking (CENT_ERR_SRAM_OVERFLOW) and can be inspected in hexadecimal through format_hex_dump. src/isa_emitter.cpp): M is partitioned into num_blocks = (M + 15) / 16 blocks. For each block blk (row_addr = blk), the emitter writes 4 instructions into SRAM ( WR_GB, WR_BIAS, MAC_ABK, and RD_MAC) and, when trace_path is provided, writes the corresponding lines to the
AiM WR_GB 64 0 0xffffffff
AiM WR_BIAS 0 0xffffffff
AiM MAC_ABK 64 0xffffffff 0
AiM RD_MAC 0 0xffffffff
After all blocks complete, it emits a CENT_OP_EOC instruction (AiM EOC), required to terminate simulation in aim_simulator.
src/runtime_api.cpp and y = x Β· W^T in row-major order using a double (FP64) accumulator cast to float (FP32) in the output pointer (M Γ K + K) Γ 2 bytes, transferring both (K + M) Γ 2 bytes, where only The repository includes automated Python unit tests (make test), a native C++ testbench ( make test-core), and a validation script exercising real Llama-2 7B layer dimensions (examples/llama2_7b_demo.py).
git clone https://github.com/josedercysilva/cent-pim-runtime.git
cd cent-pim-runtime
./setup_env.sh
source .venv/bin/activate
make test # Tests C ABI, numerical accuracy, GEMM rejection, SRAM, and .trace
make test-core # Native C++ testbench
make demo # Runs Llama-2 7B Attention and MLP layers
make simulate # Optional: runs Ramulator 2.0 on the generated .trace files
python
import torch
import torch.nn.functional as F
import cent_runtime
x = torch.randn(1, 4096) # Input activation
W = torch.randn(4096, 4096) # Stationary weights in GDDR6-AiM memory
y, telemetry = cent_runtime.matmul(x, W, layer_id=0)
out = F.silu(y)
print(f"Output: {out.shape}")
print(f"CXL Board: #{telemetry['dvid']} | Mask: {telemetry['channel_mask_hex']}")
print(f"Instructions: {telemetry['total_instructions']} ({telemetry['instruction_buffer_bytes']} B SRAM)")
print(f"Bus traffic saved: {telemetry['memory_traffic_saved_pct']:.2f}%")
The table below summarizes the results obtained for the Attention projection (q/k/v/o_proj, M = 4096, K = 4096, layer_id = 0) and the MLP projection ( gate/up/down_proj, M = 11008, K = 4096, layer_id = 1):
| Metric | Attention Layer ( 4096 Γ 4096 ) |
MLP Layer ( 11008 Γ 4096 ) |
|---|---|---|
CXL device ( DVid ) |
Board #0 |
Board #1 |
Active channels ( CHmask ) |
0xffffffff (32 channels) |
0xffffffff (32 channels) |
16-bank blocks ( ceil(M/16) ) |
256 blocks | 688 blocks |
Total instructions ( 4/block + EOC ) |
1,025 ( 256Γ4 + 1 ) |
2,753 ( 688Γ4 + 1 ) |
SRAM footprint ( 14 B/inst ) |
14,350 bytes (14.01 KB) | 38,542 bytes (37.64 KB) |
| GPU HBM vs. CENT CXL traffic (BF16) | ~32.01 MB vs. 16.00 KB (99.95% lower) | ~86.01 MB vs. 29.50 KB (99.97% lower) |
Ramulator 2.0 cycles ( CH31_AiM_ISR_WR_GB_cycles ) |
16,416 cycles | 44,064 cycles |
The ratio between the two layers reflects the matrix dimensions directly: the MLP output dimension is 11,008 / 4,096 = 2.6875Γ larger than Attention, and both the total SRAM instruction count and the silicon cycle count reported by aim_simulator scale by the same 2.68Γ factor (44,064 / 16,416).
Inspecting the first 16 SRAM bytes via cent_get_instruction_buffer_hex shows the first 14-byte packet serialized in little-endian order (opcode = 0x02 for WR_GB, dvid = 0x00, channel_mask = FF FF FF FF, bank = 00 00, row = 00 00 00 00, opsize = 64 = 0x0040 -> 40 00), followed by the first two bytes of the second instruction (03 00, WR_BIAS):
[SRAM] 02 00 FF FF FF FF 00 00 00 00 00 00 40 00 03 00
cent_runtime connects the CENT architectural specification to an executable Python and C++17 workflow:
josedercysilva/cent-pim-runtime`` apresentacao_cent.pdf