cd /news/ai-infrastructure/cent-runtime-c-17-runtime-and-c-abi-… Β· home β€Ί topics β€Ί ai-infrastructure β€Ί article
[ARTICLE Β· art-172996] src=dev.to β†— pub= topic=ai-infrastructure verified=true sentiment=↑ positive

cent_runtime: C++17 Runtime and C ABI for the CENT Near-Bank PIM Architecture

A developer built cent_runtime, a C++17 runtime with a C ABI and Python bindings that connects PyTorch linear layers from models such as Llama-2 7B to the CENT near-bank processing-in-memory instruction set architecture. The project targets the gap between the CENT paper's simulated gains for GPU-free LLM inference over CXL 3.0 and executable software, dispatching work to the official aim_simulator (Ramulator 2.0) and the GDDR6-AiM execution model. The report frames the effort around decoding's memory-bound GEMV phase, which accounts for roughly 98% of inference time.

key takeaway

cent_runtime provides a C++17/C ABI/Python layer that maps PyTorch linear layers onto the CENT PIM instruction set, letting Llama-2 7B layers run against the aim_simulator instead of requiring a custom software stack.

read12 min views7 publishedNov 20, 2026

The CENT architecture (PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference, Gu et al., ASPLOS 2025) proposes running Large Language Model (LLM) inference without GPUs, replacing HBM memory with GDDR6-AiM modules connected over CXL 3.0.

While the analytical and simulation gains reported in the paper are substantial, a practical gap separates academic results from software execution: there is no system layer capable of taking PyTorch tensors and dispatching them onto the architecture's execution model.

This technical report documents the architectural context, the physical bottlenecks motivating the project, the role of a hardware dispatcher, how the official simulator (aim_simulator / Ramulator 2.0) operates, and the implementation of cent_runtime β€” a C++17, C ABI, and Python library connecting linear layers of models such as Llama-2 7B to the CENT instruction set architecture. The seminar slide deck accompanying this work is available at the repository root in apresentacao_cent.pdf.

aim_simulator (Ramulator 2.0)cent_runtime In classical computer architecture (Patterson & Hennessy, Ch. 5), the memory hierarchy balances speed, capacity, and cost across successive tiers:

CPU / GPU  <--->  L1 Cache  <--->  L2 Cache  <--->  L3 Cache  <--->  Main Memory (DRAM)  <--->  SSD / Disk
 (fastest, smallest capacity)                                                        (slowest, largest capacity)

This organization relies on temporal and spatial locality: a block fetched from DRAM must be reused dozens or hundreds of times in caches before eviction. LLM inference breaks this assumption during its longest execution phase.

Inside a DRAM chip, data is organized into two-dimensional cell arrays. Activating a row loads an entire 8,192-bit block at once into the internal Row Buffer in roughly 1 internal access cycle β€” meaning high internal bandwidth exists within the memory die itself.

The bottleneck sits at the chip's output interface. Moving those 8,192 bits to the processor (CPU or GPU) requires crossing the external bus and physical I/O pins operating at roughly 64 bits per cycle (or 32 bits in specific pin configurations). Transferring a single row that was already sitting in the Row Buffer takes roughly 128 bus cycles. While data crosses the bus, GPU compute units remain in a stall state waiting on memory.

GEMM) vs. Decoding ( GEMV) LLM inference splits into two phases with opposite physical profiles:

Characteristic Phase 1: Prefill (Initial Prompt) Phase 2: Decoding (Token-by-Token Generation)
Share of total runtime ~2% of inference time ~98% of inference time
Mathematical operation Matrix Γ— Matrix ( GEMM , e.g.,4KΓ—4K Γ—4KΓ—4K ) Vector Γ— Matrix ( GEMV ,batch_size = 1 )
Arithmetic intensity High (~2,000 Ops/Byte) Low (1 to 16 Ops/Byte)
Execution regime Compute-Bound (saturates Tensor Cores) Memory-Bound (limited by memory bandwidth)

During decoding, generating each token requires reading entire weight matrices (tens of megabytes per layer, totaling tens of gigabytes across the model) only to compute a single vector-matrix product and discard the weights immediately after. On a 4Γ— NVIDIA A100 node, this lack of reuse drops effective Tensor Core utilization to roughly 21% (with 79% of execution time spent stalled on HBM reads).

Processing-in-Memory (PIM) β€” specifically the GDDR6-AiM (Accelerator-in-Memory) design adopted in CENT β€” places multiply-accumulate execution units (PUs/ALUs) directly beside each DRAM bank, ahead of the narrow I/O pin interface.

Each GDDR6-AiM chip therefore consumes weights directly at the Row Buffer output without streaming weight matrices over the external bus:

Instead of dedicated GPUs and HBM interposers, CENT connects a Host CPU to 32 CXL 3.0 devices through a 48-port CXL Switch:

                       +------------+
                       |  Host CPU  |
                       +-----+------+
                             |
                             v
                   +--------------------+
                   |   CXL 3.0 Switch   |
                   +---------+----------+
                             |
        +--------------------+--------------------+
        |                    |                    |
        v                    v                    v
+---------------+    +---------------+    +---------------+
| CXL Device #0 |    | CXL Device #1 | .. | CXL Device #31|
+-------+-------+    +---------------+    +---------------+
        |
        +---> CXL Port + Inter-device Communication Controller
        +---> Instruction Buffer (2 MB SRAM) -> PC -> Decoder
        +---> PNM Units + Shared Buffer (64 KB)
        +---> 16x PIM Ctrl ---> 16x GDDR6-PIM (32 channels per board)

Inside each CXL device, execution is divided across two tiers:

Softmax, RMSNorm, SiLU, and RoPE, transferring only compact partial vectors (<16 KB). In the paper's evaluation against a 4Γ— NVIDIA A100 server (80 GB per GPU):

At the same time, the architecture has three clear technical limitations:

This third gap is the focus of our work: building the system layer that connects high-level tensors to the architecture's instruction format and cycle-accurate simulator.

To validate the microarchitecture, the CENT authors released aim_simulator, built on top of Ramulator 2.0 (a cycle-accurate academic DRAM simulator).

aim_simulator compiles into a ramulator2 executable (paired with libramulator.so). It is not a neural network runtime, but a discrete-event hardware simulator:

  [ configs/gddr6_aim.yaml ] ──┐
  (Physical parameters:        β”‚
   32 channels, 16 banks,      β”œβ”€β”€β–Ί [ ramulator2 (aim_simulator) ] ──► Silicon Counters
   GDDR6 timings, FRFCFS)      β”‚                                       (e.g. CH31_AiM_ISR_WR_GB_cycles,
                               β”‚                                             memory_system_cycles)
  [ file.trace ] β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  (Static list of textual
   CENT ISA commands)

When invoked from the command line (ramulator2 -f configs/gddr6_aim.yaml -t file.trace), it models Row Buffer accesses, bank conflicts, FRFCFS command scheduling, AllBank refresh, and AiM instructions cycle by cycle across 32 GDDR6-AiM channels, printing per-instruction cycle counters on completion.

.trace files:AiM WR_GB, AiM MAC_ABK, etc., already decomposed into physical rows and channel masks.y for the next model layer. In systems software, a dispatcher is the intermediary layer sitting between a high-level framework and the accelerator controller. When PyTorch runs torch.matmul(x, W) on a GPU, the call passes through the internal dispatcher into native C/C++ libraries (libcudart.so, libcublas.so) that configure and submit work to the device.

For CENT, the hardware dispatcher fulfills four responsibilities:

batch_size = 1). Prefill batch_size > 1) are rejected so they remain on the host. DVid) hosts the layer and which of the 32 GDDR6 channels ( CHmask) are enabled for dimension K.[M Γ— K] weight matrix into blocks aligned to the 16 DRAM banks per channel, writes the binary descriptors into the 2 MB SRAM buffer, and emits the corresponding cent_runtime structures this dispatcher into two main layers β€” Python (PyTorch/NumPy) and C++17 connected across a stable C ABI (libcent_runtime.so) β€” plus an optional integration module for ramulator2.

+-----------------------------------------------------------------------------+
| 1. PYTHON LAYER (python/cent_runtime/)                                      |
|    cent_runtime.matmul(x, W, layer_id=0, trace_path=None, simulate=False)   |
|    - Validates GEMV contract (batch=1) in ops.py                            |
|    - Normalizes to contiguous float32 and extracts pointer via ctypes       |
+--------------------------------------+--------------------------------------+
                                       | ctypes (_bindings.py)
                                       v
======================= [ C ABI BOUNDARY (extern "C") ] =======================
| 2. PUBLIC CONTRACT (include/cent/runtime_api.h -> build/libcent_runtime.so) |
|    cent_dispatch_gemv(x, M, K, W, y, layer_id, trace_path, &telemetry)      |
+--------------------------------------+--------------------------------------+
                                       |
          +----------------------------+----------------------------+
          v                                                         v
+-----------------------------------+     +-----------------------------------+
| 3. C++17 ENGINE (src/)            |     | 4. NUMERICAL COMPUTATION          |
|  - topology.cpp   -> DVid/CHmask  |     |  - Row-major loop over W [M x K]  |
|  - sram_buffer.cpp -> 2 MB SRAM   |     |  - FP64 accumulation -> FP32 out  |
|  - isa_emitter.cpp -> PIM blocks  |     |  - Writes directly to pointer 'y' |
+-----------------+-----------------+     +-----------------------------------+
                  |
                  +---> Binary packets (14 B) in SRAM + Hex Dump
                  +---> .trace file (CENT ISA) ---> simulator.py -> ramulator2

python/cent_runtime/) The Python package exposes cent_runtime.matmul, simulate_on_ramulator, and get_instruction_buffer_hex:

ops.py: torch.Tensor or numpy.ndarray (falling back to NumPy when PyTorch is not installed). Validates x as (K,) or (1, K) and W as (M, K). 3D+ tensors or ValueError. It then ensures a contiguous float32 layout on CPU (performing no extra copy when the tensor already satisfies this layout), allocates the output tensor ctypes._bindings.py: build/libcent_runtime.so (running make shared automatically if the library has not been built yet) and maps the C struct CentDispatchTelemetry (56 bytes on x86_64) into a Python dictionary.simulator.py: simulate=True (creating a temporary tempfile if trace_path=None), resolves the ramulator2 binary in configurable order (explicit ramulator_bin β†’ CENT_RAMULATOR_BIN environment variable β†’ external/ramulator2 β†’ ../CENT/aim_simulator/build/ramulator2 β†’ PATH), invokes the simulator via subprocess, and extracts wr_gb_cycles, mac_abk_cycles, and memory_system_cycles..h) and C++17 Core (.hpp) In C++, namespaced or overloaded functions undergo name mangling (for instance, cent::compute_dvid(int) becomes _ZN4cent12compute_dvidEi in the symbol table), and the C++ ABI differs across compilers. To guarantee direct FFI interoperability with ctypes, the codebase separates:

isa.h, telemetry.h, runtime_api.h): extern "C", exporting unmangled symbols ( cent_init_runtime, cent_dispatch_gemv, cent_get_instruction_buffer_hex). topology.hpp, sram_buffer.hpp, isa_emitter.hpp): namespace cent, classes, RAII, and std::array. The native library compiles with -std=c++17 -O3 -fPIC -Wall -Wextra -Werror and uses no C++ exceptions (throw / try-catch) across the boundary. All exported functions are marked noexcept and return explicit status codes via enum CentStatus (CENT_OK = 0, CENT_ERR_NULL_PTR = -1, CENT_ERR_INVALID_SHAPE = -2, CENT_ERR_SRAM_OVERFLOW = -3, CENT_ERR_TRACE_IO = -4).

CentInstruction) include/cent/isa.h defines the controller parameters (CENT_SRAM_BUFFER_BYTES = 2 MB, 32 CXL devices, 32 channels per device, 16 banks per channel, and opsize = 64 bursts), the 5 CENT ISA opcodes, and the packed binary instruction layout:

enum CentOpcode : uint8_t {
    CENT_OP_WR_GB   = 0x02, // Writes activation vector to active channels' Global Buffer
    CENT_OP_WR_BIAS = 0x03, // Initializes accumulator registers across DRAM banks
    CENT_OP_RD_MAC  = 0x05, // Reads accumulated result from banks to channel buffer
    CENT_OP_MAC_ABK = 0x0F, // Parallel Multiply-Accumulate across all 16 banks (All-Bank)
    CENT_OP_EOC     = 0xFF  // End of Computation (terminates the aim_simulator trace)
};

#pragma pack(push, 1)
typedef struct CentInstruction {
    uint8_t  opcode;       //  8 bits : CENT ISA opcode
    uint8_t  dvid;         //  8 bits : Target CXL device ID (0 to 31)
    uint32_t channel_mask; // 32 bits : Bitmask of the 32 GDDR6 channels
    uint16_t bank_index;   // 16 bits : DRAM bank index (0 to 15)
    uint32_t row_addr;     // 32 bits : Physical row address (Row Buffer)
    uint16_t opsize;       // 16 bits : Operation size in bursts
} CentInstruction;
#pragma pack(pop)

static_assert(sizeof(CentInstruction) == 14,
              "ABI Error: CentInstruction must be exactly 14 bytes packed");

The #pragma pack(push, 1) directive verified by static_assert eliminates alignment padding between fields, fixing each instruction at 14 contiguous bytes (1 + 1 + 4 + 2 + 4 + 2).

During cent_dispatch_gemv (src/runtime_api.cpp), four stages run in sequence:

src/topology.cpp): DVid = ((layer_id % 32) + 32) % 32 to distribute layers across the 32 CXL devices. The channel mask 0xFFFFFFFFu) when K >= 1024, 16 channels ( 0x0000FFFFu) when K >= 512, and 8 channels ( 0x000000FFu) for smaller dimensions. src/sram_buffer.cpp): SramBuffer manages a preallocated static 2 MB std::array<uint8_t, CENT_SRAM_BUFFER_BYTES>, avoiding dynamic allocation ( new/ delete) during dispatch. Instructions are copied via std::memcpy with bounds checking (CENT_ERR_SRAM_OVERFLOW) and can be inspected in hexadecimal through format_hex_dump. src/isa_emitter.cpp): M is partitioned into num_blocks = (M + 15) / 16 blocks. For each block blk (row_addr = blk), the emitter writes 4 instructions into SRAM ( WR_GB, WR_BIAS, MAC_ABK, and RD_MAC) and, when trace_path is provided, writes the corresponding lines to the

   AiM WR_GB 64 0 0xffffffff
   AiM WR_BIAS 0 0xffffffff
   AiM MAC_ABK 64 0xffffffff 0
   AiM RD_MAC 0 0xffffffff

After all blocks complete, it emits a CENT_OP_EOC instruction (AiM EOC), required to terminate simulation in aim_simulator.

src/runtime_api.cpp and y = x Β· W^T in row-major order using a double (FP64) accumulator cast to float (FP32) in the output pointer (M Γ— K + K) Γ— 2 bytes, transferring both (K + M) Γ— 2 bytes, where only The repository includes automated Python unit tests (make test), a native C++ testbench ( make test-core), and a validation script exercising real Llama-2 7B layer dimensions (examples/llama2_7b_demo.py).

git clone https://github.com/josedercysilva/cent-pim-runtime.git
cd cent-pim-runtime

./setup_env.sh
source .venv/bin/activate

make test        # Tests C ABI, numerical accuracy, GEMM rejection, SRAM, and .trace
make test-core   # Native C++ testbench
make demo        # Runs Llama-2 7B Attention and MLP layers
make simulate    # Optional: runs Ramulator 2.0 on the generated .trace files
python
import torch
import torch.nn.functional as F
import cent_runtime

x = torch.randn(1, 4096)        # Input activation
W = torch.randn(4096, 4096)     # Stationary weights in GDDR6-AiM memory

y, telemetry = cent_runtime.matmul(x, W, layer_id=0)

out = F.silu(y)

print(f"Output: {out.shape}")
print(f"CXL Board: #{telemetry['dvid']} | Mask: {telemetry['channel_mask_hex']}")
print(f"Instructions: {telemetry['total_instructions']} ({telemetry['instruction_buffer_bytes']} B SRAM)")
print(f"Bus traffic saved: {telemetry['memory_traffic_saved_pct']:.2f}%")

The table below summarizes the results obtained for the Attention projection (q/k/v/o_proj, M = 4096, K = 4096, layer_id = 0) and the MLP projection ( gate/up/down_proj, M = 11008, K = 4096, layer_id = 1):

Metric Attention Layer ( 4096 Γ— 4096 ) MLP Layer ( 11008 Γ— 4096 )
CXL device ( DVid ) Board #0 Board #1
Active channels ( CHmask ) 0xffffffff (32 channels) 0xffffffff (32 channels)
16-bank blocks ( ceil(M/16) ) 256 blocks 688 blocks
Total instructions ( 4/block + EOC ) 1,025 ( 256Γ—4 + 1 ) 2,753 ( 688Γ—4 + 1 )
SRAM footprint ( 14 B/inst ) 14,350 bytes (14.01 KB) 38,542 bytes (37.64 KB)
GPU HBM vs. CENT CXL traffic (BF16) ~32.01 MB vs. 16.00 KB (99.95% lower) ~86.01 MB vs. 29.50 KB (99.97% lower)
Ramulator 2.0 cycles ( CH31_AiM_ISR_WR_GB_cycles ) 16,416 cycles 44,064 cycles

The ratio between the two layers reflects the matrix dimensions directly: the MLP output dimension is 11,008 / 4,096 = 2.6875Γ— larger than Attention, and both the total SRAM instruction count and the silicon cycle count reported by aim_simulator scale by the same 2.68Γ— factor (44,064 / 16,416).

Inspecting the first 16 SRAM bytes via cent_get_instruction_buffer_hex shows the first 14-byte packet serialized in little-endian order (opcode = 0x02 for WR_GB, dvid = 0x00, channel_mask = FF FF FF FF, bank = 00 00, row = 00 00 00 00, opsize = 64 = 0x0040 -> 40 00), followed by the first two bytes of the second instruction (03 00, WR_BIAS):

[SRAM] 02 00 FF FF FF FF 00 00 00 00 00 00 40 00 03 00

cent_runtime connects the CENT architectural specification to an executable Python and C++17 workflow:

josedercysilva/cent-pim-runtime`` apresentacao_cent.pdf

── more in #ai-infrastructure 4 stories Β· sorted by recency
── more on @cent_runtime 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/cent-runtime-c-17-ru…] indexed:0 read:12min 2026-11-20 Β· β€”