What If Java Apps Could Access CUDA Ecosystem Gracefully

TornadoVM JIT-compiles JVM bytecode to CUDA, OpenCL and Metal — and now natively integrates cuBLAS, cuFFT, and cuDNN inside your Java code. Write kernels in Java, keep your build on Maven, and let the runtime generate the GPU code. No JNI glue, no second toolchain, extreme performance.

// You write this
public static void matMulVec(KernelContext ctx,
        HalfFloatArray w, FloatArray x,
        FloatArray out, int dim) {
    int row = ctx.globalIdx;
    float sum = 0f;
    for (int j = 0; j < dim; j++) {
        sum += w.get(row * dim + j).getFloat32()
             * x.get(j);
    }
    out.set(row, sum);
}
js
// TornadoVM generates this
extern "C" __global__ void matMulVec(
    const __half* w, const float* x,
    float* out, int dim)
{
  int row = blockIdx.x * blockDim.x + threadIdx.x;
  float sum = 0.0f;
  for (int j = 0; j < dim; j++)
    sum = fmaf(__half2float(w[row * dim + j]),
               x[j], sum);
  out[row] = sum;
}

Llama 3, Mistral, Qwen 3, Phi-3, and DeepSeek models, loaded from GGUF and executed through TornadoVM's CUDA backend. FP16 and Q8 with fused dequantize–compute kernels. Watch it stream:

Library tasks

put native NVIDIA library calls inside a TornadoVM TaskGraph

. They share TornadoVM-managed device buffers with your JIT-compiled kernels and run on the same CUDA stream — a kernel can feed cuBLAS or cuDNN and consume the output with no extra copies and no host synchronization.

TaskGraph tg = new TaskGraph("cuBLAS")
    .transferToDevice(EVERY_EXECUTION, matrix, vector)
    .task("preprocess", MyClass::preprocess, matrix)      // JIT-compiled Java
    .libraryTask("sgemv", CuBlas::cublasSgemv,           // native cuBLAS
            CUBLAS_OP_T.operation(), m, n, alpha,
            matrix, lda, vector, incx, beta, output, incy)
    .task("postprocess", MyClass::postprocess, output)   // JIT-compiled Java
    .transferToHost(EVERY_EXECUTION, output);

Matrix–vector (y = W·x), a memory-bound example, using the same device buffers three ways: the Loop Parallel API, the Kernel API, and the Hybrid API to call cuBLAS.

public static void matMulVecParallel(
        FloatArray w, FloatArray x,
        FloatArray out, int dim) {
    for (@Parallel int row = 0; row < dim; row++) {
        float sum = 0f;
        for (int j = 0; j < dim; j++) {
            sum += w.get(row * dim + j) * x.get(j);
        }
        out.set(row, sum);
    }
}
public static void matMulVecKernel(KernelContext ctx,
        FloatArray w, FloatArray x,
        FloatArray out, int dim) {
    int row = ctx.globalIdx;
    float sum = 0f;
    for (int j = 0; j < dim; j++) {
        sum += w.get(row * dim + j) * x.get(j);
    }
    out.set(row, sum);
}
// no kernel to write — the "task" is a
// direct call into native cuBLAS:
CuBlas::cublasSgemv(
    CUBLAS_OP_T.operation(), dim, dim, alpha,
    w, dim, x, 1, beta, out, 1);

TornadoVM can dynamically generate CUDA C and also directly bind CUDA libraries inside your code. Mix and match as you feel, TornadoVM will reuse memory, combine operations and give you the maximum performance.

All within your Java code.

$ sdk install tornadovm
bash
$ tornado --devices
bash
$ java @$TORNADOVM_HOME/tornado-argfile -cp $TORNADOVM_HOME/share/java/tornado/tornado-examples-5.0.0-jdk21.jar uk.ac.manchester.tornado.examples.compute.MatrixVectorRowMajor