What If Java Apps Could Access CUDA Ecosystem Gracefully

TornadoVM now natively integrates NVIDIA CUDA libraries cuBLAS, cuFFT, and cuDNN, allowing Java developers to write GPU kernels in Java and call native CUDA functions without JNI or separate toolchains. The framework JIT-compiles JVM bytecode to CUDA, OpenCL, or Metal, and supports hybrid pipelines where Java kernels and native library calls share device buffers on the same CUDA stream. This enables extreme performance for Java-based AI and HPC workloads on NVIDIA GPUs.

TornadoVM JIT-compiles JVM bytecode to CUDA, OpenCL and Metal — and now natively integrates cuBLAS, cuFFT, and cuDNN inside your Java code. Write kernels in Java, keep your build on Maven, and let the runtime generate the GPU code. No JNI glue, no second toolchain, extreme performance . // You write this public static void matMulVec KernelContext ctx, HalfFloatArray w, FloatArray x, FloatArray out, int dim { int row = ctx.globalIdx; float sum = 0f; for int j = 0; j < dim; j++ { sum += w.get row dim + j .getFloat32 x.get j ; } out.set row, sum ; } js // TornadoVM generates this extern "C" global void matMulVec const half w, const float x, float out, int dim { int row = blockIdx.x blockDim.x + threadIdx.x; float sum = 0.0f; for int j = 0; j < dim; j++ sum = fmaf half2float w row dim + j , x j , sum ; out row = sum; } Llama 3, Mistral, Qwen 3, Phi-3, and DeepSeek models, loaded from GGUF and executed through TornadoVM's CUDA backend. FP16 and Q8 with fused dequantize–compute kernels. Watch it stream: Library tasks put native NVIDIA library calls inside a TornadoVM TaskGraph . They share TornadoVM-managed device buffers with your JIT-compiled kernels and run on the same CUDA stream — a kernel can feed cuBLAS or cuDNN and consume the output with no extra copies and no host synchronization. TaskGraph tg = new TaskGraph "cuBLAS" .transferToDevice EVERY EXECUTION, matrix, vector .task "preprocess", MyClass::preprocess, matrix // JIT-compiled Java .libraryTask "sgemv", CuBlas::cublasSgemv, // native cuBLAS CUBLAS OP T.operation , m, n, alpha, matrix, lda, vector, incx, beta, output, incy .task "postprocess", MyClass::postprocess, output // JIT-compiled Java .transferToHost EVERY EXECUTION, output ; Matrix–vector y = W·x , a memory-bound example, using the same device buffers three ways: the Loop Parallel API, the Kernel API, and the Hybrid API to call cuBLAS. public static void matMulVecParallel FloatArray w, FloatArray x, FloatArray out, int dim { for @Parallel int row = 0; row < dim; row++ { float sum = 0f; for int j = 0; j < dim; j++ { sum += w.get row dim + j x.get j ; } out.set row, sum ; } } public static void matMulVecKernel KernelContext ctx, FloatArray w, FloatArray x, FloatArray out, int dim { int row = ctx.globalIdx; float sum = 0f; for int j = 0; j < dim; j++ { sum += w.get row dim + j x.get j ; } out.set row, sum ; } // no kernel to write — the "task" is a // direct call into native cuBLAS: CuBlas::cublasSgemv CUBLAS OP T.operation , dim, dim, alpha, w, dim, x, 1, beta, out, 1 ; TornadoVM can dynamically generate CUDA C and also directly bind CUDA libraries inside your code. Mix and match as you feel, TornadoVM will reuse memory, combine operations and give you the maximum performance. All within your Java code. bash $ sdk install tornadovm bash $ tornado --devices bash $ java @$TORNADOVM HOME/tornado-argfile -cp $TORNADOVM HOME/share/java/tornado/tornado-examples-5.0.0-jdk21.jar uk.ac.manchester.tornado.examples.compute.MatrixVectorRowMajor