{"slug": "what-if-java-apps-could-access-cuda-ecosystem-gracefully", "title": "What If Java Apps Could Access CUDA Ecosystem Gracefully", "summary": "TornadoVM now natively integrates NVIDIA CUDA libraries cuBLAS, cuFFT, and cuDNN, allowing Java developers to write GPU kernels in Java and call native CUDA functions without JNI or separate toolchains. The framework JIT-compiles JVM bytecode to CUDA, OpenCL, or Metal, and supports hybrid pipelines where Java kernels and native library calls share device buffers on the same CUDA stream. This enables extreme performance for Java-based AI and HPC workloads on NVIDIA GPUs.", "body_md": "TornadoVM JIT-compiles JVM bytecode to CUDA, OpenCL and Metal — and **now** natively integrates cuBLAS, cuFFT, and cuDNN inside your Java code. Write kernels in Java, keep your build on Maven, and let the runtime generate the GPU code. No JNI glue, no second toolchain, **extreme performance**.\n\n```\n// You write this\npublic static void matMulVec(KernelContext ctx,\n        HalfFloatArray w, FloatArray x,\n        FloatArray out, int dim) {\n    int row = ctx.globalIdx;\n    float sum = 0f;\n    for (int j = 0; j < dim; j++) {\n        sum += w.get(row * dim + j).getFloat32()\n             * x.get(j);\n    }\n    out.set(row, sum);\n}\njs\n// TornadoVM generates this\nextern \"C\" __global__ void matMulVec(\n    const __half* w, const float* x,\n    float* out, int dim)\n{\n  int row = blockIdx.x * blockDim.x + threadIdx.x;\n  float sum = 0.0f;\n  for (int j = 0; j < dim; j++)\n    sum = fmaf(__half2float(w[row * dim + j]),\n               x[j], sum);\n  out[row] = sum;\n}\n```\n\nLlama 3, Mistral, Qwen 3, Phi-3, and DeepSeek models, loaded from GGUF and executed through TornadoVM's CUDA backend. FP16 and Q8 with fused dequantize–compute kernels. Watch it stream:\n\n`Library tasks`\n\nput native NVIDIA library calls inside a TornadoVM `TaskGraph`\n\n. They share TornadoVM-managed device buffers with your JIT-compiled kernels and run on the same CUDA stream — a kernel can feed cuBLAS or cuDNN and consume the output with no extra copies and no host synchronization.\n\n```\nTaskGraph tg = new TaskGraph(\"cuBLAS\")\n    .transferToDevice(EVERY_EXECUTION, matrix, vector)\n    .task(\"preprocess\", MyClass::preprocess, matrix)      // JIT-compiled Java\n    .libraryTask(\"sgemv\", CuBlas::cublasSgemv,           // native cuBLAS\n            CUBLAS_OP_T.operation(), m, n, alpha,\n            matrix, lda, vector, incx, beta, output, incy)\n    .task(\"postprocess\", MyClass::postprocess, output)   // JIT-compiled Java\n    .transferToHost(EVERY_EXECUTION, output);\n```\n\nMatrix–vector (y = W·x), a memory-bound example, using the same device buffers three ways: the Loop Parallel API, the Kernel API, and the Hybrid API to call cuBLAS.\n\n```\npublic static void matMulVecParallel(\n        FloatArray w, FloatArray x,\n        FloatArray out, int dim) {\n    for (@Parallel int row = 0; row < dim; row++) {\n        float sum = 0f;\n        for (int j = 0; j < dim; j++) {\n            sum += w.get(row * dim + j) * x.get(j);\n        }\n        out.set(row, sum);\n    }\n}\npublic static void matMulVecKernel(KernelContext ctx,\n        FloatArray w, FloatArray x,\n        FloatArray out, int dim) {\n    int row = ctx.globalIdx;\n    float sum = 0f;\n    for (int j = 0; j < dim; j++) {\n        sum += w.get(row * dim + j) * x.get(j);\n    }\n    out.set(row, sum);\n}\n// no kernel to write — the \"task\" is a\n// direct call into native cuBLAS:\nCuBlas::cublasSgemv(\n    CUBLAS_OP_T.operation(), dim, dim, alpha,\n    w, dim, x, 1, beta, out, 1);\n```\n\nTornadoVM can dynamically generate CUDA C and also directly bind CUDA libraries inside your code. Mix and match as you feel, TornadoVM will reuse memory, combine operations and give you the maximum performance.\n\nAll within your Java code.\n\n``` bash\n$ sdk install tornadovm\nbash\n$ tornado --devices\nbash\n$ java @$TORNADOVM_HOME/tornado-argfile -cp $TORNADOVM_HOME/share/java/tornado/tornado-examples-5.0.0-jdk21.jar uk.ac.manchester.tornado.examples.compute.MatrixVectorRowMajor\n```\n\n", "url": "https://wpnews.pro/news/what-if-java-apps-could-access-cuda-ecosystem-gracefully", "canonical_source": "https://www.tornadovm.org/", "published_at": "2026-07-13 16:53:36+00:00", "updated_at": "2026-07-13 17:05:40.868511+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "ai-tools"], "entities": ["TornadoVM", "NVIDIA", "cuBLAS", "cuFFT", "cuDNN", "CUDA", "OpenCL", "Metal"], "alternates": {"html": "https://wpnews.pro/news/what-if-java-apps-could-access-cuda-ecosystem-gracefully", "markdown": "https://wpnews.pro/news/what-if-java-apps-could-access-cuda-ecosystem-gracefully.md", "text": "https://wpnews.pro/news/what-if-java-apps-could-access-cuda-ecosystem-gracefully.txt", "jsonld": "https://wpnews.pro/news/what-if-java-apps-could-access-cuda-ecosystem-gracefully.jsonld"}}