Show HN: Rgpu – a PyTorch device whose tensors live on a remote GPU

A developer released rGPU, an open-source project that lets PyTorch programs run GPU work on a remote NVIDIA machine via a custom `rgpu` device or a CUDA shim covering libcuda, CUDA Runtime, cuBLAS, cuBLASLt and cuDNN. The project installs with `pip install rgpu` and is launched through `rgpu-run --host user@gpu-host --ssh-port 2222`, with a smoke test of `torch.ones(4, device="rgpu")` expected to print 8.0. rGPU's documentation warns that neither protocol authenticates or encrypts connections, so the CUDA server's port 9713 must be restricted by firewall rules, and it is licensed under Apache License 2.0.

rGPU runs GPU work on a remote NVIDIA machine while the application stays on the client. It currently offers two paths: | Path | Use it for | Interface | |---|---|---| | PyTorch device | PyTorch programs that can opt into an rgpu device | torch operations over TCP | | CUDA shim | Existing Linux CUDA programs, including stock CUDA PyTorch | libcuda , CUDA Runtime, cuBLAS, cuBLASLt, and cuDNN shims | The PyTorch device is the simpler integration. The CUDA shim covers existing binaries but has a larger compatibility surface. The Fumadocs site in website/ https://github.com/ymcrcat/rgpu/blob/main/website is the product documentation: Engineering records and experiments are indexed in docs/README.md https://github.com/ymcrcat/rgpu/blob/main/docs/README.md . Install rGPU with pip install rgpu , or pip install -e ./python from this checkout, then follow the quickstart https://github.com/ymcrcat/rgpu/blob/main/website/content/docs/quickstart.mdx to deploy the server. Save this as smoke.py in your workload directory: python import torch import rgpu x = torch.ones 4, device="rgpu" print x 2 .sum .item 8.0 Run it in the environment where rGPU is installed, using your server's SSH destination and options: rgpu-run --host user@gpu-host --ssh-port 2222 -i ~/.ssh/gpu key \ python smoke.py The program selects the device; rgpu-run opens the tunnel and configures the connection. The expected output is 8.0 . For existing Linux CUDA programs, follow the CUDA shim guide https://github.com/ymcrcat/rgpu/blob/main/website/content/docs/cuda-shim.mdx , starting with ./scripts/build client.sh . Neither protocol authenticates or encrypts connections. Keep rgpu-opserver on its default localhost bind and use SSH. The CUDA server listens on all IPv4 interfaces: restrict port 9713 with host/cloud firewall rules before starting it, even when using an SSH tunnel. See deployment https://github.com/ymcrcat/rgpu/blob/main/website/content/docs/operations.mdx . C++ client and fake-driver tests ./scripts/build client.sh Python tests python -m pip install -e './python test ' python -m pytest python/tests Static documentation npm --prefix website ci npm --prefix website run build See scripts/README.md https://github.com/ymcrcat/rgpu/blob/main/scripts/README.md for the remaining build, cloud, and hardware commands. Generated C++ is committed; its policy and regeneration steps are in codegen/README.md https://github.com/ymcrcat/rgpu/blob/main/codegen/README.md . | Path | Purpose | |---|---| | client/ | CUDA client shims and transport | | server/ | CUDA server and dispatch | | common/ | Shared protocol and generated API metadata | | python/ | PyTorch device and launcher | | tests/ | C++, Python, CUDA, and hardware checks | | codegen/ | CUDA header parser and source generators | | website/ | Fumadocs product documentation | | docs/ | Design records, measurements, and experiment reports | | jax/ | Experimental JAX work; not a supported product path | | scripts/ | Build, deployment, cloud, and test helpers | | skills/ | Installable agent guidance for using rGPU | Historical implementation notes and experimental results are indexed in docs/README.md https://github.com/ymcrcat/rgpu/blob/main/docs/README.md . Licensed under the Apache License 2.0 https://github.com/ymcrcat/rgpu/blob/main/LICENSE .