The Julia CUDA stack only requires users to have a functional NVIDIA driver. It is not necessary to install the CUDA toolkit. On Windows, also make sure you have the Visual C++ redistributable installed.
For most users, installing the latest tagged version of CUDA.jl will be sufficient. You can easily do that using the package manager:
pkg> add CUDA
Or, equivalently, via the
julia> import Pkg; Pkg.add("CUDA")
In some cases, you might need to use the
master version of this package, e.g., because it includes a specific fix you need. Often, however, the development version of this package itself relies on unreleased versions of other packages. This information is recorded in the manifest at the root of the repository, which you can use by starting Julia from the CUDA.jl directory with the
$ cd .julia/dev/CUDA.jl # or wherever you have CUDA.jl checked out $ julia --project pkg> instantiate # to install correct dependencies julia> using CUDA
In the case you want to use the development version of CUDA.jl with other packages, you cannot use the manifest and you need to manually install those dependencies from the master branch. Again, the exact requirements are recorded in CUDA.jl's manifest, but often the following instructions will work:
pkg> add GPUCompiler#master pkg> add GPUArrays#master pkg> add LLVM#master
We support the same operation systems that NVIDIA supports: Linux, and Windows. Similarly, we support x86, ARM, PPC, ... as long as Julia is supported on it and there exists an NVIDIA driver and CUDA toolkit for your platform. The main development platform (and the only CI system) however is x86_64 on Linux, so if you are using a more exotic combination there might be bugs.
To use the Julia GPU stack, you need to install the NVIDIA driver for your system and GPU. You can find detailed instructions on the NVIDIA home page.
If you're using Linux you should always consider installing the driver through the package manager of your distribution. In the case that driver is out of date or does not support your GPU, and you need to download a driver from the NVIDIA home page, similarly prefer a distribution-specific package (e.g., deb, rpm) instead of the generic runfile option.
If you are using a shared system, ask your system administrator on how to install or load the NVIDIA driver. Generally, you should be able to find and use the CUDA driver library, called
libcuda.so on Linux,
libcuda.dylib on macOS and
nvcuda64.dll on Windows. You should also be able to execute the
nvidia-smi command, which lists all available GPUs you have access to.
On some enterprise systems, CUDA.jl will be able to upgrade the driver for the duration of the session (using CUDA's Forward Compatibility mechanism). This will be mentioned in the
CUDA.versioninfo() output, so be sure to verify that before asking your system administrator to upgrade:
julia> CUDA.versioninfo() CUDA runtime 10.2 CUDA driver 11.8 NVIDIA driver 520.56.6, originally for CUDA 11.7
Finally, to be able to use all of the Julia GPU stack you need to have permission to profile GPU code. On Linux, that means loading the
nvidia kernel module with the
NVreg_RestrictProfilingToAdminUsers=0 option configured (e.g., in
/etc/modprobe.d). Refer to the following document for more information.
The recommended way to use CUDA.jl is to let it automatically download an appropriate CUDA toolkit. CUDA.jl will check your driver's capabilities, which versions of CUDA are available for your platform, and automatically download an appropriate artifact containing all the libraries that CUDA.jl supports.
If you really need to use a different CUDA toolkit, it's possible (but not recommended) to load a different version of the CUDA runtime, or even an installation from your local system. Both are configured by setting the
version preference (using Preferences.jl) on the CUDARuntimejll.jl package, but there is also a user-friendly API available in CUDA.jl.
You can choose which version to (try to) download and use by calling
julia> using CUDA julia> CUDA.set_runtime_version!(v"11.8") [ Info: Set CUDA.jl toolkit preference to use CUDA 11.8.0 from artifact sources, please re-start Julia for this to take effect.
This generates the following
LocalPreferences.toml file in your active environment:
[CUDA_Runtime_jll] version = "11.8"
This preference is compatible with other CUDA JLLs, e.g., if you load
CUDNN_jll it will only select artifacts that are compatible with the configured CUDA runtime.
To use a local installation, you set the
local_toolkit keyword argument to
julia> using CUDA julia> CUDA.versioninfo() CUDA runtime 11.8, artifact installation ... julia> CUDA.set_runtime_version!(local_toolkit=true) [ Info: Set CUDA.jl toolkit preference to use CUDA from the local system, please re-start Julia for this to take effect.
After re-launching Julia:
julia> using CUDA julia> CUDA.versioninfo() CUDA runtime 11.8, local installation ...
Calling the above helper function generates the following
LocalPreferences.toml file in your active environment:
[CUDA_Runtime_jll] local = "true"
This preference not only configures CUDA.jl to use a local toolkit, it also prevents downloading any artifact, so it may be interesting to set this preference before ever importing CUDA.jl (e.g., by putting this preference file in a system-wide depot).
If CUDA.jl doesn't properly detect your local toolkit, it may be that certain libraries or binaries aren't on a globally-discoverable path. For more information, run Julia with the
JULIA_DEBUG environment variable set to
Note that using a local toolkit instead of artifacts any CUDA-related JLL, not just of
CUDA_Runtime_jll. Any package that depends on such a JLL needs to inspect
CUDA.local_toolkit, and if set use
CUDA_Runtime_Discovery to detect libraries and binaries instead.
CUDA.jl can be precompiled and imported on systems without a GPU or CUDA installation. This simplifies the situation where an application optionally uses CUDA. However, when CUDA.jl is precompiled in such an environment, it cannot be used to run GPU code. This is a result of artifacts being selected at precompile time.
In some cases, e.g. with containers or HPC log-in nodes, you may want to precompile CUDA.jl on a system without CUDA, yet still want to have it download the necessary artifacts and/or produce a precompilation image that can be used on a system with CUDA. This can be achieved by informing CUDA.jl which CUDA toolkit to run time by calling
When using artifacts, that's as simple as e.g. calling
CUDA.set_runtime_version!(v"11.8"), and afterwards re-starting Julia and re-importing CUDA.jl in order to trigger precompilation again and download the necessary artifacts. If you want to use a local CUDA installation, you also need to set the
local_toolkit keyword argument, e.g., by calling
CUDA.set_runtime_version!(v"11.8"; local_toolkit=true). Note that the version specified here needs to match what will be available at run time. In both cases, i.e. when using artifacts or a local toolkit, the chosen version needs to be compatible with the available driver.
Finally, in such a scenario you may also want to call
CUDA.precompile_runtime() to ensure that the GPUCompiler runtime library is precompiled as well. This and all of the above is demonstrated in the Dockerfile that's part of the CUDA.jl repository.