cuBLAS

cuBLAS.enable_logging(enable::Bool=true)

Forward log messages from cuBLAS and cuBLASLt to Julia's logging system. API traces are reported at Debug level, performance hints at Info level, and problems at Warn or Error level. Starting Julia with JULIA_DEBUG=cuBLAS enables this automatically, and also shows the Debug-level messages.

cuBLASLt messages are only forwarded on CUDA 11 or later; older libraries do not export the cuBLASLt logging API.

gemm!(transA, transB, alpha, A, B, beta, C; compute_type=nothing, algo=CUBLAS_GEMM_DEFAULT)

Compute C = alpha * op(A) * op(B) + beta * C, where op is selected by transA and transB ('N', 'T' or 'C'). The element types of A, B and C can differ, as long as cuBLAS supports the combination.

The compute type is selected based on these element types, e.g., computing products of Float16 matrices in Float16. Pass compute_type=Float32 to compute them in Float32 instead; Float16, Float64 and Int32 are supported as well. The math mode (see CUDA.math_mode!) then determines which variant of the compute type is used, e.g., computing Float32 products with TF32 tensor cores in fast math mode.

alpha and beta are numbers, or device-side scalars (e.g. a CuRef or a one-element CuArray) of the type cuBLAS expects for the selected compute type.

gemm_batched!(transA, transB, alpha, A, B, beta, C; compute_type=nothing,
              algo=CUBLAS_GEMM_DEFAULT)

Compute C[i] = alpha * op(A[i]) * op(B[i]) + beta * C[i] for every matrix in the vectors A, B and C. All products need to have the same size; see gemm_grouped_batched! for products of different sizes. Element types, scalars and the compute type are handled like for gemm!.

gemm_grouped_batched!(transA, transB, alpha, A, B, beta, C; compute_type=nothing)

Compute C[i][j] = alpha[i] * op(A[i][j]) * op(B[i][j]) + beta[i] * C[i][j] for every group i of matrices in the vectors of vectors A, B and C. The products in a group need to have the same size, but those in different groups don't. transA, transB, alpha and beta hold a value per group, or a single value for all of them.

With vectors of matrices for A, B and C, every product is a group of its own.

The element types of A, B and C need to be the same: Float16, BFloat16, Float32 or Float64. Products of 16-bit matrices are computed in Float32, and so are alpha and beta, which reside on the host. compute_type can only select that precision: Float32, or Float64 for Float64 matrices. Float16 and BFloat16 need cuBLAS 12.5, the other types cuBLAS 12.4.

gemm_strided_batched!(transA, transB, alpha, A, B, beta, C; compute_type=nothing,
                      algo=CUBLAS_GEMM_DEFAULT)

Compute C[:, :, i] = alpha * op(A[:, :, i]) * op(B[:, :, i]) + beta * C[:, :, i] for every slice of the 3-dimensional arrays A, B and C. A or B can have a single slice, which is then used for every product. Element types, scalars and the compute type are handled like for gemm!.