Numerical Linear Algebra
At the base of Numerical Linear Algebra (NLA) there are basic operations such as
- Operations with vectors and scalars
- Operations with matrices and vectors
- Operations among matrices
These operations are typically implemented in highly optimized kernels collected in libraries specialized for specific architectures. Because they represent the building block of almost any numerical linear algebra computations they have been encoded in a stable interface called BLAS (Basic Linear Algebra Subroutines) adopted by any specialized library such as MKL (Intel), cuBLAS (NVIDIA), rocBLAS (AMD). There exist also more general libraries that can adapt to several different platforms such as OpenBLAS, BLIS, ATLAS, etc. The operations specified by each bullet points are additionally divided in three levels of the BLAS interface:
- BLAS level 1 (BLAS-1) is dedicated to operation with vectors
- BLAS level 2 (BLAS-2) is dedicated to operations between matrices and vectors
- BLAS level 3 (BLAS-3) is dedicated to operation with matrices
Most of NLA operations are built on top of BLAS kernels and can be divided roughly into three main categories. Any additional linear algebra problem can, in theory, be reduced to the following:
- Factorizations (e.g. Choleski decomposition for positive definite matrices)
- Linear system solves
- Eigenvalue solvers