Going through a few simple* functions

*more or less simple

Part of the 'Zero-to-GANs Jovian course'

An short introduction about PyTorch and about the chosen functions.

torch.cholesky(input,upper=False,out=None)
torch.ceil(input, out=None)
torch.cos(input, out=None)
torch.reshape(from=0, to=None,,generator=None*)
torch.diagonal(input, offset=0, dim1=0, dim2=1)

# Import torch and other required modules
import torch

Function 1 - torch.cholesky

The cholesky function takes a hermitian positive definite Matrix and calculates a triangular Matrix from it. In the options, one is able to specify which triangular Matrix the function returns, either the upper or the lower one.

There are some notations that could use further explanation. Let's start in order.

Hermitian is a generalized case of Symmetry, so to speak. For example a matrix with real number is hermitian and symmetric if $M=M^\text{T}$ .

However if complex numbers are added to the mix, symmetry with respect to real numbers is not enough anymore. The definition of a hermitian matrix is that off diagonal mirror elements need to be conjugated complex numbers. Long story short, a real symmetric matrix is a special case of the hermitian matrix, where no complex numbers are involved.

Example for hermitian matrix:

$M = \begin{pmatrix} 1 & 2+3i\\ 2-3i & 5+i \end{pmatrix}$

Example for symmetric matrix:

$M = \begin{pmatrix} 1 & 2\\ 2 & 5 \end{pmatrix}$

Positive definite has a very simple definition. Given a vector $x$ , a Matrix $M$ is called positive definite if $xMx^\text{T} > 0$

Last but not least we need to define the term triangular. A triangular Matrix only has elements unequal to zero below the diagonal.

For example:

$A_{m,n} = \begin{pmatrix} a_{1,1} & & & 0 \\ a_{2,1} & a_{2,2} & & \\ \vdots & \vdots & \ddots & \\ a_{m,1} & a_{m,2} & \cdots & a_{m,n} \end{pmatrix}$

In this case $A$ is a lower triangular Matrix.

# Define a symmetric, positive definite Matrix.

a=torch.randn(3,3)
a=torch.mm(a,a.t())
print("a: ",a)

l = torch.cholesky(a)
print("l= ",l)

a:  tensor([[ 4.8267,  0.1128,  4.4727],
        [ 0.1128,  1.8052, -2.2771],
        [ 4.4727, -2.2771,  8.2146]])
l=  tensor([[ 2.1970,  0.0000,  0.0000],
        [ 0.0513,  1.3426,  0.0000],
        [ 2.0358, -1.7739,  0.9609]])
u=  tensor([[ 2.1970,  0.0513,  2.0358],
        [ 0.0000,  1.3426, -1.7739],
        [ 0.0000,  0.0000,  0.9609]])

The torch.cholesky function is used to calulate the lower matrix since the default value for upper=False. Furthermore two other pytorch functions are used in this example, the mm is a matrix multiplication alogorithm and randn, fills the tensor with random values.