Basic Linear Algebra with Python

7 min readNov 6, 2020

Linear algebra it’s a very important field in mathematics and also in data science which it’s getting more popular nowadays so know the fundamental of linear algebra for to apply in data science is crucial to this field. Having that in mind this post will not cover the basics of Python like functions and build in functions. If you want to learn more about Python I recommend you to read the following post:

https://medium.com/fintechexplained/everything-about-python-from-beginner-to-advance-level-227d52ef32d2

I assume you have read or have a basic knowledge of Python we are good to go.

Vectors:

Vectors are objects that abstractly can be defined as an object that can be summed and multiplied by a scalar like numbers and also can be used to form new vectors. In data science vectors are like points in some space (vector space) with finite dimension. To give an example to you, suppose you have an array which contains two information, salary and house value, you can store this information as a bi dimensional vector (salary, house_value). In python the most basic approach is to think vectors are like lists, so for the same example you can define a vector in the following way:

salary_housevalue = [4000,  # Salary
                     21312] # House value

Other example you can find in some student grades,

grades = [9,   # Test 1 
          7.8] # Test 2

Using matplotlib library you can plot this as a line or a vector, for the grades example, the graphic representations of the vector is

As it was said before, you can define a summation of two or more vectors, for that we will define a function called vector_summation in which this case will receive two parameters (two vectors).

def vector_summation(v, w):
    return [v_i + w_i for v_i, w_i in zip(v,w)]

In linear algebra, sum two or more vectors it is basically summed each component of each vector (i.e suppose you have this vectors [1,1] and [1,2], the result of this summation is [1+1,1+2] or [2, 3]). A graphical representation of the summation of two vectors are

Of course, it is possible to define a vector subtraction, one example of this function is

def vector_sub(v, w):
    return [v_i - w_i for v_i, w_i in zip(v,w)]

A graphical representation of the summation of two vectors of the previous example

There are more than one way to implement a sum of multiple vectors of the same dimension, you can use reduce function or even a build a function which receives as argument one list with all the vectors, and also other way to do that is with arbitrary arguments (*args), one example of each one is shown below.

# Function receive a list of vectors 
def vector_addition(vec):
   vec_f = vec[0] # vec_f is the final vector
   for vector in vec[1:]:
       result = vector_summation(vec_f, vector) 
   return vec_f# With reduce function 
def vector_addition2(vec):
    return reduce(vector_addition, vec)# With *args (very similar to vector_addition function)
def vector_addition3(*args):
    vec_f = args[0]
    for vec in args[1:]:
        vec_f = vector_summation(vec_f, vec)
    return vec_f

Now it’s interesting to multiply our vector by a scalar number, very simple to implement that, one example of code implementation and graphical meaning is

def mutiply(n, v):
    return [n*v_i for v_i in v] # using list comprehension

Mathematically what we are doing is increasing the total length of the vector which is know as magnitude, defined as square root of the sum of the square value of each component of the vector, suppose we have the vector a = [x, y], the magnitude of this vector is |a| = (x²+y²)^0.5. Before we implement a python function of this, let me show a graphical representation of how we derive this formula using Pythagorean theorem. Suppose we have the vector a = [1,1]

The x and y components can be described as

With Pythagorean theorem you easily derive the formula for the vector magnitude. An example of function for that is

def magnitude(v):
    return (v[0]**2 + v[1]**2)**0.5

We will implement a better function for this before talk a little more of dot product. The dot product basically is the summation of the product of each component, so imagine we have the vector a=[x, y] and vector b=[u, v], the dot product is a·b = x×u+y×v. A function for that

def dot(v,w):
    return sum(v_i*w_i for v_i, w_i in zip(v,w))

Mathematically the dot product represents the projection of one vector in the other, mathematically representation of that is

a·b = |a||b|cos(O)

Where O is the angle between a and b.

Now we can create also a function to sum the squares of the vector simply calculating the dot product of the vector by itself.

def sum_squares(v):
    return dot(v,v)

And now we can create a better function for the magnitude

def magnitude(v):
    return (sum_squares(v))**0.5

One last thing to comment about vectors are the distance between two or more vectors. Suppose we have the vectors a = [x,y] and b=[v,w], the distance of this two is d=((x-v)²+(y-w)²)^0.5. In python

def distance(v,w):
    squares_difference = sum_squares(vector_sub(v,w))
    return squares_difference**0.5

Matrix

We can define a matrix as a collections of n-dimensional numbers but here we will be working with bi dimensional matrix. In python, we can create a matrix as list of lists, or a list of vectors in one variable which each vector represents a line and each component of the vector represents the column, for a bidimensional case. By convection, we denote a matrix with upper case letter like A. Therefore, one example of matrix in python is

A = [[1,1,1],   # A has two lines and three columns
     [2,2,2]]B = [[1,1,1],   # B has three lines and three columns
     [2,10,2],
     [3,3,3]]

Now you can guess that B is a made by three vectors representing some information in a vector space and it is correct!. Suppose we have the following matrix for students grades

GR = [[7,8],  
      [2,6],
      [5,6.5]]

Each line represents a different student and each column represents the respective grade of that student. Graphically

There are a two way to access the values of a matrix in Python, A[i, j] or A[i][j] where A[i][j] is the i-th line and j-th column element. Although A[i, j] is more common in math here in python is more common to use A[i][j] notation.

If we want to access de element of second line and column of A or B

print(A[1][1]) # Output: 2  | Remember, i=1 and j=1
print(B[1][1]) # Output: 10

There are some interesting functions to build when we are dealing with matrix, like:

Shape of the matrix, this is how many lines and columns
Get row or columns
Make matrix

Starting with the shape function, one example is

def shape(M):
    rows = len(M) # Lenght of the matrix is equal to de nº of lines
    cols = len(M[0])
    return rows, colsprint(shape(GR)) # Output: (3, 2)

To get the column or row

def get_row(M, i):
    return M[i]def get_col(M, j):
    return [M_i[j] for M_i in M]

For make a matrix function we need to define what kind of matrix we want, in order to do that. In this case we want to create the zero matrix so the main inputs are the number of columns and rows so

def create_matrix(i, j):
    return [[0 for col in range(j)] for lines in range(i)]

Other way is to replace 0 by an input number asked the user or create a function to return a given number depending on the i-th and j-th position. Of course here we create the vectors and matrix only using python and if the number of columns or rows become too large the functions here will be slow. One option is to use Numpy library which implements a faster algorithm and the implementation is almost the same. I encourage you to search about it.

This post was based on the 4th cap of Data Science from zero-Joel Grus which is an amazing book to introduce you in data science I strongly recommend you. Thank you for read this post.

Andres.

Basic Linear Algebra with Python

Vectors:

Matrix

Written by Andres Benoit