The norm of a vector allows you to gauge the distance or the magnitude of a vector. It is pretty similar to how you can calculate the distance between two real scalar values a, b by using the formula abs(a-b). If you have a vector with norm or length one then that is simply a scalar.
Norms are regularly used in regularization methods and are used to evaluate the errors of a model. The latter is evaluated by calculating the actual output of a neural network versus what is expected as the actual outcome or value.
If given a vector space Rn, a vector norm is defined as a function: f:Rn→R. The norms are represented by the double-bar notation, and the function helps map the vector to a positive value.
The norm of a vector cannot be negative, and if you consider it as the length of the vector, it would be easier to understand why it cannot be negative.
Characteristics of Norms
Vector norms are any function that fulfil the following criteria:
Norms are non-negative values.
Norms are 0 only if the vector itself is a zero vector.
A scalar multiplied to the norm of a vector is equal to the absolute value of this scalar multiplied by the norm of the vector.
There are different functions used to calculate the norm of a vector, here we will be discussing the most commonly occurring vector norms of 1, 2 and p norms.
Vector Norms in Python
To understand the implementation of vector norms in Python, we will be creating a simple Python script and explaining it step by step. The first step is to create an array filled with random digits by using the NumPy’s randint() function.
Here, you have created an array of size 10 that will have any random values between -5 and 10. This random array can be understood as a vector and we can find out its norm by using the following code block:
If you run this code, it will create a random array, print it out, and then calculate its norm and print it. While running this code, it yielded the following output:
What we just calculated using Python is the 1-norm of a vector. To better understand it, consider the following equation, which provides the 1-norm of a vector x.
The 1-norm is also known as Taxicab norm and is usually denoted by L1. A more optimized way to calculate the Taxicab norm using Python is given in the following code block:
In this code, the only difference is that instead of using the slow for loop, we are using NumPy’s inbuilt optimized sum() function to iterate through the array and calculate its sum.
The 2-norm of a vector is also known as Euclidean distance or length and is usually denoted by L2. The 2-norm of a vector x is defined as:
The calculation of 2-norm is pretty similar to that of 1-norm but you raise the value by the power of two and take the square root at the end. The following block of code performs the 2-norm by using Python:
This line of code does the same job as the code block we previously explained, but faster and more neatly.
The third kind of norm that will be discussed here is the p-norm, which is defined as:
Here, we can set the value of p as 1 or 2 to find the 1-norm and 2-norm of the vector respectively. Similarly, we can set the value of p to any real number, however, the number cannot be less than zero or be complex.
The following code calculates 1-norm by using the p-norm equation given above.
Running this code will yield the following output:
As evident, both functions yield the same result, and this is how you can calculate the norm of a vector to any degree by using Python.
In this tutorial, we have just skimmed the surface of calculating norms of vectors by using Python. The topic is too vast to be covered in a single tutorial and to learn more about norms, you can read from Wikipedia and other educational sources.