Broadcasting in in numpy

By Martin McBride, 2021-02-27
Tags: arrays broadcasting vectorisation
Categories: numpy


When we combine two arrays using vectorisation or a ufunc, it will often be the case that both arrays are identical shape. For example, they might both be 2-dimensional array of shape 4 by 5.

However, you might sometimes want to combine arrays of different shapes. For example, you might want to take a vector (1-dimensional array) and add it to each row of a matrix (2-dimensional array). NumPy allows you to do this, and other things, using broadcasting.

We will look at some common cases, and then the general rule.

Broadcasting from 1 to 2 dimensions

We will look at the example suggested above, with the following two arrays:

m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9],
              [10, 11, 12]])
v = np.array([10, 20, 30])

r = m + v

m has shape (4, 3), v has shape (3,)

You might think that m + v would fail, but in fact NumPy automatically broadcasts v across 4 rows, to make an effective array that is the same shape as m. The broadcast v becomes:

[[10, 20, 30],
 [10, 20, 30],
 [10, 20, 30],
 [10, 20, 30]]

NumPy doesn't actually create this array, it simulates it by looping multiple times over the array v, as we will see later.

Where is the result of m + v:

r = [[11 22 33]
     [14 25 36]
     [17 28 39]
     [20 31 42]]

Broadcasting 1 to 3 dimensions

Now consider an array of shape (3, 2, 3):

m = np.array([[[1, 2, 3],
               [4, 5, 6]],

              [[7, 8, 9],
               [10, 11, 12]],

              [[13, 14, 15],
               [16, 17, 18]]])
v = np.array([10, 20, 30])

r = m + v

This time the value v is broadcast over every row of every sheet, giving an effective array of:

[[[10 20 30]
  [10 20 30]]

 [[10 20 30]
  [10 20 30]]

 [[10 20 30]
  [10 20 30]]]

The result is:

r = [[[11 22 33]
      [14 25 36]]

     [[17 28 39]
      [20 31 42]]

     [[23 34 45]
      [26 37 48]]]

Broadcasting 2 to 3 dimensions

Now consider an array of shape (2, 3) being broadcast to the previous array (3, 2, 3):

m = np.array([[[1, 2, 3],
               [4, 5, 6]],

              [[7, 8, 9],
               [10, 11, 12]],

              [[13, 14, 15],
               [16, 17, 18]]])
v = np.array([[10, 20, 30],
              [40, 50, 60]])

r = m + v

In this case, the 2 by 3 matrix is broadcast to each of the 3 sheets, like this:

[[[10 20 30]
  [40 50 60]]

 [[10 20 30]
  [40 50 60]]

 [[10 20 30]
  [40 50 60]]]

Giving the result:

r = [[[11 22 33]
      [44 55 66]]

     [[17 28 39]
      [50 61 72]]

     [[23 34 45]
      [56 67 78]]]

Broadcasting rules

We have seen several examples of broadcasting, now we will formalise the general rules. If two arrays have different shapes, NumPy will attempt to make them compatible by following two steps.

Step 1 if the arrays have different rank (ie number of dimensions), they are equalised by adding dimensions of length 1 to the start of the shape tuple.

Step 2 for each dimension, if the two values are equal then the shapes already match in that dimension so nothing needs to be done. If the two values are not equal but one of the values is 1, then the array with the dimension of 1 will be replicated along that axis to match the other array.

If, for any dimension, the two arrays do not match but neither has the value 1, then the arrays are not compatible and broadcasting will not occur.

Here are some examples.

An 2-dimensional array of shape (4, 3) and a 1-dimensional array of shape (3,), see the example above:

  • Ones are added to the start of the shape of the second array to make its shape (1, 3).
  • The array is replicated 4 times along axis 0 to make its shape (4, 3).

An 3-dimensional array of shape (3, 2, 3) and a 1-dimensional array of shape (3,), see the example above:

  • Ones are added to the start of the shape of the second array to make its shape (1, 1, 3).
  • The array is replicated 3 times along axis 0, and 2 times along axis to make its shape (3, 2, 3).

Finally a failure example, a 2-dimensional array of shape (2, 6) and a 1-dimensional array of shape (3,), see the example above:

  • Ones are added to the start of the shape of the second array to make its shape (1, 3).
  • The second axis doesn't match - it is 6 for the first array, but 3 for the second array. NumPy thows a ValueError.

Broadcasting a column vector

We can also broadcast a column vector across m, like this:

m = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9],
              [10, 11, 12]])
c = np.array([[10],
              [20],
              [30],
              [40]])

r = m + c

This time, c has a shape (4, 1) - it has 4 rows and 1 column. m has a shape (4, 3). The two arrays are already both 2-dimensional, but c must be broadcast along axis 1 to make them match. This gives:

[[10 10 10]
 [20 20 20]
 [30 30 30]
 [40 40 40]]

This is compatible with m, so they can be added together.

Broadcasting a row vector and a column vector

We can also broadcast a row vector and a column vector across m, like this:

v = np.array([1, 2, 3])
c = np.array([[10],
              [20],
              [30]])
r = v + c

This is quite an interesting case:

  • c has a shape (3, 1).
  • v has a shape (3,), which will be extended to (1, 3) to match the dimensions of c.
  • To match these two shapes, c is broadcast along axis 1, and v is broadcast along axis 0.
  • The result is a 3 by 3 matrix!

Here is the result:

r = [[11 12 13]
     [21 22 23]
     [31 32 33]]

Broadcasting scalars

We have already seen code like this:

a = np.array([1, 2, 3, 4])
b = a + 2

But how does this actually work?

One way to think about it is the scaler value 2 is a bit like a 1-dimensional array of length 1. This array can be broadcast to any shape, simply by replicating it in every required dimension.

See also

If you found this article useful, you might be interested in the book NumPy Recipes or other books by the same author.

Join the PythonInformer Newsletter

Sign up using this form to receive an email when new content is added:

Popular tags

2d arrays abstract data type alignment and angle animation arc array arrays bar chart bar style behavioural pattern bezier curve built-in function callable object chain circle classes clipping close closure cmyk colour combinations comparison operator comprehension context context manager conversion count creational pattern data science data types decorator design pattern device space dictionary drawing duck typing efficiency ellipse else encryption enumerate fill filter font font style for loop formula function function composition function plot functools game development generativepy tutorial generator geometry gif global variable gradient greyscale higher order function hsl html image image processing imagesurface immutable object in operator index inner function input installing iter iterable iterator itertools join l system lambda function latex len lerp line line plot line style linear gradient linspace list list comprehension logical operator lru_cache magic method mandelbrot mandelbrot set map marker style matplotlib monad mutability named parameter numeric python numpy object open operator optimisation optional parameter or pandas partial application path pattern permutations pie chart pil pillow polygon pong positional parameter print product programming paradigms programming techniques pure function python standard library radial gradient range recipes rectangle recursion reduce regular polygon repeat rgb rotation roundrect scaling scatter plot scipy sector segment sequence setup shape singleton slice slicing sound spirograph sprite square str stream string stroke structural pattern subpath symmetric encryption template tex text text metrics tinkerbell fractal transform translation transparency triangle truthy value tuple turtle unpacking user space vectorisation webserver website while loop zip zip_longest