In place operator overloading

By Martin McBride, 2019-01-09
Tags: in-place operator
Categories: magic methods


Following on from the previous article on operator overloading, we will now take a quick look in place operators.

In place operators

In place operators take the form +=:

x = 3
x += 2
print(x)     # 5

There are equivalent operators for all the numerical operators: -=, *=, etc

In place operators for mutable objects

When we are working with immutable objects such as numbers, strings or tuples, we can treat the following statements as being equivalent:

x = x + 2
x += 2

But when we are using mutable objects, there is a difference. Here is the first case:

a = [10, 20, 30]
b = a
a = a + [40]
print(a, b)       # [10, 20, 30, 40] [10, 20, 30]

We create a list [10, 20, 30] and assign it to a. We then assign the a to b. So a and b reference the same list object.

In the third line, the expression a + [40] creates a new list object with the value [10, 20, 30, 40]. This is assigned to a.

In this case, a references the new list, but b still references the old, unchanged list, so a and b have different values.

Now look at this:

a = [10, 20, 30]
b = a
a += [40]
print(a, b)       # [10, 20, 30, 40] [10, 20, 30, 40]

This start out the same, but the line a += [40] does something different. It appends [40] to the end of the existing list in a.

So now, a still references the original list, whose value has changed in place (ie without creating a new list). b of course also still references the original list, so both variables now contain the same value - the updated list.

In place operators and efficiency

Now suppose a contained a huge list and we wanted to append a value to it.

In the first case above, we would create a brand new copy of the list, with the extra value added on.

In the second case we would simply append the value to the existing list. In place operators can be much more efficient.

In this example, of course, we could also use the list append method to do the same thing, but in other cases where you are susing large data objects, consider using in place operators to avoid memory copying.

In place operations with the Matrix object

Now consider our Matrix class with the __add__ method to implement +, and the __str__ method so print works correctly:

class Matrix:

    def __init__(self, a, b, c, d):
        self.data = [a, b, c, d]

    def __str__(self):
        return '[{}, {}][{}, {}]'.format(self.data[0],
                                         self.data[1],
                                         self.data[2],
                                         self.data[3])

    def __add__(self, other):
        if isinstance(other, Matrix):
            return Matrix(self.data[0] + other.data[0],
                          self.data[1] + other.data[1],
                          self.data[2] + other.data[2],
                          self.data[3] + other.data[3])
        else:
            return NotImplemented

What happens if we try to use +=?

a = Matrix(10, 20, 30, 40)
b = a
a += Matrix(1, 2, 3, 4)
print(a, b)                 # [11, 22][33, 44] [10, 20][30, 40]

Well, the good thing is we don't get an error! In fact += works as if by magic.

Unfortunately, it isn't quite correct. This is isn't surprising really, because we haven't actually told Python how we expect += to work for a Matrix. So Python has used the existing + operator and used it to simulate the += operator by effectively doing this:

a = a + Matrix(1, 2, 3, 4)

As we saw with the list example, this isn't ideally what we want.

Implementing +=

To properly implement += we need to provide another in place operator to Matrix, called __iadd__ (short for in place add):

    def __iadd__(self, other):
        if isinstance(other, Matrix):
            self.data[0] += other.data[0]
            self.data[1] += other.data[1]
            self.data[2] += other.data[2]
            self.data[3] += other.data[3]
            return self
        else:
            return NotImplemented

This is similar to __add__, but instead of creating a new Matrix, it updates the values of the object itself. Notice that it also returns self (referring to the object itself) at the end.

When we do this, our += works as expected.

There are in place variants of all the numerical operators (__isub__, __imul__ etc).

See also

If you found this article useful, you might be interested in the book NumPy Recipes or other books by the same author.

Join the PythonInformer Newsletter

Sign up using this form to receive an email when new content is added:

Popular tags

2d arrays abstract data type alignment and angle animation arc array arrays bar chart bar style behavioural pattern bezier curve built-in function callable object chain circle classes clipping close closure cmyk colour combinations comparison operator comprehension context context manager conversion count creational pattern data science data types decorator design pattern device space dictionary drawing duck typing efficiency ellipse else encryption enumerate fill filter font font style for loop formula function function composition function plot functools game development generativepy tutorial generator geometry gif global variable gradient greyscale higher order function hsl html image image processing imagesurface immutable object in operator index inner function input installing iter iterable iterator itertools join l system lambda function latex len lerp line line plot line style linear gradient linspace list list comprehension logical operator lru_cache magic method mandelbrot mandelbrot set map marker style matplotlib monad mutability named parameter numeric python numpy object open operator optimisation optional parameter or pandas partial application path pattern permutations pie chart pil pillow polygon pong positional parameter print product programming paradigms programming techniques pure function python standard library radial gradient range recipes rectangle recursion reduce regular polygon repeat rgb rotation roundrect scaling scatter plot scipy sector segment sequence setup shape singleton slice slicing sound spirograph sprite square str stream string stroke structural pattern subpath symmetric encryption template tex text text metrics tinkerbell fractal transform translation transparency triangle truthy value tuple turtle unpacking user space vectorisation webserver website while loop zip zip_longest