Lecture 23 — Advanced Python Topics and Functional Programming¶

Problems We’d Like to Solve¶

Some of these are toy problems, but they illustrate use of tools we’d like to develop and use:

How many values are in a list of lists?
What is the maximum distance from the origin of the points in a list?
What is the sum of squares of the first n integers?
Can you sum the positive values in a list?
Can you sort a list of points by y value (2nd coordinate) and then by x value?

Solution Techniques¶

We can solve most of these with a for loop, but they can be solved even more effectively / efficiently / compactly using advanced Python methods.
Leads to notions of:

map and filter

functions as parameters

lambda functions

stable sort

list comprehensions

Most are examples of functional programmming

Map: Apply a function to each element of a list¶

Suppose we want to count the number of values in a list of lists. We can use map to apply the len function to each sublist.

>>> v = [ [2, 3, 5, 7], [11,13,17,19], [23, 29], [31,37] ]
>>> print( list(map( len, v)) )
[4, 4, 2, 2]

map is an iterator class:
- It produces values in a sequence, one after another, by applying the function (1st argument) to the values of the second argument.
- Technically, an iterator class is one that has the __next__ method implemented (correctly).
- Using list gives us the list of lengths of the sublists explicitly.
To complete the solution we need to just apply sum:
```
>>> print sum(map(len,v))
12
```
Notice that this does not explicitly form an intermediate list.

Passing Functions as Parameters¶

The above example passes the len function as an argument!
- We also passed functions as arguments to our callbacks in our GUI programs
This illustrates the concept that Python treats function as “first-class” objects - in other words functions can be used just like variables and other data.
- What’s passed as an argument to map() is the location of the function code.

Now suppose we want to find the maximum distance of a list of points from the origin. Here we’ll have to write a function

def dist2D( p ):
    return (p[0]**2 + p[1]**2)**0.5

pts = [ (4.5, 3), (2.1,-1), (6.8,-3), (1.4, 2.9) ]
print(max( map(dist2D,pts) ))

Lambda functions: Anonymous functions¶

We can avoid the need to write a separate function here by writing an anonymous function called a lambda function.

Our first example is just squaring the values of a list

>>> list(map( lambda x: x**2, [ 1, 2, 3, 4 ] ))
[ 1, 4, 9, 16 ]

Now, we can sum the squares from 1 to n

>>> n = 100
>>> sum( map( lambda x: x**2, range(1,n+1)))

We can also implement the dist2D function anonymously:

>>> max( map( lambda p: (p[0]**2 + p[1]**2)**0.5, pts) )
7.432361670424818

Notice that we did not need to explicitly form a list in each of the preceeding examples. This leads to substantial savings when the list is large!
Aside: the notion of a lambda function goes all the way back to the origin of computer science

In-Class Practice Problem:¶

Starting with the following list of x,y point coordinate types, we will use map(), a lambda function, and max() to find the maximum x coordinate (the 0-th coordinate) in a list of points.
```
pts = [ (6,-1), (8,4), (7.5,-3), (4.4,12), (7,2) ]
```

Lecture Exercises, Problems 1 and 2:¶

At this point students will be given the chance to work on the first two lecture exercises.

Filter: Extract / eliminate values from a list¶

Consider a different problem: how to eliminate all of the negative values from a list. Based on what we know so far, this requires a for loop with append.

We can simplify this using the built-in Python construct called filter

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> list(filter( lambda x: x>0, v))
[1, 9, 10]

Here,
- The lambda function must produce a boolean value and if that value is True the list item is kept; otherwise it is eliminated.
- The result of filter is an iterator object, just like the result of map is. We convert to a list in order to display the answer.
If we want to sum up the non-negative values, then we don’t need to explicitly generate a list:
```
>>> sum(filter( lambda x: x>0, v))
20
```

Passing Functions to Sort¶

Consider the problem of sorting a list of (x,y) points by their y values first and their x values for tied y values, both in decreasing order. For example, given

pts = [ (2,5), (12,3), (12,1), (6,5), (14, 10), (12, 10), \
          (8,12), (5,3) ]

we’d like the sorted order to be

[(8, 12), (14, 10), (12, 10), (6, 5), (2, 5), (12, 3), \
     (5, 3), (12, 1)]

The Python sort function

>>>  sorted( pts, reverse=True )
[(14, 10), (12, 10), (12, 3), (12, 1), (8, 12), (6, 5), \
    (5, 3), (2, 5)]

gives the ordering by x value and then by y value. This is not what we want.

The first step to a solution is to provide a key function to sorted() to pull out the information (the y value in this case) from each tuple to use as the basis for sorting:
```
>>> sorted( pts, key = lambda p: p[1], reverse=True)
[(8, 12), (14, 10), (12, 10), (2, 5), (6, 5), (12, 3), \
    (5, 3), (12, 1)]
```
This is close but not quite right because the two points with y=5 are out of order.

The trick is to sort by x first and then sort by y!

>>> by_x = sorted(pts,reverse=True)
>>> by_x
[(14, 10), (12, 10), (12, 3), (12, 1), (8, 12), (6, 5), \
   (5, 3), (2, 5)]
>>> sorted( by_x, key = lambda p: p[1], reverse=True)
[(8, 12), (14, 10), (12, 10), (6, 5), (2, 5), (12, 3), \
   (5, 3), (12, 1)]

This works because sorted() uses what’s known as a stable sort: when two values are “tied” according the sorting criteria (y value in the second sort) their relative ordering (by x value from the first sort) in the final list is preserved.
- Therefore, (6,5) comes earlier than (2,5), while (12,3) comes earlier than (5,3)
A number of variations on sorting use this “stable sort” property, but not all fast sorting algorithms are stable.

Practice Problem¶

Use filter to eliminate all words that are shorter than 4 letters from a list of words

List Comprehensions¶

Instead of map and filter some people prefer another example of functional programming in Python called list comprehensions

Here is an example to generate a list of the squares of the first n integers:

n = 8
>>> [ i*i for i in range(1,n+1) ]
[1, 4, 9, 16, 25, 36, 49, 64]

The form of this is an expression followed by a for loop statement.

We can get the effect of filter by adding a conditional at the end:

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x for x in v if x>0 ]
[1, 9, 10]

Here, the values are only generated in the resultant list when the if condition passes.
We can combine these as well. As a slightly silly example, we can eliminate the negative values and square the positive values
```
>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x*x for x in v if x>0 ]
[1, 81, 100]
```

We can get even more sophisticated by nesting for loops. Here is an example where we generate all pairs of numbers between 1 and 4, except for the pairs where the numbers are equal

>>> [ (i,j) for i in range(1,5) for j in range(1,5) if i != j ]
[(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2),
    (3, 4), (4, 1), (4, 2), (4, 3)]

Exercises¶

Write a list comprehension statement to generate a list of all pairs of odd positive integer values less than 10 where the first value is less than the second value.

Summary and Discussion¶

We’ve explored programming that is more compact and at a higher level of abstraction. It can be used to effectively interact with data.
map and filter each take a function and a sequence (an “iterable”) as arguments and produce an iterator as a result:
- map produces the result of applying the function to each element of the iterable
- filter produces each element of the iterable for which the function returns True
Both map and filter are made more compact by using lambda functions
lambda functions can also be used to change the result of sorting
A stable sort preserves the relative order of “tied” values
List comprehensions can be used in place of map and filter:
- Some people prefer list comprehensions because they often do not require lambda functions, but...
- List comprehensions explicitly construct the list of results rather than generating them one-by-one, which is what map and filter do. This makes them less efficient for large data sets.
These are all examples of functional programming.
We’ve also used the other major programming paradigms this semester
- imperative programming
- object oriented programming
Many modern languages like Python provide tools that allow programming using a combination of paradigms