Lecture 23 — Advanced Python Topics and Functional Programming¶

Overview¶

Advanced Python techniques and topics:
- functions as parameters
- sorting and keys
- map, reduce, filter
- lambda functions
- xranges, iterators and generators
Efficiency, simplicity and abstraction
Functional programmming

Motivation¶

We’ll consider functions that are applied to entire lists: sum, min, max, slicing, and we will compare them to the code we would write to how we would generate them with just while loops.
We should notice important commonality with relatively minor differences
Functional programming, which we discuss in this lecture, allows us to exploit this commonality and write simpler expressions at a higher level of abstraction
It also leads us into a different style of programming called functional programming

Functions as parameters¶

Sort comparison function must produce -1 for less than, 0 for equal, and +1 for greater than.

|  sort(...)
|      L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*;
|      cmp(x, y) -> -1, 0, 1

We write this function and then pass it as an argument to the sort function associated with the list object.
In other words, the function is treated exactly the same way as any other argument passed to a function.
We will look at example of sorting names by last name and then by first.

Map: Apply a function to each element of a list¶

Suppose we want to count the number of values in a list of lists. We can use map to apply the len function to each sublist.
```
>>> v = [ [2, 3, 5, 7], [11,13,17,19], [23, 29], [31,37] ]
>>> print map( len, v)
[4, 4, 2, 2]
```
In order to get our final answer, we just apply `sum`:
```
>>> print sum(map(len,v))
12
```

Now suppose we want to find the maximum distance of a list of points from the origin. Here we’ll have to write a function

def dist2D( p ):
    return (p[0]**2 + p[1]**2)**0.5

pts = [ (4.5, 3), (2.1,-1), (6.8,-3), (1.4, 2.9) ]
print map( dist2D, pts)
print max( map(dist2D,pts) )

Lambda functions: Anonymous functions¶

We can avoid writing a separate function here by writing an anonymous function called a lambda function.
- Aside: the notion of a lambda function goes all the way back to the origin of computer science

Our first example is just squaring the values of a list

>>> map( lambda x: x**2, [ 1, 2, 3, 4 ] )
[ 1, 4, 9, 16 ]

Now, we can sum the squares from 1 to n

>>> n = 100
>>> sum( map( lambda x: x**2, range(1,n+1)))

Our second example implements the dist2D function anonymously:

>>> max( map( lambda p: (p[0]**2 + p[1]**2)**0.5, pts) )
7.432361670424818

Exercises:¶

Use map to generate a new list where all values are replaced by their absolute values. You don’t need to use a lambda function.
Use map and a lambda function to convert a list of Fahrenheit temperatures to a list of Celsius temperatures.
Use map and a lambda function to find the maximum x coordinate (the 0-th coordinate) in a list of points. You will need to apply max to the result of the map

Reduce: Combine entries of a list into a single summary value¶

The functions sum and max reduce a list to a single value.
Python contains a special function called reduce that allows you to apply functions that already exist or that you write - lambda functions or named functions - to create this behavior in other ways
- These functions must involve two arguments

The simplest is to just recreate the summation function:

>>> reduce( lambda x,y: x+y, xrange(1,101))
5050

Note that this also gives us another (!) way to concantentate strings.

Filter: Extract / eliminate values from a list¶

Let’s think about how to eliminate all of the negative values from a list, using a for loop. We will work on this in class

Once again we can we can simplify this using the built-in Python construct called filter

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> filter( lambda x: x>0, v)
[1, 9, 10]

Here the lambda function must produce a boolean value and if that value is True the list item is preserved.

Exercises¶

Use filter to eliminate all words that are shorter than 4 letters from a list of words
Use filter to determine the percentage of Fahrenheit temperatures in a list are within the range 32 to 80
Use reduce to find the lower left corner (minimum x and minimum y value) for a list of point locations

List Comprehensions¶

Instead of map and filter some people prefer another example of functional programming in Python called list comprehensions

Here is an example to generate the squares of the first n integers:

n = 8
>>> [ i*i for i in xrange(1,n+1) ]
[1, 4, 9, 16, 25, 36, 49, 64]

The form of this is an expression followed by a for loop statement.

We can get the effect of filter by adding a conditional at the end:

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x for x in v if x>0 ]
[1, 9, 10]

Here, the values are only generated in the resultant list when the if condition passes.
We can combine these as well. As a slightly silly example, we can eliminate the negative values and square the positive values
```
>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x*x for x in v if x>0 ]
[1, 81, 100]
```

We can get even more sophisticated by nesting for loops. Here is an example where we generate all pairs of numbers between 1 and 4, except for the pairs where the numbers are equal

>>> [ (i,j) for i in xrange(1,5) for j in xrange(1,5) if i != j ]
[(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2),
 (3, 4), (4, 1), (4, 2), (4, 3)]

Exercises¶

Write a list comprehension statement to convert a list of Fahrenheit temperatures to Celsius
Write a list comprehension statement to generate a list of all pairs of odd positive integer values less than 10 where the first value is less than the second value.

Discussion¶

Programming that is more compact and at a higher level of abstraction
lambda, map, reduce and filter
- Distributed computation tools such as Hadoop feature variations on the map and reduce concepts.
List comprehensions can replace map and filter
These are all examples of functional programming.
We’ve also used the other major programming paradigms this semester
- imperative programming
- object oriented programming
Many modern languages like Python provide tools that allow programming using a combination of paradigms