# Lecture 23 — Advanced Python Topics and Functional Programming¶

## Overview¶

• Advanced Python techniques and topics:
• functions as parameters
• sorting and keys
• map, reduce, filter
• lambda functions
• xranges, iterators and generators
• Efficiency, simplicity and abstraction
• Functional programmming

## Motivation¶

• We’ll consider functions that are applied to entire lists: `sum`, `min`, `max`, slicing, and we will compare them to the code we would write to how we would generate them with just while loops.
• We should notice important commonality with relatively minor differences
• Functional programming, which we discuss in this lecture, allows us to exploit this commonality and write simpler expressions at a higher level of abstraction
• It also leads us into a different style of programming called functional programming

## Functions as parameters¶

• Sort comparison function must produce -1 for less than, 0 for equal, and +1 for greater than.

```|  sort(...)
|      L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*;
|      cmp(x, y) -> -1, 0, 1
```
• We write this function and then pass it as an argument to the `sort` function associated with the `list` object.

• In other words, the function is treated exactly the same way as any other argument passed to a function.

• We will look at example of sorting names by last name and then by first.

## Map: Apply a function to each element of a list¶

• Suppose we want to count the number of values in a list of lists. We can use `map` to apply the `len` function to each sublist.

```>>> v = [ [2, 3, 5, 7], [11,13,17,19], [23, 29], [31,37] ]
>>> print map( len, v)
[4, 4, 2, 2]
```
• In order to get our final answer, we just apply ``sum``:

```>>> print sum(map(len,v))
12
```
• Now suppose we want to find the maximum distance of a list of points from the origin. Here we’ll have to write a function

```def dist2D( p ):
return (p[0]**2 + p[1]**2)**0.5

pts = [ (4.5, 3), (2.1,-1), (6.8,-3), (1.4, 2.9) ]
print map( dist2D, pts)
print max( map(dist2D,pts) )
```

## Lambda functions: Anonymous functions¶

• We can avoid writing a separate function here by writing an anonymous function called a lambda function.

• Aside: the notion of a lambda function goes all the way back to the origin of computer science
• Our first example is just squaring the values of a list

```>>> map( lambda x: x**2, [ 1, 2, 3, 4 ] )
[ 1, 4, 9, 16 ]
```
• Now, we can sum the squares from 1 to n

```>>> n = 100
>>> sum( map( lambda x: x**2, range(1,n+1)))
```
• Our second example implements the `dist2D` function anonymously:

```>>> max( map( lambda p: (p[0]**2 + p[1]**2)**0.5, pts) )
7.432361670424818
```

## Exercises:¶

1. Use `map` to generate a new list where all values are replaced by their absolute values. You don’t need to use a lambda function.
2. Use `map` and a lambda function to convert a list of Fahrenheit temperatures to a list of Celsius temperatures.
3. Use `map` and a lambda function to find the maximum x coordinate (the 0-th coordinate) in a list of points. You will need to apply `max` to the result of the `map`

## Reduce: Combine entries of a list into a single summary value¶

• The functions `sum` and `max` reduce a list to a single value.

• Python contains a special function called `reduce` that allows you to apply functions that already exist or that you write - lambda functions or named functions - to create this behavior in other ways

• These functions must involve two arguments
• The simplest is to just recreate the summation function:

```>>> reduce( lambda x,y: x+y, xrange(1,101))
5050
```
• Note that this also gives us another (!) way to concantentate strings.

## Filter: Extract / eliminate values from a list¶

• Let’s think about how to eliminate all of the negative values from a list, using a for loop. We will work on this in class

• Once again we can we can simplify this using the built-in Python construct called `filter`

```>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> filter( lambda x: x>0, v)
[1, 9, 10]
```
• Here the lambda function must produce a boolean value and if that value is `True` the list item is preserved.

## Exercises¶

1. Use `filter` to eliminate all words that are shorter than 4 letters from a list of words
2. Use `filter` to determine the percentage of Fahrenheit temperatures in a list are within the range 32 to 80
3. Use `reduce` to find the lower left corner (minimum x and minimum y value) for a list of point locations

## List Comprehensions¶

• Instead of `map` and `filter` some people prefer another example of functional programming in Python called list comprehensions

• Here is an example to generate the squares of the first n integers:

```n = 8
>>> [ i*i for i in xrange(1,n+1) ]
[1, 4, 9, 16, 25, 36, 49, 64]
```
• The form of this is an expression followed by a `for` loop statement.

• We can get the effect of `filter` by adding a conditional at the end:

```>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x for x in v if x>0 ]
[1, 9, 10]
```
• Here, the values are only generated in the resultant list when the `if` condition passes.

• We can combine these as well. As a slightly silly example, we can eliminate the negative values and square the positive values

```>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x*x for x in v if x>0 ]
[1, 81, 100]
```
• We can get even more sophisticated by nesting `for` loops. Here is an example where we generate all pairs of numbers between 1 and 4, except for the pairs where the numbers are equal

```>>> [ (i,j) for i in xrange(1,5) for j in xrange(1,5) if i != j ]
[(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2),
(3, 4), (4, 1), (4, 2), (4, 3)]
```

## Exercises¶

1. Write a list comprehension statement to convert a list of Fahrenheit temperatures to Celsius
2. Write a list comprehension statement to generate a list of all pairs of odd positive integer values less than 10 where the first value is less than the second value.

## Discussion¶

• Programming that is more compact and at a higher level of abstraction
• `lambda`, `map`, `reduce` and `filter`
• Distributed computation tools such as Hadoop feature variations on the `map` and `reduce` concepts.
• List comprehensions can replace `map` and `filter`
• These are all examples of functional programming.
• We’ve also used the other major programming paradigms this semester
• imperative programming
• object oriented programming
• Many modern languages like Python provide tools that allow programming using a combination of paradigms