# Lecture 24 — Advanced Python Topics and Functional Programming¶

## Problems We’d Like to Solve¶

Some of these are toy problems, but they illustrate use of tools we’d like to develop and use:

1. How many values are in a list of lists?
2. What is the maximum distance from the origin of the points in a list?
3. What is the sum of squares of the first n integers?
4. Can you sum the positive values in a list?
5. Can you sort a list of points by y value (2nd coordinate) and then by x value?

## Solution Techniques¶

• We can solve most of these with a for loop, but they can be solved even more effectively / efficiently / compactly using advanced Python methods.
• Leads to notions of:
• map and filter
• functions as parameters
• lambda functions
• stable sort
• list comprehensions
• Most are examples of functional programmming

## Map: Apply a function to each element of a list¶

• Suppose we want to count the number of values in a list of lists. We can use map to apply the len function to each sublist.

>>> v = [ [2, 3, 5, 7], [11,13,17,19], [23, 29], [31,37] ]
>>> print( list(map( len, v)) )
[4, 4, 2, 2]

• map is an iterator class:

• It produces values in a sequence, one after another, by applying the function (1st argument) to the values of the second argument.
• Technically, an iterator class is one that has the __next__ method implemented (correctly).
• Using list gives us the list of lengths of the sublists explicitly.
• To complete the solution we need to just apply sum:

>>> print sum(map(len,v))
12


Notice that this does not explicitly form an intermediate list.

## Passing Functions as Parameters¶

• The above example passes the len function as an argument!

• We also passed functions as arguments to our callbacks in our GUI programs
• This illustrates the concept that Python treats function as “first-class” objects - in other words functions can be used just like variables and other data.

• What’s passed as an argument to map() is the location of the function code.
• Now suppose we want to find the maximum distance of a list of points from the origin. Here we’ll have to write a function

def dist2D( p ):
return (p[0]**2 + p[1]**2)**0.5

pts = [ (4.5, 3), (2.1,-1), (6.8,-3), (1.4, 2.9) ]
print(max( map(dist2D,pts) ))


## Lambda functions: Anonymous functions¶

• We can avoid the need to write a separate function here by writing an anonymous function called a lambda function.

• Our first example is just squaring the values of a list

>>> list(map( lambda x: x**2, [ 1, 2, 3, 4 ] ))
[ 1, 4, 9, 16 ]

• Now, we can sum the squares from 1 to n

>>> n = 100
>>> sum( map( lambda x: x**2, range(1,n+1)))

• We can also implement the dist2D function anonymously:

>>> max( map( lambda p: (p[0]**2 + p[1]**2)**0.5, pts) )
7.432361670424818

• Notice that we did not need to explicitly form a list in each of the preceeding examples. This leads to substantial savings when the list is large!
• Aside: the notion of a lambda function goes all the way back to the origin of computer science

## In-Class Practice Problem:¶

1. Starting with the following list of x,y point coordinate types, we will use map(), a lambda function, and max() to find the maximum x coordinate (the 0-th coordinate) in a list of points.

pts = [ (6,-1), (8,4), (7.5,-3), (4.4,12), (7,2) ]


## Filter: Extract / eliminate values from a list¶

• Consider a different problem: how to eliminate all of the negative values from a list. Based on what we know so far, this requires a for loop with append.

• We can simplify this using the built-in Python construct called filter

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> list(filter( lambda x: x>0, v))
[1, 9, 10]

• Here,

• The lambda function must produce a boolean value and if that value is True the list item is kept; otherwise it is eliminated.
• The result of filter is an iterator object, just like the result of map is. We convert to a list in order to display the answer.
• If we want to sum up the non-negative values, then we don’t need to explicitly generate a list:

>>> sum(filter( lambda x: x>0, v))
20


## Lecture Exercises, Problems 1 and 2:¶

• At this point students will be given the chance to work on the first two lecture exercises.

## Passing Functions to Sort¶

• Consider the problem of sorting a list of (x,y) points by their y values first and their x values for tied y values, both in decreasing order. For example, given

pts = [ (2,5), (12,3), (12,1), (6,5), (14, 10), (12, 10), \
(8,12), (5,3) ]


we’d like the sorted order to be

[(8, 12), (14, 10), (12, 10), (6, 5), (2, 5), (12, 3), \
(5, 3), (12, 1)]

• The Python sort function

>>>  sorted( pts, reverse=True )
[(14, 10), (12, 10), (12, 3), (12, 1), (8, 12), (6, 5), \
(5, 3), (2, 5)]


gives the ordering by x value and then by y value. This is not what we want.

• The first step to a solution is to provide a key function to sorted() to pull out the information (the y value in this case) from each tuple to use as the basis for sorting:

>>> sorted( pts, key = lambda p: p[1], reverse=True)
[(8, 12), (14, 10), (12, 10), (2, 5), (6, 5), (12, 3), \
(5, 3), (12, 1)]


This is close but not quite right because the two points with y=5 are out of order.

• The trick is to sort by x first and then sort by y!

>>> by_x = sorted(pts,reverse=True)
>>> by_x
[(14, 10), (12, 10), (12, 3), (12, 1), (8, 12), (6, 5), \
(5, 3), (2, 5)]
>>> sorted( by_x, key = lambda p: p[1], reverse=True)
[(8, 12), (14, 10), (12, 10), (6, 5), (2, 5), (12, 3), \
(5, 3), (12, 1)]

• This works because sorted() uses what’s known as a stable sort: when two values are “tied” according the sorting criteria (y value in the second sort) their relative ordering (by x value from the first sort) in the final list is preserved.

• Therefore, (6,5) comes earlier than (2,5), while (12,3) comes earlier than (5,3)
• A number of variations on sorting use this “stable sort” property, but not all fast sorting algorithms are stable.

• Of course, we can also extend our lambda to reverse the tuple provided to sort()

>> sorted( pts, key = lambda p: (p[1], p[0]), reverse=True)
[(8, 12), (14, 10), (12, 10), (6, 5), (2, 5), (12, 3), \
(5, 3), (12, 1)]


## Practice Problem¶

1. Use filter to eliminate all words that are shorter than 4 letters from a list of words

## List Comprehensions¶

• Instead of map and filter some people prefer another example of functional programming in Python called list comprehensions

• Here is an example to generate a list of the squares of the first n integers:

>>> n = 8
>>> [ i*i for i in range(1,n+1) ]
[1, 4, 9, 16, 25, 36, 49, 64]

• The form of this is an expression followed by a for loop statement.

• We can get the effect of filter by adding a conditional at the end:

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x for x in v if x>0 ]
[1, 9, 10]

• Here, the values are only generated in the resultant list when the if condition passes.

• We can combine these as well. As a slightly silly example, we can eliminate the negative values and square the positive values

>>> v = [ 1, 9, -4, -8, 10, -3 ]
>>> [ x*x for x in v if x>0 ]
[1, 81, 100]

• We can get even more sophisticated by nesting for loops. Here is an example where we generate all pairs of numbers between 1 and 4, except for the pairs where the numbers are equal

>>> [ (i,j) for i in range(1,5) for j in range(1,5) if i != j ]
[(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2),
(3, 4), (4, 1), (4, 2), (4, 3)]


## Exercises¶

1. Write a list comprehension statement to generate a list of all pairs of odd positive integer values less than 10 where the first value is less than the second value.

## Summary and Discussion¶

• We’ve explored programming that is more compact and at a higher level of abstraction. It can be used to effectively interact with data.
• map and filter each take a function and a sequence (an “iterable”) as arguments and produce an iterator as a result:
• map produces the result of applying the function to each element of the iterable
• filter produces each element of the iterable for which the function returns True
• Both map and filter are made more compact by using lambda functions
• lambda functions can also be used to change the result of sorting
• A stable sort preserves the relative order of “tied” values
• List comprehensions can be used in place of map and filter:
• Some people prefer list comprehensions because they often do not require lambda functions, but...
• List comprehensions explicitly construct the list of results rather than generating them one-by-one, which is what map and filter do. This makes them less efficient for large data sets.
• These are all examples of functional programming.
• We’ve also used the other major programming paradigms this semester
• imperative programming
• object oriented programming
• Many modern languages like Python provide tools that allow programming using a combination of paradigms