Lecture 23 — Advanced Python Topics and Functional Programming


  • Advanced Python techniques and topics:
    • functions as parameters
    • sorting and keys
    • map, reduce, filter
    • lambda functions
    • xranges, iterators and generators
  • Efficiency, simplicity and abstraction
  • Functional programmming


  • We’ll consider functions that are applied to entire lists: sum, min, max, slicing, and we will compare them to the code we would write to how we would generate them with just while loops.
  • We should notice important commonality with relatively minor differences
  • Functional programming, which we discuss in this lecture, allows us to exploit this commonality and write simpler expressions at a higher level of abstraction
  • It also leads us into a different style of programming called functional programming

Functions as parameters

  • Sort comparison function must produce -1 for less than, 0 for equal, and +1 for greater than.

    |  sort(...)
    |      L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*;
    |      cmp(x, y) -> -1, 0, 1
  • We write this function and then pass it as an argument to the sort function associated with the list object.

  • In other words, the function is treated exactly the same way as any other argument passed to a function.

  • We will look at example of sorting names by last name and then by first.

Map: Apply a function to each element of a list

  • Suppose we want to count the number of values in a list of lists. We can use map to apply the len function to each sublist.

    >>> v = [ [2, 3, 5, 7], [11,13,17,19], [23, 29], [31,37] ]
    >>> print map( len, v)
    [4, 4, 2, 2]
  • In order to get our final answer, we just apply `sum`:

    >>> print sum(map(len,v))
  • Now suppose we want to find the maximum distance of a list of points from the origin. Here we’ll have to write a function

    def dist2D( p ):
        return (p[0]**2 + p[1]**2)**0.5
    pts = [ (4.5, 3), (2.1,-1), (6.8,-3), (1.4, 2.9) ]
    print map( dist2D, pts)
    print max( map(dist2D,pts) )

Lambda functions: Anonymous functions

  • We can avoid writing a separate function here by writing an anonymous function called a lambda function.

    • Aside: the notion of a lambda function goes all the way back to the origin of computer science
  • Our first example is just squaring the values of a list

    >>> map( lambda x: x**2, [ 1, 2, 3, 4 ] )
    [ 1, 4, 9, 16 ]
  • Now, we can sum the squares from 1 to n

    >>> n = 100
    >>> sum( map( lambda x: x**2, range(1,n+1)))
  • Our second example implements the dist2D function anonymously:

    >>> max( map( lambda p: (p[0]**2 + p[1]**2)**0.5, pts) )


  1. Use map to generate a new list where all values are replaced by their absolute values. You don’t need to use a lambda function.
  2. Use map and a lambda function to convert a list of Fahrenheit temperatures to a list of Celsius temperatures.
  3. Use map and a lambda function to find the maximum x coordinate (the 0-th coordinate) in a list of points. You will need to apply max to the result of the map

Reduce: Combine entries of a list into a single summary value

  • The functions sum and max reduce a list to a single value.

  • Python contains a special function called reduce that allows you to apply functions that already exist or that you write - lambda functions or named functions - to create this behavior in other ways

    • These functions must involve two arguments
  • The simplest is to just recreate the summation function:

    >>> reduce( lambda x,y: x+y, xrange(1,101))
  • Note that this also gives us another (!) way to concantentate strings.

Filter: Extract / eliminate values from a list

  • Let’s think about how to eliminate all of the negative values from a list, using a for loop. We will work on this in class

  • Once again we can we can simplify this using the built-in Python construct called filter

    >>> v = [ 1, 9, -4, -8, 10, -3 ]
    >>> filter( lambda x: x>0, v)
    [1, 9, 10]
  • Here the lambda function must produce a boolean value and if that value is True the list item is preserved.


  1. Use filter to eliminate all words that are shorter than 4 letters from a list of words
  2. Use filter to determine the percentage of Fahrenheit temperatures in a list are within the range 32 to 80
  3. Use reduce to find the lower left corner (minimum x and minimum y value) for a list of point locations

List Comprehensions

  • Instead of map and filter some people prefer another example of functional programming in Python called list comprehensions

  • Here is an example to generate the squares of the first n integers:

    n = 8
    >>> [ i*i for i in xrange(1,n+1) ]
    [1, 4, 9, 16, 25, 36, 49, 64]
  • The form of this is an expression followed by a for loop statement.

  • We can get the effect of filter by adding a conditional at the end:

    >>> v = [ 1, 9, -4, -8, 10, -3 ]
    >>> [ x for x in v if x>0 ]
    [1, 9, 10]
  • Here, the values are only generated in the resultant list when the if condition passes.

  • We can combine these as well. As a slightly silly example, we can eliminate the negative values and square the positive values

    >>> v = [ 1, 9, -4, -8, 10, -3 ]
    >>> [ x*x for x in v if x>0 ]
    [1, 81, 100]
  • We can get even more sophisticated by nesting for loops. Here is an example where we generate all pairs of numbers between 1 and 4, except for the pairs where the numbers are equal

    >>> [ (i,j) for i in xrange(1,5) for j in xrange(1,5) if i != j ]
    [(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2),
     (3, 4), (4, 1), (4, 2), (4, 3)]


  1. Write a list comprehension statement to convert a list of Fahrenheit temperatures to Celsius
  2. Write a list comprehension statement to generate a list of all pairs of odd positive integer values less than 10 where the first value is less than the second value.


  • Programming that is more compact and at a higher level of abstraction
  • lambda, map, reduce and filter
    • Distributed computation tools such as Hadoop feature variations on the map and reduce concepts.
  • List comprehensions can replace map and filter
  • These are all examples of functional programming.
  • We’ve also used the other major programming paradigms this semester
    • imperative programming
    • object oriented programming
  • Many modern languages like Python provide tools that allow programming using a combination of paradigms