Lecture 23 — Advanced Python Topics and Functional Programming
===============================================================

Overview
--------

- Advanced Python techniques and topics:

  - functions as parameters

  - sorting and keys

  - map, reduce, filter

  - lambda functions

  - xranges, iterators and generators

- Efficiency, simplicity and abstraction

- Functional programmming


Motivation
----------

- We'll consider functions that are applied to entire lists:  ``sum``,
  ``min``, ``max``, slicing, and we will compare them to the code we
  would write to how we would generate them with just while loops.

- We should notice important commonality with relatively minor
  differences

- Functional programming, which we discuss in this lecture, allows us
  to exploit this commonality and write simpler expressions at a
  higher level of *abstraction*

- It also leads us into a different style of programming called
  *functional programming*


Functions as parameters
-----------------------

- Sort comparison function must produce -1 for less than, 0 for equal,
  and +1 for greater than.

  ::

    |  sort(...)
    |      L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*;
    |      cmp(x, y) -> -1, 0, 1


- We write this function and then pass it *as an argument* to the
  ``sort`` function associated with the ``list`` object.

- In other words, the function is treated exactly the same way as any
  other argument passed to a function.

- We will look at example of sorting names by last name and then by
  first.


Map: Apply a function to each element of a list
-----------------------------------------------

- Suppose we want to count the number of values in a list of lists.
  We can use ``map`` to apply the ``len`` function to each sublist.

  ::

    >>> v = [ [2, 3, 5, 7], [11,13,17,19], [23, 29], [31,37] ]
    >>> print map( len, v)
    [4, 4, 2, 2]

- In order to get our final answer, we just apply ```sum```:

  ::

    >>> print sum(map(len,v))
    12

- Now suppose we want to find the maximum distance of a list of points
  from the origin.  Here we'll have to write a function

  ::

    def dist2D( p ):
        return (p[0]**2 + p[1]**2)**0.5

    pts = [ (4.5, 3), (2.1,-1), (6.8,-3), (1.4, 2.9) ]
    print map( dist2D, pts)
    print max( map(dist2D,pts) )


Lambda functions:  Anonymous functions
--------------------------------------

- We can avoid writing a separate function here by writing an
  *anonymous* function called a *lambda* function.

  - Aside: the notion of a lambda function goes all the way back to
    the origin of computer science

- Our first example is just squaring the values of a list

  ::

    >>> map( lambda x: x**2, [ 1, 2, 3, 4 ] )
    [ 1, 4, 9, 16 ]

- Now, we can sum the squares from 1 to n

  ::

    >>> n = 100
    >>> sum( map( lambda x: x**2, range(1,n+1)))

- Our second example implements the ``dist2D`` function anonymously:

  ::

    >>> max( map( lambda p: (p[0]**2 + p[1]**2)**0.5, pts) )
    7.432361670424818

Exercises:
----------

#. Use ``map`` to generate a new list where all values are replaced by
   their absolute values. You don't need to use a lambda function.

#. Use ``map`` and a lambda function to convert a list of Fahrenheit
   temperatures to a list of Celsius temperatures.

#. Use ``map`` and a lambda function to find the maximum x coordinate
   (the 0-th coordinate) in a list of points.  You will need to apply
   ``max`` to the result of the ``map``


Reduce: Combine entries of a list into a single summary value
-------------------------------------------------------------

- The functions ``sum`` and ``max`` *reduce* a list to a single value.

- Python contains a special function called ``reduce`` that allows you
  to apply functions that already exist or that you write - lambda
  functions or named functions - to create this behavior in other ways

  - These functions must involve two arguments

- The simplest is to just recreate the summation function:

  ::

    >>> reduce( lambda x,y: x+y, xrange(1,101))
    5050

- Note that this also gives us another (!) way to concantentate
  strings.


Filter:  Extract / eliminate values from a list
-----------------------------------------------

- Let's think about how to eliminate all of the negative values from a
  list, using a for loop.  We will work on this in class

- Once again we can we can simplify this using the built-in Python
  construct called ``filter``

  ::

    >>> v = [ 1, 9, -4, -8, 10, -3 ]
    >>> filter( lambda x: x>0, v)
    [1, 9, 10]

- Here the lambda function must produce a boolean value and if that
  value is ``True`` the list item is preserved.
  

Exercises
---------

#. Use ``filter`` to eliminate all words that are shorter than 4
   letters from a list of words

#. Use ``filter`` to determine the percentage of Fahrenheit
   temperatures in a list are within the range 32 to 80

#. Use ``reduce`` to find the lower left corner (minimum x and minimum
   y value) for a list of point locations


List Comprehensions
-------------------

- Instead of ``map`` and ``filter`` some people prefer another example
  of functional programming in Python called *list comprehensions*

- Here is an example to generate the squares of the first n integers:

  ::

    n = 8
    >>> [ i*i for i in xrange(1,n+1) ]
    [1, 4, 9, 16, 25, 36, 49, 64]

- The form of this is an expression followed by a ``for`` loop
  statement.

- We can get the effect of ``filter`` by adding a conditional at the
  end:

  ::

    >>> v = [ 1, 9, -4, -8, 10, -3 ]
    >>> [ x for x in v if x>0 ]
    [1, 9, 10]

- Here, the values are only generated in the resultant list when the
  ``if`` condition passes.

- We can combine these as well.  As a slightly silly example, we can
  eliminate the negative values and square the positive values

  ::

    >>> v = [ 1, 9, -4, -8, 10, -3 ]
    >>> [ x*x for x in v if x>0 ]
    [1, 81, 100]

- We can get even more sophisticated by nesting ``for`` loops. Here is
  an example where we generate all pairs of numbers between 1 and 4,
  except for the pairs where the numbers are equal

  ::

    >>> [ (i,j) for i in xrange(1,5) for j in xrange(1,5) if i != j ]
    [(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2),
     (3, 4), (4, 1), (4, 2), (4, 3)]

Exercises
---------

#. Write a list comprehension statement to convert a list of
   Fahrenheit temperatures to Celsius

#. Write a list comprehension statement to generate a list of all
   pairs of odd positive integer values less than 10 where the first value is
   less than the second value.


Discussion
----------

- Programming that is more compact and at a higher level of
  abstraction

- ``lambda``, ``map``, ``reduce`` and ``filter``

  - Distributed computation tools such as *Hadoop* feature variations
    on the ``map`` and ``reduce`` concepts.

- List comprehensions can replace ``map`` and ``filter``

- These are all examples of *functional programming*.

- We've also used the other major programming paradigms this semester

  - *imperative programming*

  - *object oriented programming*

- Many modern languages like Python provide tools that allow
  programming using a combination of paradigms