CSci 4968 and 6270: Computational Vision, Fall 2009

• Overview
• Requirements
• Academic Integrity
• Books
• On-line resources
• Programming
• Schedule, including lecture notes and homework assignments

Course Overview

Cameras and images are everywhere, with a resulting wide-spread need for algorithms and software that can combine, summarize and interpret these images. This is the field of computer vision and image analysis. Applications include face, object and scene recognition, security and biometrics, photomontaging and virtual tours, special effects in graphics, photography and the movies, autonomous robots, human-computer interaction, and medical diagnosis and treatment. Given the ubiquitous cell phone camera, the potential for new applications in the future seems unlimited. Standing in sharp contrast to this need, computer vision presents us with difficult problems. Each image is a large, noisy, quantized, 2d array. The world, by contrast, is (mostly, at our level) continuous, complicated and three-dimensional. The challenge of computer vision has led researchers and applications programmers to employ a wide variety of physical, engineering, mathematical, statistical, algorithmic and software techniques to develop computer vision systems. Therefore, our goal in this course will be to learn about the problems, the applications, the challenges, and the techniques of computer vision.

The first 40% of the course will focus on a single application - photomontaging - using it to drive the study of image processing, feature extraction, cameras and transformations, matching, estimation, and blending. After this we will turn to image segmentation, computational photography, three-dimensional modeling, motion and tracking, and various types of recognition.

Prerequisites include a data structures and algorithms course (e.g. CSci 2300), a course in multivariable calculus and linear algebra (Math 2100), and some "mathematical maturity". Students can either earn four undergraduate credits (CSci 4968) or three graduate credit (CSci 6270) for this course.

Course Requirements

Requirements include class attendance and participation (10%), 7 homework assignments (40%), one exam (20%), and a final project (30%). The homework and the final project will emphasize programming. Students are expected to submit their own solutions to the homework, although they may work together to sketch solution ideas.

Students have three "late" days they can use on homework assignments throughout the semester, with at most two days used for any one homework. A late day is defined as any whole or partial day after the submission deadline. For example, submitting a homework assignment 24 hours and 15 minutes late constitutes two late days. After all late days have been used, no late homework submission will be accepted without an excuse from the Dean of Students office. Late days may not be used for the final project.

Academic Integrity

Students are expected to submit their own solutions to the homework, although they may work together to sketch solution ideas. The final project should be done individually. Students who find resources to help with the final project must (a) reference these resources in their write-up and (b) add to these with their own work. Students will be graded on what is their own.

Books

Many books on computer vision have been published over the years. We will be using the on-line textbook, Computer Vision: Algorithms and Applications , which Dr. Richard Szeliski of Microsoft Research is currently writing. Here are several other books that students might find useful:

• Computer Vision a Modern Approach, Forsyth and Ponce, Prentice Hall, 2003. One of the most commonly-used books over the past decade, written by senior researchers in the field.
• Multiple View Geometry in Computer Vision, Hartley and Zisserman, Cambridge University Press, 2004. An excellent text on projective geometry, cameras, and estimation of 3d geometry.
• Introductory Techniques for 3-D Computer Vision, Trucco and Verri, Prentice Hall, 1998. An algorithmic approach.
• Pattern Classification, Duda, Hart and Stork, Wiley Interscience, 2001. Statistical techniques for two-dimensional pattern recognition. An excellent reference.
• Pattern Recognition and Machine Learning, Bishop, Springer, 2006. An important book on the use of machine learning in image analysis.
• An Introduction to Linear Algebra, Strang, Wellesley-Cambridge Press, 4th edition, 2009. My favorite book on linear algebra.

On-line Resources

• The Computer Vision Homepage (links to vision sites around the country).
• CVonline (a compendium of Computer Vision Bibliographic Information).

Programming

Programming can be done in Matlab or in C or C++. If you choose to use C or C++, you will want to make use of either OpenCV or VXL. The former is widely used. The latter is used in my lab, mostly for historical reasons, and it powerful and flexible, if a bit daunting.

Most students are likely to want to use Matlab, together with the Image Processing Toolkit. Students who don't already have these on their computer may sign up and download through RPI's campus system. Here is a selection of on-line resources for learning Matlab:

• Matlab Summary and Tutorial
• Matlab Tutorial
• The MathWorks - MATLAB Tutorial
• Matlab Primer
• On-line Matlab Help
• Writing Fast Matlab Code (pdf)
• Code Vectorization Guide
• Matlab Programmin Style Guidelines (pdf)

Course Schedule, Notes and Homework