This lesson will discuss some of the important programming concepts that you will need for this course. Even if you think that you are an excellent programmer, you should read this section closely.
The C Language
All of the example code for this course will be in C. This is because a lot of what we will be doing will be making system calls, and the system call interface is identical to the C function call interface for both of the operating systems that we will cover in this course. We might look at some Unix kernel source code and most versions of Unix are written mostly in C, with a little assembler.
Most of the Windows operating system code is also written in C, although the source code is not publicly available.
C is a subset of C++. Most students have been programming in C++ rather than C, so this may take some adjustments. If you have not programmed in C before, here is a brief C tutorial for C++ programmers
The Unix C compiler that we will use is gcc (gnu c compiler).
To use gcc in its simplest form to compile a C program called hello.c, enter this at the command line.
> gcc hello.c
If there are no errors, the prompt will be displayed almost immediately. The name of the executable program will be a.out To run the program, just type ./a.out at the command line.
Digression: The Unix shell (in fact all Unix processes) have an area of memory called the environment, where a number of environment variables are define. The Unix command printenv displays all of the currently set environment variables. These always take the form
VARNAME=value
By convention, the variable names are in all upper case. Here is a sample
USER=ingallsr
HOME=/cs/ingallsr
TERM=dttermOne of these environment variables is called PATH. This consists of a list of directories, delimited by colons, where the shell searches for executable files when you type a command. To display the value of PATH, enter
> echo $PATH
The output might be something like this./usr/local/bin/:/usr/local/sbin/:/usr/bin/:/usr/local/X11R6/bin/When you type a command such as a.out, the shell has to find where this executable is. It searches the first directory in the path (/usr/local/bin), then the second (/usr/local/sbin), then the third (/usr/bin), and so on until it finds an executable file of that name, and it executes it. If there is no executable program called a.out in any of the directories in the path, the shell will display command not found. If there are executable files called a.out in more than one of the directories in the path, it will execute only the first one that it finds.
The first dot in the command ./a.out refers to the current directory. This tells the shell not to search the directories in the path, but only to search in the current working directory. There is some controversy about whether . should be in the list of PATH directories, and if so, where it should be. One camp says that . should be the first entry in the path. If that is the case, then you can just type a.out, and it will execute the program that you just compiled. Another camp says that the dot should be at the end of the path. In this case, if you type a.out, it will execute the file that you just created with the compile, unless there happens to be another executable file called a.out in one of the directories in the path, in which case, that will be executed instead. A third camp says not to put the dot in your path at all. The system administrator will set a default path for you and usually dot is not in the default path.
The argument against putting a dot in your path is that it is a potential security problem. If someone breaks into your computer, they can put an executable called ls in your home directory, which does something malicious. The next time that you log in and type ls, which is the most commonly used command, instead of executing the system ls it will execute the malicious version.
There should be a hidden file in your home directory called .bashrc. (In Unix, a hidden file has a dot as the first character. If you want ls to show hidden files, use the -a option (ls -a)). This file contains statements that are run whenever bash is started. If you want to put . at the end of your path, add the following line to .bashrc.
export PATH=$PATH:.The export command is a shell command that sets the environment to be exported to any program that it launches. This command says to set the value of PATH to its current value with a :. concatenated onto the end.
To put . at the start of your path, type this
export PATH=.:$PATH
Back from the digression.
The C compiler has literally hundreds of options. You can learn about all of them by typing man gcc. There are only a few that will be important for this course.
Here is a link to a short overview of gdb.
Any large program will have more than one source file, and it is possible to pass in several source files as arguments to gcc, and these will be combined to produce a single executable. There must be one and only one function called main in one of these files, and this will be the entry point.
For example
>gcc FileOne.c FileTwo.c FileThree.c
will produce a single executable called a.out
>gcc -o hello -g FileOne.c FileTwo.c FileThree.c
will produce an executable file called hello and you will
be able to use gdb to debug it.
What happens during a compile
It is worth spending some time looking at what happens when a C program is compiled. The example will be from the Unix compiler but the principles apply to any C compiler. The process of creating the executable file involves at least four separate steps, preprocessing the input, the actual compilation to produce an assembler file, assembling this to create an object file, and linking to create an executable.
The C Preprocessor
A preprocessor is a program that takes as input a C program which has preprocessor directives in it, and it expands these directives. The output is a C program with the preprocessor directives expanded. Preprocessor directives are sometimes called macros, particularly in assembler. Expansions always take the form of string substitution.
C preprocessor directives are indicated by a # in
column 1 of a line.
One directive that you are almost
certainly familiar with is
#include
This is followed by the name of a file (by convention the file
has a .h suffix) and the contents of that file are copied
into the output file. The file name is enclosed either in < > or in double
quotes. If the file name is in < >, the preprocessor
will look in the directory /usr/include (or some other directory
set by the administrator). If the file is in double quotes, the preprocessor
will look in the current directory or will treat it as an absolute
pathname.
Another simple preprocessor directive is
#define
This usually takes two arguments. The preprocessor would simply
replace any instance of the first string by the second string.
For example:
#define BUFSIZE 1024
The preprocessor would replace all instances of BUFSIZE in the
input file with 1024 in the output file.
The string char buffer[BUFSIZE]
would be replaced by
char buffer[ 1204 ]
Alert: A common error is to put a semicolon at the end of the second
string. This results in a compiler error which is hard to detect.
For example
#define BUFSIZE 1024;
would result in the output string
char buffer[ 1024; ] which is syntactically wrong.
It is possible to write macro expansions that take arguments.
For example
#define SQUARE(X) X * X
If there were a line in the text that looked like this:
n = SQUARE(m);
This would be expanded to
n = m * m;
Here is a more complex example
#define SWAP(TYPE,M,N) {TYPE temp; temp=M; M=N; N=temp;}
The following line in the text
SWAP(int,a,b)
would be expanded to
{int temp; temp=a; a=b; b=temp;}
Note that no actual processing, or even checking for correct syntax takes place during preprocessing. The preprocessor is simply substituting one string for another.
The preprocessor can define a variable without actually setting
a value. For example:
#define _MYHEADER_H_
This is used for controlling Conditional Compilation, another
feature of the preprocessor. Conditional compilation means that
certain code will be compiled only if a variable is defined or not
defined, with the preprocessor keywords #ifdef (if defined) and
#ifndef (if not defined) along with a matching #endif
For example:
#ifndef _MYHEADER_H_
#define _MYHEADER_H_
/* code for my header.h, which will only be
compiled if _MYHEADER.H_ had not been
previously defined
*/
#endif
It is possible to define variables on the gcc command line as well
with the -D option. For example
> gcc -D __sparc__ -o outfile infile.c
You could then put code in infile.c like this:
#ifdef __sparc__ /* code to be compiled for sparc, but not for other architectures. */ #endif
The C preprocessor is cpp. The input is a file with preprocessor directives
and the output is a file with all of the preprocessor directives expanded.
This file will be given a temporary name with an .i suffix.
The temporary file is automatically deleted after it has been
used, but you can stop the process after this step by passing the -E flag
to gcc. The output file will be written to standard output, so
you can redirect it to a file if you would like to see what the preprocessor
did. For example
> gcc -E myfile.c > myfile.i
Click here for an exercise on the preprocessor
The actual compilation
The C compiler itself is called cc1. Its input is the file which was the output of the preprocessor, in other words, a pure C file. The output is an assembler file. On Solaris, assembler files have a .s suffix. If you would like to look at the assembler file, you can stop the process before assembly by passing the -S flag to gcc. Otherwise, after the assembly process, the .s file is deleted.
Assembly
The gcc script next evokes the assembler on the host machine. On most Unix machines, the native assembler is as, and the GNU equivalent is gas. The input is the assembler file produced by the compiler, the output is an object file, with a .o suffix (a .obj suffix on Windows).
You can stop the process after the creation of the object file but before the creation of the actual executable by using the -c flag with gcc.
If there is more than one input file, the above process (preprocessing, compiling, assembling) is repeated for each input file before the next step.
Linking
Suppose we have two source files which look like this.
First File, file1.c
/* file1.c */
#include <stdio.h>
int g; /* a global variable */
extern double dg; /* another global var, defined
in some other file */
void fctnOne(); /* a function prototype */
int main()
{
int x; /* a variable local to main
(an automatic variable) */
x = 3;
dg = 3.14;
g = 17;
fctnOne();
printf("x is %d, g is %d, dg is %f\n",
x, g, dg);
return 0;
}
void fctnTwo()
{
int x;
x = 5;
g = 11;
dg = dg * 2;
}
Second File file2.c
/* file2.c */
extern int g;
double dg;
void fctnTwo(); /* function prototype */
void fctnOne()
{
int x = 44;
g = x;
dg = dg + 2;
fctnTwo();
}
If the compile line looks like this;
> gcc -g -Wall file1.c file2.c
preprocessing, compiling and assembling the two source files would
produce two object files called file1.o
and file2.o. However, both of these
have unresolved references, i.e. references to variables and
functions which are defined in some other file. When file1.o is
produced, it will contain a call to a function called fctnOne, but it
does not know the address of this function at assembly time.
Likewise, there is a reference to a variable dg and the assembler does not know the address of
this variable. There is also a call to printf, which is not defined
in either file. The file file2.o has
unresolved references to fctnTwo and
g.
The job of the linker is to resolve all of these unresolved
references. It does this by using two tables which are
attached to each of the object files. One table, the
definition table, lists all of the global functions
and variables which are defined in that file, along with
the address of each of these. The other table, the
use table lists each instance where an undefined
variable or function is used.
Here are the four tables for these two files.
| Definition table for file1.c | Use table for file1.c |
|---|---|
| g | gd (line 13) |
| main() | fctnOne() (line 15) |
| fctnTwo() | printf() (line 16) |
| dg (line 17) | |
| dg (line 26, first instance) | dg (line 26, second intance) |
| Definition table for file2.c | Use table for file2.c |
|---|---|
| gd | g (line 9) |
| fctnOne() | fctnTwo() (line 11) |
The linker usually does its job in two passes. It first goes through all of the definition tables and builds a global definition table consisting of all variables and functions defined in any of the files along with their addresses. It then goes through all of the files again replacing all unresolved references listed in the use table with the actual address.
Finally, the linker has to search libraries to resolve yet more unresolved references. The linker is usually configured to automatically search the standard C library libc which contains code for functions such as printf. You can tell the linker to search other libraries with the -l flag (which can be passed to the call to gcc. For example, if you use functions in the math library, you can tell the linker to link to this with the -lm flag.
Most modern compilers use dynamic linking to link to library functions. With static linking, the executables for the libraries are directly linked as a part of the executable. With dynamic linking, library symbolic names are stored in the executable, and while the program is running, when a call to a library function is encountered, the operating system has to look up the location of the code for that executable before it actually executes it. The advantage of dynamic linking is that there is only one instance of the code for often used library functions like printf (with static linking, the code for printf is copied into each executable that uses it). The disadvantage of dynamic linking is that there is a small run time penalty because the system has to look up the address of a library function each time that it is called.
The linker also has to combine all of the various inputs into a single executable image, and this may need to involve relocation of addresses in some or all of the modules.
Here is an exercise on the linker
Passing arguments to programs
When you run a program from the command line, you can pass arguments
to the program. The function main() can access these arguments.
To do this, write your main as if it had two arguments int argc
and char *argv[] . The variable argc will
automatically be set at run time to contain the total number of arguments,
including the name of the executable itself. The variable argv
(the argument vector) is an array of pointers to character strings. The size of
the array is argc. Here is a short program which displays its
arguments, one per line.
#include <stdio.h>
int main(int argc, char *argv[])
{
int i;
for (i=0;i<argc;i++) {
printf("%s\n",argv[i]);
}
return 0;
}
If this is the command linea.out first second thirda.out first second thirdThe value of
argc would be 4.
Compiling C or C++ programs on Windows
On Windows machines, we will be using the Microsoft .NET compiler. If there are students who do not have convenient access to a Microsoft Compiler, you can use any of a number of freeware compilers. However you need prior approval from the instructor because the graders will need to be able to grade your projects, and the compiler will have to have access to the WIN32 APIs.
Hungarian Notation
If you read Microsoft C/C++ documentation or Microsoft sample code, you will need to understand Hungarian Notation. This is a method of naming variables which is a Microsoft convention. For example, the online help which describes the function ReadFile looks like this.
BOOL ReadFile(
HANDLE hFile,
LPVOID lpBuffer,
DWORD nNumberOfBytesToRead,
LPDWORD lpNumberOfBytesRead,
LPOVERLAPPED lpOverlapped
);
Hungarian Notation was developed by Charles
Simonyi - the original Microsoft chief software architect (he was
presumably of Hungarian extraction, although I have also read
that the term Hungarian Notation was sarcastic because it made
the code so hard to read). It is a naming convention that allows the
programmer to determine the type and use of an identifier (variable,
function, constant, etc.)
A variable name consists of an optional prefix, a tag which indicates the variable type, and the variable name. The prefix is lower case, the variable name itself starts with an upper case letter.
Common tags include
| Flag | Type | Example |
| b | Boolean | bGameOver |
| ch or c | single char | chGrade |
| dw | double word (32 bits) | dwBytesRead |
| n | integer | nStringLength |
| d | double precision real | dBalance |
| sz | null terminated char string | szLastName |
| p | pointer | pBuffer |
| lp | long pointer | lpBuffer |
| C | Class Name | CWidget |
In addition, most Microsoft sample code includes the header file
<windows.h> which redefines most data types.
You should never see a variable of type int in
Microsoft code. By convention, these are all uppercase.
Here are some examples.
| DWORD | unsigned long (stands for double word) |
| WORD | unsigned short (16 bits) |
| BOOL | boolean |
| BYTE | unsigned char |
| LPDWORD | pointer to a DWORD |
| LPVOID | Pointer to type void (a generic pointer) |
| LPCTSTR | Pointer to a const string |
Modern operating systems can run in two (or more) modes, typically called user mode and kernel mode. An ordinary user who compiles and runs an ordinary program is in user mode, which means that the program only has access to its own memory space. However, most user programs need to access services provided by the kernel. An obvious example is reading from a file. While the typical user is not allowed access to the low level code that reads individual sectors, there needs to be some mechanism for user programs to access these services. These calls to kernel services are called system calls in the Unix world and Application Programmer Interfaces or APIs in the Microsoft world.
Unix System Calls
There are about 190 Unix system calls (the number varies somewhat from one system to another). They always take the form of a C function call; i.e. they take arguments which can be either ordinary variables or pointers to ordinary variables and they return a value. The value returned is usually of type int. In general, a positive or zero return value indicates that the call was successful, and a negative return value indicates that the system call failed for some reason. If the system call failed, a global external int variable errno will be set, and your program can look at this value to determine why the system call failed.
Although errno is an int, its values have symbolic names. You need to look at the man pages for each system call to determine the meaning of these values. For example, the system call to open a file is open. On success, it returns a file descriptor which can be used by other system calls to access the file. However, there are numerous reasons why an attempt to open a file may fail. Here is part of the on-line man page for open.
ERRORS
The open() function will fail if:
EACCES
Search permission is denied on a component of the path
prefix, or the file exists and the permissions speci-
fied by oflag are denied, or the file does not exist
and write permission is denied for the parent direc-
tory of the file to be created, or O_TRUNC is speci-
fied and write permission is denied.
EDQUOT
The file does not exist, O_CREAT is specified, and
either the directory where the new file entry is being
placed cannot be extended because the user's quota of
disk blocks on that file system has been exhausted, or
the user's quota of inodes on the file system where
the file is being created has been exhausted.
EEXIST
The O_CREAT and O_EXCL flags are set, and the named
file exists.
EINTR A signal was caught during open().
EFAULT
The path argument points to an illegal address.
EISDIR
The named file is a directory and oflag includes
O_WRONLY or O_RDWR.
EMFILE
OPEN_MAX file descriptors are currently open in the
calling process.
ENFILE
The maximum allowable number of files is currently
open in the system.
Every possible error condition has a symbolic name (the words beginning with E in all caps) which are defined in a header file. The variable errno will be set on failure, and you can write code to find out what caused the error. Here is a skeleton of a C program that does this.
>
...
#include <errno.h>
extern int errno;
...
int main()
...
int returnval;
...
returnval = open(...)
if (returnval < 0) /* the open failed */
switch (errno) {
case EACCES: ..
case EDQUOT: ...
...
...
}
It is good programming practice to always check the return value of a system call that might fail (some system calls cannot fail), and take appropriate steps in the event of a failure. This will be required for all programming in this course.
Unix has a library function void perror(const char *msg) which displays your message along with a standard error message (based on the value of errno) on standard error.
Here is a code snippet showing how this might be used.
int fd; fd = open( ... ) if (fd < 0) perror(``Error opening file'')
If the call to open failed because there was no file of that name, this message would be displayed on the terminal.
Error opening file: No such file or directory
There is also a function char *strerror(int errno) (make sure to include the header file string.h) which returns a string corresponding to the error number passed in as an argument.
Posix
Because of the proliferation of Unix operating systems, there have been a number of attempts to standardize Unix system calls. The most successful of these is POSIX (Portable Operating System Interface), an IEEE standard. There are a number of parts to Posix, POSIX.1 defines C programming interfaces to system calls.
For portability, Posix has defined a number of data types.
For example, the size of a file (an int on any system that
I know of) should be of type size_t. This may be confusing
at first, but you will get used to it.
Windows APIs
In the Microsoft world, a system call is referred to as an
Win32 API (Application Program Interface). If you
use any of these, you should include the header file windows.h.
This file contains definitions, macros, and structures for source
code.
As in Unix, the Win32 APIs have the same form as C function calls.
The number of Win32 APIs is much larger than the number of Unix system calls. One of the major reason why the number is so much larger is that all of the graphics calls which create and manipulate windows are part of the Win32 APIs, while the comparable Unix calls to XWindows run in user space. We will probably not be using any of the graphics calls in this course.
All versions of the Windows operating systems use the Win32 APIs, but each uses a slightly different subset. I will try to only include those APIs which are common across all versions of the Windows operating systems.
The Win32 APIs will soon be replace by the Win64 APIs. The numbers 32 and 64 refer to word size.
There are a number of conventions that apply to Win32 APIs.
Most kernel resources are objects, and are referred to with an opaque data type called a HANDLE. You can think of a handle as a pointer to the object, but you do not know the members of the object (that is what the term Opaque Data Type means). Some APIs will return a handle, and you can then pass that handle to other APIs. For example, the API which opens a file returns a handle and you can then pass that handle to an API which reads from that file.
The return types of Win32 APIs are more variable than Unix system
calls. Some will return a HANDLE. If the API fails, you can check
this by comparing the return value to INVALID_HANDLE_VALUE.
There is a function GetLastError() which returns the error
code indicating why it failed.
For example, the Win32 API which opens a file is CreateFile(...)
and it returns a handle.
Here is some skeleton code to demonstrate this.
HANDLE h;
h = CreateFile(...
if (h == INVALID_HANDLE_VALUE) {
printf("Error, could not open the file, error: %d\n",
GetLastError());
...
Displaying an error message on the screen which tells why
the error occurred is a little tricky. Here is a function
which does this, but it is far beyond the scope of this course
to explain it. Your Windows programs should call this function
when a call to an API fails. It returns an explanation of the
error.
char *GetErrorMessage()
{
char *ErrMsg;
FormatMessage(
FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM |
FORMAT_MESSAGE_IGNORE_INSERTS,
NULL,
GetLastError(),
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), // Default language
(LPTSTR) &ErrMsg,
0,
NULL
);
return ErrMsg;
}
You can download this file so you don't need
to retype it.
Here is a link to the first programming assignment. It uses some Posix system calls and some of the WIN32 APIs.