CSCI.4210 Operating Systems Fall, 2008 Class 24
Security III

Read more about Alice, Bob and Eve here

Types of Security Attacks

The generic term malware is used to describe any kind of software which is loaded without the user's knowledge and which has malicious intent.

Trojan Horses

A Trojan horse is malware embedded inside a seemingly legitimate program. Someone may download software from the web that seems to perform as advertised, a game for example, but has code embedded in it that does evil things.

There are other ways besides downloading from the web as well. Early in the course we talked about the PATH environment variable and whether to put the dot at the front of the path, at the end of the path, or not at all. You can write a trojan horse version of ls, put it in your home directory, put dot at the first entry in your path, then induce the superuser to run it by doing something that would attract attention, such as creating a fork bomb (a program that just calls fork repeatedly).

Viruses

A virus is a set of executable statements that attaches itself to another executable file or replaces it completely. These can potentially insert themselves into any executable code, including kernel code such as interrupt vectors or even the boot sector. In this sense they are similar to biological viruses, which are unable to exist independently but are able to implant themselves in the genome of cells of other organisms.

Viruses have been installed in many different places in the Operating System. Traditionally, they have infected a program which is part of the kernel, but they can be in any executable code.

Your text gives a number of examples.

On a Windows computer, create an executable with the same name as a commonly used utility, only with the suffix .com instead of .exe. .com is a early file format that is not much used now, but if the user enters dir in a console window, the system first looks for a file called dir.com, and if it can't find it, it will search for dir.exe. This means that if you have inserted a file called dir.com in one of the directories in the path, it will execute this instead.

You can also change the target of a shortcut so that it executes your program rather than what it is supposed to point to.

Viruses can also be inserted even deeper into the system. A memory resident virus is in a part of the kernel which is always loaded, such as the interrupt handler vector, or a device driver. A boot sector virus replaces the boot sector of a disk with its own code. The boot sector contains a disk address which the OS jumps to when the system is started, so it is a simple task to change this address so that it jumps to the virus code instead of the normal boot.

In all of these examples, after executing the virus code, the last instruction is a jump to the normal code, so that the user does not suspect anything.

A variant is the macro virus. Many applications, such as spreadsheets, allow the user to write a macro, which is a script consisting of a number of keystrokes or commands, which can then be executed as a single command. These macros are very powerful, and in some cases can run Visual Basic programs. Thus the bad guy can embed such a virus in a spreadsheet or word document and email it to people. The unsuspecting user opens it, triggering the virus.

There are a number of ways to spread viruses. They can be embedded in executable programs on a website or just emailed to people. Anyone that runs the program then gets infected. Because a virus can do anything, one popular thing that they do now is to locate the user's email addressbook and send itself to everyone (or selected people) in the address book.

One of the most common and dangerous types of viruses now are botnets. A virus (often called a zombie) is distributed to many different unsuspecting users. It sits undetected for a while, and then wakes up and accesses the internet. These can be used to send out oceans of spam or to perform a distributed denial of service attack (DDoS). In the latter example, all of the zombies repeatedly try to connect to a web site with the intention of overloading it so that legitimate users cannot access it. This is one of the most difficult types of security threats to defend against, and because the attacks come from innocent victims, it is difficult to identify the real culprits.

Protecting against security violations

There are a number of tools which system administrators can use to detect and prevent outside attacks. For example, there is a freely downloadable program called SATAN (System Administrator's Tool for Analyzing Networks) which a system administrator can run to detect a large number of known security holes on Unix systems, including the following:

A newer product is Octave, (Operationally Critical Threat, Asset and Vulnerability Evaluation) - a suite of tools for risk based information security assessment and planning.

Virus detection

There are organizations, both commercial and noncommercial, that track viruses in order to detect them. Whenever a new virus is reported, they try to isolate it and find a signature, that is, a fragment of code that will uniquely identify the virus. It is then possible to scan every executable file on a system to see it has the signature of any known virus.

There is continual escalation between the virus creators and the virus detectors. The virus creator can create many versions of the virus, which differ in subtle ways. These are called polymorphic viruses. This means that the virus detection software has to perform a fuzzy search, not only looking for the signature, but also looking for small variants in the signature. This is more time consuming and more likely to result in false positives, legitimate programs that happen to look similar to the virus.

This method cannot detect brand new viruses.

Another method of detecting viruses is to scan all of the executable files on the disk which are known to be virus free, and calculate a value for each one. This can be as simple as the length of the file, or it can be a checksum, or an MD5 hash function. Periodically run the virus scan software and see if any of these values no longer match.

One problem with this is that the virus might be able to find the file where these values are stored, and update the appropriate value.

Virus avoidance

There are some obvious guidelines that users can follow to reduce the risk of getting infected.

First, don't download and install any software unless it comes from a site that you know and trust. This includes plugins, and virus detection software.

Some websites now do digital signing. The software vendor generates a public/ private key pair. A digital signature such as MD5 is calculated for each application, and encrypted with the vendor's private key. After the application has been downloaded, the user calculates the MD5 value on the downloaded software, and then uses the vendor's public key to decrypt the attached signature. If they are the same, the user knows that the software came from where it was supposed to, and not from a malicious intruder.

Second, don't do your everyday computing on an account that has administrative privileges (almost everyone ignores this one).

Third, don't run email attachments that could have malicious macros in them, such as word or excel.

Back up your files often so that if a virus is detected, you can reinstall the OS without losing your stuff.

Encapsulation

Lots of software now runs in a web browser. An example is java applets. Browsers have built-in protection schemes so that applets run in their own protected space and are not allowed to execute instructions outside of this space. This is called sandboxing. Often a program is given two blocks of memory, one for code and one for data. The code section is not permitted to be modified.

Firewalls

Firewalls have become an almost universal protection tool. A firewall is a process, or often a separate computer, that sits between the world and a network (or between two networks) and it only lets certain types of packets pass in and out. A firewall is configurable on a port basis so that you can exclude packets coming to particular ports. There are two basic schemes, they can contain a list of the types of packets they will accept and reject all others, or they can contain a list of the types of packets they will reject and accept all others. Configuring a firewall is a constant tradeoff between the ease of getting work done vs. the risk of an intrusion.

Firewalls offer good protection against one common sort of intrusion, port scanning The bad guy will run a program that systematically tests all of the ports on your system to see which ones accept connections, and whenever they find such a port, they try to intrude through that port.

Here is a link to a good website about firewalls

Orange Book Security

The Department of Defense is interested in computer security, and they have developed a set of security standards for computer systems called the Trusted Computer System Evaluation Criteria, more commonly known as the Orange book. The Orange Book has four layers of security, although some of these have sublayers.

Here is a link to a web site that describes these levels in more detail.

ssh and its relatives

Two of the most widely used application layer protocols on the Internet used to be telnet which allowed a user to remotely log into a computer, and ftp, the file transfer protocol, which allowed a user to download files. Both of these are now generally considered to be obsolete because they required the user to enter a password which was sent over the network unencrypted, and this was a major security loophole.

telnet and similar programs such as rlogin (remote login) and rsh (remote shell) have been replaced by ssh, the secure shell, which most of you have been using all semester. This protocol requires that all communication, including passwords, be encrypted. In some ways it is similar to PGP, discussed in the previous lesson.

There is an ssh client and server as you would expect. Here is a typical set of steps that ssh goes through in establishing a connection using the SSH1 protocol.

Protocols like scp (secure copy) work in more or less the same way.

The standard for secure communication for the World Wide Web is the Secure Socket Layer (SSL), developed by Netscape. This protocol provides data encryption, server authentication, message integrity, and optional client authentication for a TCP/IP connection. SSL is built into all the major browsers and web servers.

In order to activate SSL, a server has to obtain a digital certificate. This is obtained from a trusted Certificate Authority (CA). There are a number of such authorities such as VeriSign. Applicants for certificates have to provide extensive documentation to the CA confirming that they are who they say they are. This prevents a malicious web site from pretending to be a vendor such as Amazon in order to get credit card numbers or other information.

When a client (browser) connects to a server, before any information is exchanged, the server sends an authentication packet, containing a time stamp, the digital certificate, and other information, all encrypted with their private key. The client decrypts the packet with the server's public key, obtained from the CA, and confirms that the information is correct.

Often a client may also have a digital certificate. This allows a server to confirm that clients are who they say they are.

Once each side has authenticated the other side, they agree on an encyrption method, and a session key (exchanged encrypted), and begin communicating. All subsequent communication is encrypted.

Other types of security problems

Even the most secure operating systems are vulnerable with certain types of new technology. It is possible for a person to be sitting in a car outside your home or office and with a device that picks up the electomagnetic signals from your monitor. Certain types of (very expensive) equipment can decipher these signals to reconstruct everything that is on your screen. The Federal Government has a program called temptest to develop defenses against this. Here is a website about Tempest if you wish to learn more about it.

Your text describes a technique called steganography in which a secret message is encoded in an ordinary image. An image which is stored as a matrix of pixel data generally stores the data at a higher resolution than the human eye can detect. Steganography stores the message by overwriting the least significant bit of each pixel. If the red value of a particular pixel is supposed to be 100, but it is 99 instead, no human can detect the difference. A color image has three bytes for each pixel, one each for the red, green, and blue values, so a 1024 by 768 image can contain a secret message of up to 2.3M bits.

The Computer Emergency Response Team (CERT)

In 1988 a Cornell Grad student released a worm into the Internet. It brought down thousands of Unix computers all over the world. It exploited a buffer overflow bug in the finger daemon (a server which allowed users to locate email addresses on the system). One effect of this was the creation of The Computer Emergency Response Team (CERT). at Carnegie Mellon. This provides a central place to report breakins, viruses etc. It is staffed with security experts to track down security breaches.

Security and Authentication on Windows

Windows provides a uniform access control facility that applies to processes, threads, files, semaphores, and other objects. Access control is governed by two entities, an access token assocated with each process and a security descriptor associated with each object.

Users are authenticated with a typical password system. The login process creates an access token. This token is inherited by any processes which are created by this initial process. The access token contains the following information.

Whenever an object (file, process, thread, semaphore, etc) is created, a Security Descriptor is created for it. This contains the owner's SID, a set of flags defining the types of privileges, A System Access Control List (SACL) set by the system, and a Discretionary Access Control List (DACL) which determines which users and groups can access this object. Recall that one of the arguments to the CreateProcess API was a pointer to a security attributes structure. If this is NULL, then the security attributes associated with the new process is the default, but it is possible for the user to set different security attributes (within limits).

Kerberos

The gold standard for computer and network security is Kerberos. Kerberos was developed at MIT as part of Project Athena, which also developed the X Windows system. It is designed to provide complete security for services on a network. The name comes from the name of a three headed dog that guarded Hades in Greek mythology.

Networking services opens up the possibility of new kinds of security violations. One such violation is called the "Man in the Middle", in which the bad guy is not only able to intercept all communication between a client and server, but also attempts to pretend that his program is the server (or client) by changing the network address. For example, a legitimate client could contact a file server requesting a file, and the man in the middle could intercept this request, read the request, and perhaps send back a different file. The client assumes that it has contacted the file server, but in fact has not.

Similarly, the Man in the Middle could contact the file server pretending to be a legitimate client, and so the file server could send a file or files over the network, but they are intercepted by the bad guy. As above, the file server thinks that it has successfully sent the files to a legitimate client.

Recall that one of the problems that SSL fixed was the WWW version of Man in the Middle, where a browser thought that it was communicating with, say, Amazon.com, but in fact, all of the packets had been intercepted. SSL solved this problem by having a Certificate Authority whom everyone trusted who issued certificates.

Kerberos is a trusted third party authentication system because it requires the user to identify themselves every time a service is requested. It effectively prevents the man in the middle attack. Every user has a password and every service has a password, and the only entity that knows both is the authentication server. The authentication server operates under considerable physical security. Typically it runs on a separate machine for which only a few people have access.

Here is what happens when Alice logs into a Kerberos system and attempts to use a service.

Here are some observations about Kerberos.

In order to use a service, a ticket is required. All tickets except the first are obtained from TGS. The first is obtained from the authentication server.

The tickets are not comprehensible to the workstation because they are encrypted using the key of the service

Every ticket is associated with a session key that is assigned every time a ticket is allocated.

Tickets are reusable. They have lifetimes of about eight hours.

But a new authenticator is required for each request, and these expire within a few minutes.

The server maintains a list of all recent requests so that it can detect duplicate requests which might arise from a stolen ticket.

Since both ticket and authenticator contain the network address, another workstation cannot use a stolen ticket without changing their system to impersonate the owners network address (and they have to do this quickly because authenticators have a time stamp).

Note that Alice's password is never sent over the network, so it is impossible for the bad guys to steal passwords by network sniffing. Note also that the Man in the Middle attack is foiled, because if the request is interrupted, the bad guy cannot read it because it does not know the encryption key of the service, so it cannot pretend to be that service.

Kerberos is in some ways cumbersome. It should be installed for all system services, such as file servers, mail servers, and printers, as well as the login process. It obviously slows down almost any operation because of all of the encryption and decryption, and it also causes additional network traffic to obtain tickets. But the advantage is that all network traffic is encrypted, and this prevents a huge amount of security violations.

The Andrew File System (AFS) has Kerberos built in.

Kerberos is believed to be secure, with the huge caveat that if the security of the authenticator or other services is breached, all security is lost.

Here is a link to the Official Kerberos Web site at MIT

Return to the course home page