HP OpenVMS Guide to System Security

Security for the System Administrator

System Security Breaches

Routine System Surveillance

Securing a Cluster

Handling a Security Breach  

1. Detection of a problem
2. Identification of the perpetrator
3. Prevention of further security violations
4. Repair of damage

The following sections describe these phases for both attempted and successful break-ins.

In all phases, train personnel to retain information and data as evidence, should there be a need to apprehend and prosecute the perpetrator.

Unsuccessful Intrusion Attempts  

Unsuccessful intrusion attempts include situations where someone has attempted to guess passwords or browse through files.

Detecting Intrusion Attempts  

You usually detect intrusion attempts through the following sources:

• Reports from users about unexplained login failures



• Unusual system activity or unavailability of dialup lines



• Security alarms for login failures, break-in attempts, and file-protection violations



• Examination of the intrusion database

Identifying the Perpetrator  

Enabling file auditing simplifies identification of file browsers. If, however, browsing is being initiated from another node in the network, you must inspect the network server log file (NETSERVER.LOG) that corresponds to the times of the protection violations. Coordinate your investigation with the security administrator at the remote node.

Identifying a perpetrator who is guessing passwords is considerably more difficult, especially when the source is anonymous, as from a dialup line. Usually, you must trade identification for prevention. Often the only way to positively identify an outsider attempting to enter the system requires that you permit further attempts while establishing the perpetrator's identity.

Preventing Intrusion Attempts  

The prevention phase for this kind of attack involves preventing the would-be intruder from actually gaining access to the system and making future attempts more difficult.

Password Guessing

To reduce the opportunities for successful password guessing:

• Make certain your users choose appropriate passwords. Consider use of the password generator (see Generated Passwords).



• Enable system passwords at the points of entry. While a minor inconvenience to your users, system passwords are the best protection against further probing. If you already had a system password enabled, change it (see System Passwords).



• Enable auditing of successful logins to catch the event if the intruder succeeds in getting in (see Security Auditing).

File Browsing

To reduce the opportunities for successful file browsing:

• If you can identify the perpetrator, take action as established at your site.



• Warn your users about the importance of adequate protection of their files, and consider inspecting the protection of user files.



• If file browsing from other nodes in the network becomes a persistent problem, eliminate the default FAL account and authorize individual users through proxy login accounts (see Setting Up a Proxy Database).

Successful Intrusions  

A successful security breach can include a successful password guessing scheme, theft or modification of either information or system resources, and placement of damaging software on the system. An intrusion may require a considerable amount of time to repair, depending upon the skill and intent of the perpetrator.

Identifying the Successful Perpetrator  

Identification is often the most difficult part of handling an intrusion. First, you must establish whether the perpetrator is an authorized user or not. This determines the nature of the preventive measures that you will take. However, the distinction between insiders and outsiders may be difficult to achieve.

Tradeoff Between Identification and Prevention

You may have to make a tradeoff between a positive identification of the intruder and preventing future attacks. Often, the data available initially does not allow complete identification. If it is important to identify the perpetrator, you will often find it necessary to permit continued intrusions while you analyze the intrusion activity. Increase your auditing. Consider planting traps in system procedures that are under your control (such as SYLOGIN.COM) to obtain additional information. Increase your system backup efforts to permit easier recovery if files become damaged.

Identification of Outsiders

Identifying external intruders is particularly difficult, especially if they use any switched forms of communication (such as dialup lines or public data networks). DECnet for OpenVMS software provides many features to help you trace the activity through the network back to the source node. If a local terminal is involved, physical surveillance may be appropriate.

When a switched connection is involved, one of the major computer security problems is the telephone system itself. Tracing a telephone or public data network connection is time-consuming. Chasing an intruder through the telephone system is likely to take months and will require the assistance of law enforcement authorities. The existence of multiple long-distance telephone services compounds the problem by increasing the number of organizations with whom you must deal.

As a result, identifying an outside intruder is usually worthwhile only when you have sustained substantial financial damage. In many cases, it may be more useful if you concentrate on preventing recurrences of the problem.

Securing the System  

The actions you must take to secure your system after an intrusion depend on the nature and source of that intrusion. This section describes these actions in order of priority.

1. Restore SYSUAF.DAT, NETPROXY.DAT, NET$PROXY.DAT and RIGHTSLIST.DAT (if damaged) from backups. Alternatively, generate listings of the files and inspect them closely, looking for improper entries, additional privileges, and changed UICs. If you are unsure of when SYSUAF.DAT might first have been modified, inspect it carefully regardless of whether you are using a backup copy or proceeding with the existing one. Be sure all authorization files are secure.
2. The perpetrator may have discovered passwords by browsing either through files or from other nodes in the network and may be using seldom accessed accounts for personal use. Change passwords for accounts, and have your users appear in person to learn their new passwords. At a minimum, change passwords on all privileged accounts. Do not use the same new password for all accounts.
3. A sophisticated penetrator may have planted ways to provide future access to the system even though you have taken the obvious steps of securing your system. Therefore, you may have to restore selected components of the OpenVMS software from backups or from your OpenVMS distribution kit. If the intruder was an outsider, the two critical components are LOGINOUT.EXE and NETACP.EXE, which validate all entries to the system.
   
   However, if the intruder was an authorized user, restore all system files from backup copies. Authorized users can make use of a wide variety of illicit software patches (called trap doors ) that they insert in the executive (SYS.EXE), the file system (F11BXQP.EXE), DCL, and other system files. The penetrator may have planted damaging software in any piece of software or command procedure likely to be used by a privileged user. Thus, complete assurance of a secure system requires a wholesale restoration of files from backups. Also reinstall any image (even from layered products) installed with privileges because it can also be used for a trap door. An alternate strategy is to restore trustworthy copies of the obvious targets of attack and to rely on increased auditing for a period of time to catch suspicious events.
4. Consider implementing additional security features, such as system passwords, password generation, increased auditing, and more stringent file protection to prevent a recurrence.

Repair After a Successful Intrusion  

After an intrusion, restore corrupted files. Decide whether it is appropriate either to do a wholesale restoration of your system's data or to repair problems as they are discovered. Look for modifications to file protection that would have created paths for viruses and for Trojan horses that were introduced into the system and may still reside there.

Routine System Surveillance

Securing a Cluster