Chapter 14 Study Guide

Troubleshooting Methodology

• Proactive Maintenance

Taking the necessary steps to minimize future problems Includes performing system back-ups and identifying potential problem areas.

• Reactive Maintenance

Correcting problems when they arise Always document the solution to help quickly resolve future problems

Troubleshooting Procedures

• Gather as much information as possible

System log files Run information utilities such as ps or mount “tail –f /path/to/logfile” opens a log file for continuous viewing

• Isolate the problem

Determine if the problem is persistent, intermittent, and how many users are effected

• List possible causes and solutions

google is your best friend

• Implement and test solution
• Document your solution and process

• Prioritize problems

Solve most severe problems first Spending too much time on small problems can result in reduced productivity

• Try to solve the root of the problem

A short term solution might fail in the long term because of an underlying problem

Hardware Related Problems

• Can come from damaged hardware or improper hardware or software configuration
• Using the dmesg command or viewing the /var/log/boot.log and var/log/messages files can isolate most hardware problems
• The absence of or improper drivers prevents the OS from using the associated hardware

Use lsusb to view only usb devices Use lspci to view only PCI devices

• Lsmod command lists the drivers loaded into the kernel

By comparing the output of dmesg, lsusb, and lspci with the lsmod output, you can determine if a driver is missing

• Hard drives are the most common hardware component to fail

software Related Problems

• Can be application or OS related
• Application Related Problems

Can fail during execution due to missing program libraries and files, process restrictions, or conflicting applications Identify missing files in a package by using the –V option with the rpm command Use the ldd command to identify which shared libraries are required by certain programs It is good practice to run the ldconfig command to ensure the shared library directories are updated The ulimit command can be used to increase the number of processes the user can start in a shell

Operating System Related Problems

Typically include problems with X windows, boot loaders, and filesystems Use xwininfo or xdpyinfo commands to attemp to isolate problems with X windows Placing the word “linear” and removing “compact” from the /etc/lilo.conf file often fixes LILO boot loader problems The GRUB boot loader errors are typically the result of a missing file in the /boot directory File systems can become corrupted due to high use accessing the hard drive Corrupted filesystems can be identified by very slow write requests, errors printed to the console, or failure to mount

User Interface Related Problems

• Users need to understand how to use their desktop environment, but often will not
• Assistive technologies are tools you can use to modify your desktop experience

Accessed by opening the system menu and navigate to preferences, assistive technologies

Linux Performance

Monitor system performance using command-line included in the sysstat package. To make it easier to identify performance problems, a network administrator should run performance utilities on healthy Linux systems to develop a baseline.

Performance Problems:

• Software
• Hardware
• Combination of the two

Software Problems

Software that requires too many system resources may use CPU, memory, and peripheral devices creating poor performance. Too many processes running or rouge processes

Hardware Problems

Improperly configured hardware (May still Work) Old ( Most companies retire computer equipment after two to five years of use) Jabbering: Sending large amounts of information to the CPU when not in use.

Resolutions

Software problems can sometimes be resolved by changing hardware. Move or Remove the software Upgrading or adding another CPU Use bus mastering peripheral components (Devices that can perform processes normally performed by the CPU) Adding RAM to increase system speed Replace slower disk drives with faster ones Use disk striping RAID Keep CD and DVD drives on a separate hard disk controller

Monitoring Performance with sysstat Utilities

Using information from the /proc directory and system devices, the System Statistics (sysstat) package contains utilities that monitor the system. To install the latest version of sysstat on a Linux system, use the following method: 1. yum install sysstat

Three of the System Statistics (sysstat) package performance monitoring utilities include:

• mpstat (multiple processor statistics) command
• iostat (input/output statistics)
• sar (system activity reported) command

mpstat (multiple processor statistics)

Used to monitor CPU performance for all processors on the system since the system was started or rebooted.

To monitor a single cpu use the –P option followed by the processor number.

Example: mpstat –P 0 would display the first processor on the system. Limited in abilities

Examining the Output of the mpstat command

• %user= % of time the processor spent executing user programs and daemons
• %nice= % of time the processor spent executing programs and daemons that had nondefault nice values
• %sys= % of time the processor spent maintaining itself
• %iowait= % of time the CPU was idle when an outstanding disk I/O request existed.
• %irq= % of time the CPU is using to respond to normal interrupts that span multiple CPUs.
• %soft= % of time the CPU is using to respond to normal interrupts that span multiple CPUs
• %steal= % of time the CPU is waiting to respond to virtual CPU requests
• %guest= % of time the CPU is executing another virtual CPU
• %idle= % of time the CPU did not spend executing tasks. Should be greater than 25% over a long period of time.

iostat (input/output statistics)

Measusres the flow of information to and from disk devices. Displays CPU statistics similar to mpstat Limited in abilities Adds transfers per second (tps) and block

sar (system activity reporter)

Displays more information than the mpstat or iostat command Displays CPU statistics by default Most widely used performance monitoring tool on UNIX and Linux systems Scheduled using the cron daemon to run every 10 minutes for the current day

logged to a file in the /var/log/sa directory called sa#. The # represents the day of the month

Only one month of records is kept but can be changed by editing the cron table located at /etc/cron.d/sysstat Can display different statistics by specifying options sar (system activity reporter) Displays more information than the mpstat or iostat command Displays CPU statistics by default Most widely used performance monitoring tool on UNIX and Linux systems Scheduled using the cron daemon to run every 10 minutes for the current day

logged to a file in the /var/log/sa directory called sa#. The # represents the day of the month

Only one month of records is kept but can be changed by editing the cron table located at /etc/cron.d/sysstat Can display different statistics by specifying options

Common options with the sar caommand

• CPU Usage of ALL CPUs (sar -u) Default
• CPU Usage of Individual CPU or Core (sar -P)
• Memory Free and Used (sar -r)
• Display swapping statistics (sar -W)
• Reports run queue and load average (sar -q)
• sar -u 1 3 Displays real time CPU usage every 1 second for 3 times.

Other Performance Monitoring Utilities

• top utility (discussed in Chapter 9)
• free command can be used to display the total amounts of physical and swap memory (in Kilobytes) and their utilizations.
• vmstat command indicates more information than the free command to indicate whether more physical memory is required.

Security

Locking enviorment

Local Computer In GUI environment always lock screen when away from system In CMD enviorment, not as easy as exiting will end process, instead of exiting use nohup command to exit shell without terminating processes.

Root time minimization The root user should only be used in conditions where it is specically called for, this is done to limit the exposure to potential threats both internally and externally. Use the su command to elevate your permissions when needed

Root time minimization If only a single command is going to be run as root you can use the –c variable to modify to return to user after command is executed.

Network Protection The biggest part of protecting your system from network intrusion is to terminated unneeded services. Using a program like nmap will display what ports are currently open and available. These are stored in etc/services, ensure they are disabled in the runlevel when not needed. For Internet Super Daemon, need to add the disable = yes to the appropriate service.

Network Intrusion Prevention of the login using common users should have the shell listed in the /etc/passwd tied to /sbin/nologin. This will prevent attackers from logging into the system using these shells. It will display your preprogrammed message /etc/nologin.txt, if none exists a standard one will be displayed.

Don’t be a fool wrap your tool! TCP wrappers should be used as a tool to secure services started by the Internet Super Daemon. To use the wrapper modify /etc/zinetd.d and start the network daemon.

Host Allow vs. Deny When using the wrapper it will check against your Allow and Deny lists to ensure host connection

FIREWALL!! Arguably the most important part of a network environment is your firewall, be it operated in a hardware device or software. This is done using the netfiler/iptables Rules… rules… rules… INPUT: Coming into system FORWARD: Pass through system OUTPUT: Leaving from system with origination

Firewall setup Using the iptables command each of the arguments can be entered to determine network traffic.

SELinux Security Enhanced Linux, in newer versions in configured by default. Patches and utilities created by the NSA that is role based security. To enable SELinux edit /etc/selinux/config SELINUX = enforcing SELINUX = permissive SELINUX = disabled

Encryption Two most common on linux are SSH and GPG (GNU Privacy Guard) RSA to encrypt data DSA to sign data Keys are stored in; ssh_host_dsa_key(.pub) ssh_host_rsa_key(.pub) (.pub) on end is the public key

Intrusion Detection' When running services regularly check log files in /var/log for associated services PAM (pluggable Authentication Modules), logs information into /var/log/secure for errors and alerts Open files can be vied using lsof (list of open files)