Useful Linux Commands for NOC Work

Abstract:

This page is intended to be a somewhat-loose flow of instructions which will serve as both a lesson when taken in order and as a
searchable reference.
NOC's Linux sandbox (aus-noc-dev.trionworld.priv) can be reached with PuTTY (http://www.putty.org/).

Objective/Goal:

Provide a fully funtional introduction to performing Tier 1 NOC procedures in a Linux environment.

Keywords:

unix admin administration bash shell rhel centos ubuntu debian

This document is a:

Reference

Learning and applying these instructions will require the use of:

Your standard AD credentials should provide access; otherwise, Linux Engineering can set you up.

Some learning lab advise:

1. Type your command but do not press Enter.

2. Take your hands far away from your keyboard.
3. Picture in your mind what you expect the command will do.
4. Press the Enter key then observe the results.

Caution:

In DOS, the Find command works like the Grep command in Linux. However, Find in Linux serves a different purpose and it is a
far-reaching and dangerous command. It should not be used without a specific purpose in mind and only with intentions and results
tested on a safe server.

Instruction Body:

The basics:

Copy from your Linux session: Use the mouse to highlight the text you want to copy. Text will automatically go to the clipboard.
Paste to your Linux session: Right-click in the Linux window. Items on the clipboard will insert at the cursor position.

Ctrl+c // Cancel current action. Some actions require a few repeated cancels to generate an interrupt.

ls -l // Shows the files in the current directory.

unalias command // Turn off certain default system features.
Also see the Unalias Supplement page.
EX: unalias ls // This will cause all screen text shown during an ls of any sort to be black and
white.
EX: unalias vi // This will likewise cause special coloring to be removed, but during vi editing
sessions.

pwd // Shows your current directory.

Tab-complete // You can partially type something and press tab to resolve the remaining text.

mkdir folderName // Make a new subdirectory from your current location.

touch fileName // Quickly make an empty file.

cd folderName // Change directory to a folder beneath current directory.

cd .. // Move up one directory level.

cd ~ // Move to current user's home directory.

cd /etc/httpd/conf // Move several levels ('/' aka "slash" or "root" through 'etc/' through 'httpd/' to 'conf') in one
command.

man subject // Pull up the manual for a subject.

apropos subject // Search through a brief entry of subjects based on keywords.

EX: apropos net | grep -i snmp | grep -i nic

hostname // Shows what server you are current working in.

ssh serverName //Move to another server.

EX: ssh rwc-dev-ch1.trionworld.priv
EX: ssh rwc-dev-ch1

ssh userName Move to another server using another name.

@_serverName EX: jpierce-admin@10.16.3.84

whoami // Shows what user you are currently working as.

sudo su userName // Change to another user.

sudo su - // Become the root user. Also "sudo su - root".

sudo commandLine // Regardless of current user, function with admin rights.

EX: sudo vi inventoryPlayerDisconnects.script rift eu

clear // Clear the current screen.

cursor-up/-down // Move through the last commands used as stored in history.

history // Display a list of commands used.

uptime // See how long it has been since the last bootup.

who // See who is logged into the server.

last // See who was previously logged into the server.

type file // Show a quick blurb about a 'file'. Note that in Unix-based systems everything is a file.
EX: type php
EX: type vi
EX: type ls
EX: type alias

host ipNumber // Find the hostname based on the IPv4 number given.

nslookup ipNumber // Find the arpa name (FQDN) based on the IP address given.

nslookup ipAddress // Find the IPv4 address based on the host name given.

ifconfig // May have to be root (sudo su -) to do this. Shows ip configuration info.

netstat // Displays active connections.

tracert ipAddress // Perform a trace route.

 mtr ipAddress // Perform a My Traceroute.
EX: mtr 216.81.59.173 Run this exact MTR command and enjoy the results!

File commands:

sudo chmod // '###' is where the numeric arguments are inserted. Change the permissions of a file so that certain
### groups,
users or everyone else may read, write and/or execute the file. Some numeric argument examples are:
764 = User can read, write and execute (7); user group can read and write (6); everyone else can read (4).
500 = User can read and execute only (5); everyone else is denied read, write and execution (00).
710 = User can read, write and execute (7); user group can execute only (1); everyone else is denied (0).

cat fileName // Display the contents of a file.

EX: cat log.file | grep -i error

less fileName // Search through a file. Move down the lines with the spacebar and up the lines with the b key.

cp fileName // Create a duplicate file.

newFile

mv fileName // Rename the file. This does not preserve a duplicate file.
fileRenamed

mv fileName // Move the file into the named directory location.

subDirectory/

mv -i fileName .. // Move the file up one directory level.

rm fileName // Deletes a file. In Linux, there is no recycle bin – deleted files are immediately wiped away.

vi fileName // Create or edit a file using the vi editor. This is a huge subject and so only an introduction is mentioned
here. Most of what follows is done in Command Mode.

(in vi) i // Enter Insert Mode in vi to begin typing. This allows plain text style writing, copying and pasting.

(in vi) Esc // Escape key returns to Command Mode.

(in vi) u // Undo the last command(s) or edit(s). Similar to Ctrl+Z in Windows.

(in vi) w // Save the file. You must have write permissions.

(in vi) w newFile // Save the file as a copy with a new name.

(in vi) q // Quit out of the vi editor. wq will save and quit.

(in vi) q! // Quit without saving.

Install new software:

sudo su - // Assume root privileges.

yum install softwareName
y
EX: yum install facter

Return system information with 'facter' and 'DMI Decode':

facter
facter | grep up // This returns the various uptime info.
facter | egrep -i 'mem|har' // This returns memory and hardware info.
dmidecode // returns a very long list of hardware information.
dmidecode | less
dmidecode | grep -ih -8 processor | less // returns eight lines following every successful location of the word 'processor'.
dmidecode | egrep -ih -4 'vendor|product'

Find processor information with 'cat':

cat /proc/cpuinfo // Returns an entry for every processor.
cat /proc/cpuinfo | egrep -i 'proc|model' // Reduces the list to only show the processor ID ("0", "1", etc.) and the vendor's
model information.

Check running processes with 'top':

*There are many options that can be used with Top. You can either enter the option on the same command as calling top ("top -c")
or toggle the options on and off while Top is running by typing the letter, as shown below. The later has many more options, which
are called 'hotkeys'.
top
c // Option 'c' will show the command used to call the processes.
A // Splits the display into consumption groups.
f // Shows a menu of hotkey options.

Check running processes with 'ps aux':

ps aux
EX: ps aux | grep -i pierce // Return all processes being run by 'pierce' or by 'Pierce'.
EX: ps aux | grep -i pierce | grep -v grep // Same command but exclude the grep command
itself from the return.
EX: ps auxf | sort -nr -k 3 | head -20 // Return the top 20 CPU consumers.

For much more on processes, see Processes Supplement - Stopping, Starting, Restarting, Tracing

Check disk consumption:

df -ha // Displays all the drives, and their size info.

du -h driveName // Displays where consumption is occurring most.
EX: du -h /home/jpierce/ // Displays storage consumption in my home directory. You can get the same result in this case with
"du -h ~".
EX: *du -h ** // This shows a huge list if you are at a top level, so it is best to use it in a regex expression, such as:
EX: du -h * | less
EX: du -h * | grep M | egrep 'log|out' | grep -v grep //Search for files (and folders) with the words 'log' or 'out' (as in output) that
are at least a meg in size.
EX: du -hs * | grep G | head -n 40 // This is my personal favorite command for drilling down to the source of a heavy directory
or file. Start at the top of the suspected branch and use this to see where to go next. Go there and repeat as needed until you have
drilled enough.
find * -mmin -30 | grep log // Search for log related material that was updated within the last 30 minutes. Or "-mmin 30" finds
them from exactly 30 minutes ago. And "-mtime -1" finds them from up to one day ago.
EX: find * -mtime -7 | grep log | grep -i access // Find items from the last seven days.

Detect and resolve file system in a read-only state:

In the case of an unusually large event (like the domain's DNS system crashes) a Linux server's file system can become
vulnerable to corruption. As a means of avoiding corruption, every file on the server becomes 'read only', meaning not even the
operating system will be able to write to it or save things. NOC can quickly detect this problem and return the server from its 'safe
mode' status using the following sequence of commands.
 1. ssh root@serverName // SSH into the server as root. If you suspect that the server is in a read-only state and the SSH
command freezes with no errors or successful connections, then the server is in worse shape than this instruction can
help with --probably its in a 'kernel panic' state that requires Linux Engineering to resolve.
2. cat /var/log/messages | tail -n 15 // 1st check. Read the last several log entries of 'messages' to see if a phrase
including "..aborted.." is present. Note this in the ticket.
3. touch /tmp/deleteMe // 2nd check. Since you are logged in as root, there should be nothing stopping you from creating
a simple file with Touch. If you receive an error stating "Read-only file system" then you will have to restart the server. If
the system lets you create the file then the server is not in a read-only state and troubleshooting for that state is complete
--don't do a restart.
4. shutdown -r now and wait several seconds // This restarts the server in a clean, safe way and should return it to
normal running condition rather than the read-only state. After 10 seconds or so, if the server is still active and NOT
FROZEN, then the restart is wonky and you will have to use the more aggressive restart command. EX: reboot

Authored by: James Pierce

Other expert: Linux Engineering