Rootkits

Brent Boe
Vasanthanag Vasili

Rootkits: What is a Rootkit?

A

rootkit is a set of tools used for (covertly)

maintaining root access to a system
Rootkits allow attackers the ability to
circumvent protection mechanisms limiting
root access
Provide a much higher layer of stealth than
normal Trojan programs by hiding
processes and files

Rootkits: What is NOT a

Rootkit?
A

rootkit is not an exploit used to gain root

access

Rootkits can only work if the attacker can gain

administrative access

(Typical)

Attacker sequence of events

Locate vulnerability on target host

Run exploit to gain root access
Install Rootkit
Remove Evidence
Locate next vulnerable host

Rootkit Functionality

Maintain Access

Destroy evidence

Disable shell history (e.g. Linux - unset HISTFILE; export HISTFILE=/dev/null)

Kill syslog daemon and freeze the system log
Modify log files

Attack other systems

SSH (is for script kiddies)

Reverse shell (a bit unusual if servers initiate connections)
Covert channel backdoor a signal system buried in an arbitrary field of a
completely innocuous protocol.

Local attack tools - Password Cracking, Capture root and access and obtain
access to machines
Remote attack tools - Scanners and Autorooters
DOS tools Conduct DOS attack on remote server

Clean the host system of previous infections

More than one rootkit can cause system instability and compromise the rootkit

What does a Rootkit hide?

The

attackers files
The attackers processes (eg: sniffers, PW
crackers)
The attackers user account
Unusual environment variables (network
cards in promiscuous mode)
Specific network connections to and from
compromised machines

Necessary Background

The Kernel Space is more privileged than the User

Space
The lower a rootkit can go, the more likely it is to
avoid detection and defeat Host Intrusion Prevention
Systems

User Space
Kernel Space

Necessary Background

The Intel x86 based chips use rings for access

control with Ring 0 being the most permissive and
Ring 3 being the most restrictive
User programs run in
Ring 3
Kernel programs run in
Ring 0
Ring 0
Rings 1 and 2 are
unused

Types of Rootkits

Binary Rootkits
Kernel Rootkits
System call Rootkits
Library Rootkits
Virtual Machine
Rootkits
Database Rootkits
Runtime Kernel
Patches

User Space
Kernel Space
Kernel Space
User Space
Kernel Space and User
Space
User Space
Kernel Space

Binary Rootkits

These rootkits are collections of subverted popular

system binaries (or executables).
Trojaned to perform action conducive to attacker (eg:
hide malicious process)
Binary files usually precompiled for particular platform for
user to choose & utilize correct one
Attacker deploys kit after breaking In via installation
script which places binaries over original ones & saves
old copies
On Linux, the attacker may choose to directly modify the
source code on the target machine and recompile the
binary.

Some trojaned binaries:

inetd,

rlogin, rshd, sendmail, sshd, telnetd

may contain magic password that provide
access to attacker for remote access.
ps to hide processes from causal viewing by
system admin.
netstat provides connection hiding
ls, dir provide file hiding
login,su,ping provide local access

Binary Rootkit Detection

Before the system is infected, compute the

checksums of the binaries

CRC checksums
Cryptographic checksums
Better to store the checksums on separate media (i.e. CDROM) so an advanced attacker cannot modify the files
In practice, if a file (legitimately) changes frequently, this
may lead to frequent checksum recomputations and false
positives.

Checksum computation is used by the program

Tripwire

Kernel Rootkits
First

reported in 1997
Loadable Kernel Modules hook into system
kernel and modifies selected sys_call
addresses stored in the system call table
Replaces the addresses of the legitimate
sys_calls with the addresses of the sys_calls
that are to be installed by the hackers LKM
Eg: KNARK ( targeting Linux2.2 Kernel)

Kernel Rootkits
Use

Loadable Kernel Modules (LKMs) for

Linux or Device Drivers for Windows
Full kernel access

User Space
Kernel Space

LKM

Kernel rootkit redirecting the system call

table
Redirects

the references to system call table

to new location.
New system call table is installed in new loc.
New system call table contains the address
of malicious sys_call functons
Redirecting can be done by overwriting the
pointer to the original system call table with
the address of a new system call table that is
created by the hacker

Kernel Rootkit Detection

Look for strange/inappropriate modules/device

drivers

Prevent LKMs altogether by disallowing module

loading

Keep in mind the binaries that would help examine this

information may be compromised too.
/lib/modules

Sometimes a compile time option

StMichael

Monitors various portions of the kernel for modifications.

When rootkit activity is detected, attempts to restore to a
previous good state

Necessary Background
When

a process wants to communicate with

the kernel it uses the system call table
The process throws a specific interrupt to
pass control to the kernel

Windows push the index of the system call in

eax. Throw interrupt x2e
Linux push the index of the system call into eax.
Throw interrupt x80

System Call and Library

Rootkits
Replaces

the standard system library for

relaying kernel information to a user process
The user library (libc) provides an interface to
the system call table.
The advantage no binaries need to change
Duplicate LKM functionality without entering
the kernel space
Very easy to hide processes and files
T0rn8 kit most prominent one

System Call and Library

Rootkit Detection
System

calls like truss, strace, and ltrace can

be used to trace the execution path of the
system calls
Some integrity tools generate checksums
against the system call tables.

Virtual Machine Based

Rootkits (VMBR)

A VMBR moves the targeted system into virtual machine.

Instead of moving the attack code lower into the kernel
space, it pushes the user higher into the user space
The previous (unhooked) OS runs over a virtual machine
(as the guest software)
The guest is not allowed to interact with states outside of
its Virtual Machine

The attacker has the liberty to run anything on the machine

Any anti-rootkit software run inside of the virtual machine will not
detect any modifications to its state

Steps of VMBR installation

Modify the Boot Sequence to load the Virtual Machine

Monitor (VMM) first

Modify it after shutdown after all monitoring processes have exited.

Interfere with the disk controllers write so that only the

rootkit can store disk blocks

Working at this low level to avoid interference with monitoring

software

Overwrite the master boot record so the VMBR loads first

Reboot and
The target system is now running as a guest, you can
interfere with them, but they cant interfere with you

VM Rootkit Detection

Detecting a VM rootkit can be quite difficult (from the

inside of the guest software)
Possible to detect a rootkit using instructions that
reveal information about the kernel state (or the
emulated kernel state)

redpill uses the sidt instruction to store the interrupt

descriptor table register. Since the VMM needs to move
the emulated interrupt descriptor table, the ITDR will begin
at a much higher address then it normally would.

Easiest way to detect a VM rootkit; boot from an

alternate media.

Database Rootkits
A

database can be considered a type of

operating system

Users
Processes
Executables
Jobs
Symbolic Links

Database Rootkits

1st Generation Rootkits

Change the data dictionary (modify a view, procedure, and
change synonyms)

For example, change ALL_USERS to be select * from sys.user$ u

where u.name != HACKER;

2nd Generation Rootkits

Change the binary of the database so that all sys.user$
variables become sys.aser$
Remove the Hacker entry from sys.user$
The system is now using sys.aser$ internally, but all integrity
checks use sys.user$
3rd Generation Rootkits
For Oracle, Direct SGA (System Global Area) Manipulation
directly modify the contents of the database through modifying
the memory the database is stored in

Database Rootkit Detection

Examine

the internal views for obvious

changes
Examine the internal system variables for any
changes or new, unrecognized variables

Runtime Kernel Patching

Modifying the memory of the kernel while it resides

in memory.
Simply modify a few bytes here, a FAR JMP there to
execute the rootkit code, and youre done.
A technique called detour patching totally that can
totally circumvent executing code by modifying the
control flow at runtime
Very difficult to detect
Very difficult to pull off successfully

Need extremely specific details about the target machine

General Rootkit Detection

Behavioral Detection

Signature Detection search for unique byte patterns

Can be defeated through code obfuscation techniques

System Integrity Scans

Look for suspect behaviors, such as writes to the memory

containing important system call tables
Look for a change in the number, order, and frequency of
calls

Scan the kernel for inappropriate FAR JMP instructions

Detect unauthorized changes to loaded OS components in
memory

Offline analysis of drives

Sony BMG Rootkit Scandal

Sony BMG Music Entertainment was sued in 2005 for

surreptitious distribution of rootkit software on audio compact
discs.
It used a software called Extended Copy Protection (XCP)
designed to help prevent unlimited copying and unauthorized
redistribution of the music on the disc.
XCP interferes with the normal way in which the Microsoft
windows OS plays CDs
This causes the system vulnerable to malicious code
CD ROMS were inoperable due to the change in the registry
settings caused by the software

Conclusion

Many rootkits practice offense in depth, and are by

no means limited to only one of the techniques listed
here.
Control of a system is determined by who can
operate closer to hardware, or in the case of equal
activity levels, who can best predict the actions of
the other
The best way to fight rootkits is to prevent them from
getting on your system in the first place Intrusion
Detection Systems, Host Intrusion Prevention
Systems.

References

Beck, M et al. Linux Kernel Programming. 3rd ed. London: Addison Wesley,
2002.
Cesare, Silvio. Runtime Kernel Patching. 03 Mar 2007.
< http://www.uebi.net/silvio/runtime-kernel-kmem-patching.txt >
Chuvakin, Anton. An Overview of Unix Rootkits. iDefense Labs: Feb 2003.
< www.rootsecure.net/content/downloads/pdf/unix_rootkits_overview.pdf >
Hoglund, Greg, Jamie Butler. Rootkits: Subverting the Windows Kernel. Addison
Wesley Professional: Upper Saddle River, NJ, 22 July 2005.
King, Samuel T. et al. SubVirt: Implementing malware with virtual machines.
Mar 01
2007. < www.eecs.umich.edu/virtual/papers/king06.pdf >
Kornbrust, Alexander. Oracle Rootkits 2.0. Black Hat 2006 USA, Las Vegas,
NV. 02
Aug 06. < http://www.red-database-security.com/wp/oracle_rootkits_2.0.pdf >

References

Levine, John G. et al. A Methodology to Characterize Kernel Level Rootkits

Exploits
that Overwrite the System Call Table. IEEE. 2004.
<http://ieeexplore.ieee.org/iel5/9051/28706/01287894.pdf >
Locally checks for signs of a rootkit. 01 Mar 2007. 28 Feb 2007.
<http://www.chkrootkit.org/>
Red-database-Security in the news/press. 23 Jan 2007. Red-Database-Security
GmbH. 1 Mar 2007.
< http://www.red-database-security.com/wp/db_rootkits >
Rootkit. 5 March 2007. Wikimedia Foundation Inc.26 Feb 2007.
<http://en.wikipedia.org/wiki/Rootkit>
Rootkits how to combat them. 1996 - 2007. Kaspersky lab. 29 Feb 2007.
<http://www.viruslist.com>
What is a rootkit? . 2 Mar 2007.
<http://www.tech-faq.com/rootkit.shtml>
Zaytsev, Oleg. Rootkits, Spyware/Adware, Keyloggers and Backdoors: Detection
and
Neutralization. A-List Publishing, Sep 1 2006.