6 Stages of Linux Boot Process (Startup Sequence)

by Ramesh Natarajan on February 7, 2011 Press the power button on your system, and after few moments you see the Linux login prompt. Have you ever wondered what happens behind the scenes from the time you press the power button until the Linux login prompt appears? The following are the 6 high level stages of a typical Linux boot process.

1. BIOS

BIOS stands for Basic Input/Output System Performs some system integrity checks Searches, loads, and executes the boot loader program. It looks for boot loader in floppy, cd-rom, or hard drive. You can press a key (typically F12 of F2, but it depends on your system) during the BIOS startup to change the boot sequence. Once the boot loader program is detected and loaded into the memory, BIOS gives the control to it. So, in simple terms BIOS loads and executes the MBR boot loader.

2. MBR

MBR stands for Master Boot Record. It is located in the 1st sector of the bootable disk. Typically /dev/hda, or /dev/sda MBR is less than 512 bytes in size. This has three components 1) primary boot loader info in 1st 446 bytes 2) partition table info in next 64 bytes 3) mbr validation check in last 2 bytes. It contains information about GRUB (or LILO in old systems). So, in simple terms MBR loads and executes the GRUB boot loader.

3. GRUB

GRUB stands for Grand Unified Bootloader. If you have multiple kernel images installed on your system, you can choose which one to be executed. GRUB displays a splash screen, waits for few seconds, if you dont enter anything, it loads the default kernel image as specified in the grub configuration file. GRUB has the knowledge of the filesystem (the older Linux loader LILO didnt understand filesystem). Grub configuration file is /boot/grub/grub.conf (/etc/grub.conf is a link to this). The following is sample grub.conf of CentOS.
#boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/boot/grub/splash.xpm.gz hiddenmenu title CentOS (2.6.18-194.el5PAE) root (hd0,0) kernel /boot/vmlinuz-2.6.18-194.el5PAE ro root=LABEL=/ initrd /boot/initrd-2.6.18-194.el5PAE.img

As you notice from the above info, it contains kernel and initrd image. So, in simple terms GRUB just loads and executes Kernel and initrd images.

4. Kernel

Mounts the root file system as specified in the root= in grub.conf Kernel executes the /sbin/init program Since init was the 1st program to be executed by Linux Kernel, it has the process id (PID) of 1. Do a ps -ef | grep init and check the pid. initrd stands for Initial RAM Disk. initrd is used by kernel as temporary root file system until kernel is booted and the real root file system is mounted. It also contains necessary drivers compiled inside, which helps it to access the hard drive partitions, and other hardware.

5. Init

Looks at the /etc/inittab file to decide the Linux run level. Following are the available run levels o 0 halt o 1 Single user mode o 2 Multiuser, without NFS o 3 Full multiuser mode o 4 unused o 5 X11 o 6 reboot Init identifies the default initlevel from /etc/inittab and uses that to load all appropriate program. Execute grep initdefault /etc/inittab on your system to identify the default run level If you want to get into trouble, you can set the default run level to 0 or 6. Since you know what 0 and 6 means, probably you might not do that. Typically you would set the default run level to either 3 or 5.

6. Runlevel programs

When the Linux system is booting up, you might see various services getting started. For example, it might say starting sendmail . OK. Those are the runlevel programs, executed from the run level directory as defined by your run level. Depending on your default init level setting, the system will execute the programs from one of the following directories. o Run level 0 /etc/rc.d/rc0.d/ o Run level 1 /etc/rc.d/rc1.d/ o Run level 2 /etc/rc.d/rc2.d/ o Run level 3 /etc/rc.d/rc3.d/ o Run level 4 /etc/rc.d/rc4.d/ o Run level 5 /etc/rc.d/rc5.d/ o Run level 6 /etc/rc.d/rc6.d/ Please note that there are also symbolic links available for these directory under /etc directly. So, /etc/rc0.d is linked to /etc/rc.d/rc0.d. Under the /etc/rc.d/rc*.d/ directories, you would see programs that start with S and K. Programs starts with S are used during startup. S for startup. Programs starts with K are used during shutdown. K for kill. There are numbers right next to S and K in the program names. Those are the sequence number in which the programs should be started or killed. For example, S12syslog is to start the syslog deamon, which has the sequence number of 12. S80sendmail is to start the sendmail daemon, which has the sequence number of 80. So, syslog program will be started before sendmail.

There you have it. That is what happens during the Linux boot process.

Linux Questions: Q. Where are the file names stored on a file system? A. The actual file names are stored in the directory file they reside in. No surprise here as pretty much everything on Linux is a file. On most Linux distributions you can either cat or edit the directory name to see the file names that are within it, e.g cat /home, or vi /home Q. What is a Signal in Linux, and what signal is invoked when you use the kill command? What is the difference between kill and kill -9? A. A signal is a limited form of inter-process communication used in Unix, Unix-like, and other POSIXcompliant operating systems. It is an asynchronous notification sent to a process or to a specific thread within the same process in order to notify it of an event that occurred. When a signal is sent, the operating system interrupts the target process's normal flow of execution. The difference between invoking kill with no signal specified (which uses SIGTERM, number 15) and kill 9 is that the latter tries to kill the process without consideration to open files and resources in use. Q. What is a process? A. A process is an instance of an executing program. When a program is executed, the kernel loads the code of the program into virtual memory, allocates space for program variables, and sets up kernel bookkeeping data structures to record various information (such as process ID, termination status, user IDs, and group IDs) about the process. From a kernel point of view, processes are the entities among which the kernel must share the various resources of the computer. Q. What are the logically divided parts of a process? A. A process is logically divided into the following parts, known as segments: * Text: the read-only machine-language instructions of the program run by the process. * Data: initialized/uninitialized global and static variables used by the program; * Heap: an area from which memory (for variables) can be dynamically allocated at run time. The top end of the heap is called the program break; * Stack: a piece of memory that grows and shrinks as functions are called and return and that is used to allocate storage for local variables and function call linkage information; Q. What are the process states in Linux? A. Running: Process is either running or ready to run * Interruptible: a Blocked state of a process and waiting for an event or signal from another process * Uninterpretable: a blocked state. Process waits for a hardware condition and cannot handle any signal * Stopped: Process is stopped or halted and can be restarted by some other process * Zombie: process terminated, but information is still there in the process table. Q. What is a Socket? A. A Socket is the combination of an IP address and a port number .Based on this combination, internet sockets deliver incoming data packets to the appropriate application process or thread. Q. How do you debug a running process or a library that is being called? A. strace -p PID ltrace libraryfile Q. How to see a memory map of a process, along with how much memory a process uses? A. pmap -x PID Q. You run chmod -x /bin/chmod, how do you make chmod executable again without copying it or restoring from backup?

A. On Linux, when you execute an ELF executable, the kernel does some mapping and then hands the rest of process setup off to ld.so(1), which is treated somewhat like a (hardware backed) interpreter for ELF files, much like /bin/sh interprets shell scripts, perl interprets perl scripts, etc. And just like you can invoke a shell script without the executable bit via /bin/sh your_script, you can do: /lib64/ld-linux-x86-64.so.2 /bin/chmod +x /bin/chmod

Q. Explain the TIME_WAIT state in a TCP connection, as displayed by netstat or ss. A. A TCP connection is specified by the tuple (source IP, source port, destination IP, destination port). The reason why there is a TIME_WAIT state following session shutdown is because there may still be live packets out in the network on its way to you. If you were to re-create that same tuple and one of those packets show up, it would be treated as a valid packet for your connection (and probably cause an error due to sequencing). So the TIME_WAIT time is generally set to double the packets maximum age. This value is the maximum age your packets will be allowed to get to before the network discards them. That guarantees that, before your allowed to create a connection with the same tuple, all the packets belonging to previous incarnations of that tuple will be dead. That generally dictates the minimum value you should use. The maximum packet age is dictated by network properties, an example being satellite lifetimes are higher than LAN lifetimes since the packets have much further to go. Q. What is Huge Pages in Linux and what use is there for them? A. Hugepages is a mechanism that allows the Linux kernel to utilize the multiple page size capabilities of modern hardware architectures. Linux uses pages as the basic unit of memory, where physical memory is partitioned and accessed using the basic page unit. The default page size is 4096 Bytes in the x86 architecture. Hugepages allows large amounts of memory to be utilized with a reduced overhead. To check: cat /proc/sys/vm/nr_hugepages. To set: echo 5 > /proc/sys/vm/nr_hugepages Q. What is a Master boot Record and how do you back it up and restore it? A. The MBR is a 512 byte segment on the very first sector of your hard drive composed of three parts: 1) the boot code which is 446 bytes long, 2) the partiton table which is 64 bytes long, and 3) the boot code signature which is 2 bytes long. To backup: dd if=/dev/sda of=/tmp/mbr.img_backup bs=512 count=1 To restore: dd if=/tmp/mbr.img of=/dev/sda bs=512 count=1 Q. Your server is using a lot of cached memory. How do you free it up short of rebooting? A. Kernels 2.6.16 and newer provide a mechanism to have the kernel drop the page cache and/or inode and dentry caches on command, which can help free up a lot of memory. To free page cache, dentries and inodes: echo 3 > /proc/sys/vm/drop_caches Q. How do you track new concurrent connections? A. Concurrent connections are the number of authenticated "handshakes" between a client and/or server during any given time before all communications have been disconnected whether by force or by refusal. You can run: modprobe ip_conntrack conntrack -E -e NEW Q. What is SYN flood and how can you detect it and mitigate it? A. A SYN flood is a form of denial-of-service attack in which an attacker sends a succession of SYN requests to a target's system in an attempt to consume enough server resources to make the system unresponsive to legitimate traffic. Detection can be done by by netstat or ss and filtering for SYN-RECV connection states. Mitigation can be done by null-routing the offending IP and enabling SYN cookies in the kernel, which allow the server to sends back the appropriate SYN+ACK response to the client but discards the SYN queue entry. ss -a | grep SYN-RECV | awk '{print $4}' | awk -F":" '{print $1}' | sort | uniq -c | sort -n netstat -antp | grep SYN_RECV|awk '{print $4}'|sort|uniq -c | sort -n

Q. You have a file with 2000 IP's. How do you ping them all using bash in parallel? A. echo $(cat iplistfile) | xargs -n 1 -P0 ping -w 1 -c 1 Q. What command can you use to send unsolicited ARP updates to the neighboring servers' caches. A. arping -U -c 1 -I eth0 0.0.0.0 -s IP_ADDRESS Q. What Linux utility can craft custom packets, like TCP SYN packets and send them to a remote host? A. hping3 -S 192.168.1.1 -p 80 -i u1 Q. What is Memory Overcommit in Linux? A. By default, Linux will allow processes to allocate more virtual memory than the system actually has, assuming that they won't end up actually using it. When there's more overcommited memory than the available physical and swap memory the OOM-killer picks some process to kill in order to recover memory. One reason Linux manages memory this way by default is to optimize memory usage on fork()'ed processes; fork() creates a full copy of the process space, but in this instance, with overcommitted memory, only pages which have been written to actually need to be allocated by the kernel. Q. What is system load averag as displayed by uptime? A. Load Average is the sum of the number of processes waiting in the run-queue plus the number currently executing.If there are four CPUs on a machine and the reported one-minute load average is 4.00, the machine has been utilizing its processors perfectly for the last 60 seconds. Q. How do you list all kernel modules that are compiled in or enabled? A. You can execute: cat /boot/config-$(uname

-r)

Q. Kernel space Vs. User space pros and cons. A. The role of the operating system, in practice, is to provide programs with a consistent view of the computer's hardware. In addition, the operating system must account for independent operation of programs and protection against unauthorized access to resources. This nontrivial task is possible only if the CPU enforces protection of system software from the applications. Every modern processor is able to enforce this behavior. The chosen approach is to implement different operating modalities (or levels) in the CPU itself. The levels have different roles, and some operations are disallowed at the lower levels; program code can switch from one level to another only through a limited number of gates. Unix systems are designed to take advantage of this hardware feature, using two such levels. All current processors have at least two protection levels, and some, like the x86 family, have more levels; when several levels exist, the highest and lowest levels are used. Under Unix, the kernel executes in the highest level (also called supervisor mode), where everything is allowed, whereas applications execute in the lowest level (the so-called user mode), where the processor regulates direct access to hardware and unauthorized access to memory. We usually refer to the execution modes as kernel space and user space. These terms encompass not only the different privilege levels inherent in the two modes, but also the fact that each mode can have its own memory mappingits own address spaceas well. Unix transfers execution from user space to kernel space whenever an application issues a system call or is suspended by a hardware interrupt. Kernel code executing a system call is working in the context of a processit operates on behalf of the calling process and is able to access data in the process's address space. Code that handles interrupts, on the other hand, is asynchronous with respect to processes and is not related to any particular process. Q. What is the difference between Active and Passive FTP sessions: A. Active FTP : command channel : client port above1023 connects to server port 21 data channel: client port above 1023 is connected from server port 20

Passive FTP : command channel: client port above 1023 connects to server port 21 data channel: client port above 1023 connects to server port above 1023 MySQL Questions: Q. What are the two main MySQL storage engines, and how they differ? A. The two most popular storage engines in MySQL are InnoDB and MyISAM InnoDB supports some newer features like transactions, row-level locking, foreign keys. It's optimized for read/write high volume operations and high performance. MyISAM is simpler and better optimized for read only operations. It has limited feature set as compared to InnoDB. Q. What to consider when setting up master-to-master replication? A. Duplicate indexes can be a problem, when clients make changes to the database on both mastesr at the same time. To mitigate this configure both masters to use auto_increment_increment and auto_increment_offset values.

By having the perfect answer to the Top Ten Linux Interview Questions asked, you can easily ace any Linux Interview. Ive been working as a Freelance Linux System Administrator for the past seven years. Over all those years, Ive been in more than a dozen of roles, attended tens of face-to-face interviews and probably close to a hundred of phone interviews. Few painfully obvious things struck me during my time: A) Your encyclopedic knowledge of Linux is far more likely to get you a job than your actual problem solving skills. B) Technical questions asked during phone interviews and even face-to-face interviews tend to be fairly basic. Employers rarely ever go into detailed technical scenarios. C) You will be asked almost the same technical questions at almost all interviews you attend. This means that even if you feel your knowledge of Linux is limited at the moment, make sure to know detailed answers to the following Top Linux Interview Questions and you double your chances of getting the job. You are virtually guaranteed that at least half of the questions asked will sound very similar to the following: 1) What is the difference between TCP and UDP? The basic difference is that TCP establishes a connection before sending data and this allows it to control the dataflow and guarantee that all packets get delivered. UDP simply chucks datagrams onto the wire and if some get lost or arrive in bad order theres no way to request a resend. However UDP has low network overhead so some services such as DNS resolution, SNMP, DHCP, RIP and VOIP use UDP for its speed and any errors are usually dealt with on the application layer rather than network layer.

2) What is the TCP hand shake? TCP requires three packets to set up a socket connection, before any user data can be sent. This is called the tree way TCP handshake. First the requester sends a SYN packet and expects a SYN-ACK packet, to which the initiator replies with ACK packet plus the first chunk of user data. From there on the TCP connection is established and two sides exchange user data using features such as message acknowledgment, retransmission and timeout. 3) How does DNS resolution work? A client application requests an IP address from the name server usually by connecting to UDP port 53. The name server will attempt to resolve the FQDN based on its resolver library, which may contain authoritative information about the host requested or cached data about that name from an earlier query. If the name server does not already have the answer, it will turn to root name servers to determine the authoritative for the FQDN in question. Then, with that information, it will query the authoritative name servers for that name to determine the IP address. 4) What is an MX record? MX record numerically ranks the mail servers you would prefer to receive email for a domain. The MX record with the lowest number is preferred over the others, but you can set multiple email servers with the same value for simple load balancing. 5) Describe Linux boot-up sequence BIOS reads the MBR where Boot Loader sits, Boot Loader reads Kernel into memory, Kernel starts Init process, Init reads inittab, executes rc.sysinit, the rc script than starts services to reach the default run level and once this is done the last thing that gets run is the rc.local script. 6) What is an INODE? All files have its description stored in a structure called inode. The inode contains info about the file-size, access and modification time, permission and so on. In addition to descriptions about the file, the inode contains pointers to the data blocks of the file. 7) How do you search for a pattern and than replace it in an entire file? Using Sed or in Vi editor the search usually involves character s slash the pattern to be searched, slash the pattern to replace it with, slash g which stands for entire file. 8) How do you list and flush all IPtables? Using the iptables command with L switch first to see all the rules currently present in memory and than F to flush them.

9) How do you list compiled-in Apache modules? Run the httpd daemon as a command with l parameter. 10) What is a zombie? Zombie processes can be identified in the output of ps by the presence of Z in the STAT column. Zombies are child processes whose parent process died without reaping its children. Zombies cant be killed with the usual KILL signal. So there you go. This is the technical part of your Linux interview handled. Dont ask me why but these are the top most repeated Linux questions at interviews. If it is recruiters lack of creativity or laziness. It is simply a fact.