The Shellcoder's Handbook:

Discovering and Exploiting Security Holes

Jack Koziol, Dave Aitel, David Litchfield, Chris Anley, Sinan "noir" Eren, Neel Mehta, Riley Hassell
WILEY

Wiley Publishing, Inc.

Contents

About the Authors Credits Acknowledgments

VII

xi
1
3 3 4 6 7

Part 1
Chapter 1

Introduction to Exploitation: Linux on x86

Before You Begin Basic Concepts Memory Management Assembly Registers Recognizing C++ Code Constructs in Assembly Conclusion Stack Overflows Buffers The Stack Functions and the Stack Overflowing Buffers on the Stack Controlling EIP Using an Exploit to Get Root Privileges The Address Problem The NOP Method Defeating a Non-Executable Stack Return to libc Conclusion

8
10 11 12 13 15 18 20

Chapter 2

22
24 27 29 30 33

xiii

xlv

Contents Chapter 3 Shelkode Understanding System Calls Writing Shellcode for the exit() Syscall Injectable Shellcode Spawning a Shell Conclusion Introduction to Format String Bugs Prerequisites What Is a Format String? What Is a Format String Bug? Format String Exploits Crashing Services Information Leakage Controlling Execution for Exploitation Why Did This Happen? Format String Technique Roundup Conclusion Introduction to Heap Overflows What Is a Heap? How a Heap Works Finding Heap Overflows Basic Heap Overflows Intermediate Heap Overflows Advanced Heap Overflow Exploitation What to Overwrite Conclusion Exploiting More Platforms: Windows, Solaris, and Tru64 The Wild World of Windows How Does Windows Differ from Linux? Win32 API and PE-COFF Heaps Threading The Genius and Idiocy of the Distributed Common Object Model and DCE-RPC Recon Exploitation Tokens and Impersonation Exception Handling under Win32 Debugging Windows Bugs in Win32 Writing Windows Shellcode A Hacker's Guide to the Win32 API A Windows Family Tree from the Hacker's Perspective Conclusion
35 36 38 42 44

53 55
55 56 57 62 63 64 69 79 79

Chapter 4

82
83 84 85 86 87 93 99 100 101

Chapter 5

Part 2 Chapter 6

103
105 105 106 108 109
110 112 114 114 116 118 118 119 119 120 121

Contents Chapter 7 Windows Shellcode Syntax and Filters Setting Up Parsing the PEB Heapoverflow.c Analysis Searching with Windows Exception Handling Popping a Shell Why You Should Never Pop a Shell on Windows Conclusion Windows Overflows Stack-Based Buffer Overflows Frame-Based Exception Handlers Abusing Frame-Based Exception Handling on Windows 2003 Server Abusing an Existing Handler Find a block of code in an address not associated with a module that will get us back to our buffer Find a block of code in the address space of a module that does not have a Load Configuration Directory A Final Note about Frame-Based Handler Overwrites Stack Protection and Windows 2003 Server Heap-Based Buffer Overflows The Process Heap Dynamic Heaps Working with the Heap How the Heap Works Exploiting Heap-Based Overflows Overwrite Pointer to RtlEnterCriticalSection in the PEB Overwrite Pointer to First Vectored Handler at 77FC3210 Overwrite Pointer to Unhandled Exception Filter Overwrite Pointer to Exception Handler in Thread Environment Block Repairing the Heap Other Aspects of Heap-Based Overflows COM Objects and the Heap Overflowing Logic Program Control Data Wrapping Up the Heap Other Overflows .data section overflows TEB/PEB Overflows Exploiting Buffer Overflows and Non-Executable Stacks Conclusion 123 123 125 126 126 142 147 147 148 149 150 150 155 156 158 159 160 161 167 167 167 168 168 172 172 175 178 184 185 187 187 188 188 188 188 190 191 196

xv

Chapter 8

xvi

Contents Chapter 9 Overcoming Filters Writing Exploits for Use with an Alphanumeric Filter Writing Exploits for Use with a Unicode Filter What Is Unicode? Converting from ASCII to Unicode Exploiting Unicode-Based Vulnerabilities The Available Instruction Set in Unicode Exploits The Venetian Method An ASCII Venetian Implementation Decoder and Decoding The Decoder Code Getting a Fix on the Buffer Address Conclusion
197 197 201 202 202 203 204

205
207 210 111 212 213

Chapter 10 Introduction to Solaris Exploitation Introduction to the SPARC Architecture Registers and Register Windows The Delay Slot Synthetic Instructions Solaris/SPARC Shellcode Basics Self-Location Determination and SPARC Shellcode Simple SPARC exec Shellcode Useful System Calls on Solaris NOP and Padding Instructions Solaris/SPARC Stack Frame Introduction Stack-Based Overflow Methodologies Arbitrary Size Overflow Register Windows and Stack Overflow Complications Other Complicating Factors Possible Solutions Off-By-One Stack Overflow Vulnerabilities Shellcode Locations Stack Overflow Exploitation In Action The Vulnerable Program The Exploit Heap-Based Overflows on Solaris/SPARC Solaris System V Heap Introduction Heap Tree Structure Basic Exploit Methodology (t_delete) Standard Heap Overflow Limitations Targets for Overwrite The Bottom Chunk Small Chunk Corruption Other Heap-Related Vulnerabilities Off-by-One Overflows Double Free Vulnerabilities Arbitrary Free Vulnerabilities

215 216 216 219 219 220 220 221 111 222 223 224 224 225 225 225 226 227 228 228 230 233 234 234 254 257 258 259 260 261 261 261 262

Contents
Heap Overflow Example The Vulnerable Program Other Solaris Exploitation Techniques Static Data Overflows Bypassing the Non-Executable Stack Protection Conclusion Chapter 11 Advanced Solaris Exploitation Single Stepping the Dynamic Linker Various Style Tricks for Solaris SPARC Heap Overflows Advanced Solaris/SPARC Shellcode Conclusion Chapter 12 HP Tru64 Unix Exploitation The Alpha Architecture Alpha Registers Instruction Set Calling Conventions Retrieving the Program Counter (GetPC) System Call Invocation XOR Decoder for Shellcode .end main setuid + execve Shellcode Code the setuid(O) + execve("/bin/sh",...) systemcalls Compile the Assembly Code and Extract the Main Function Encode the Extracted opcodes with the XOR Key Plug the Encoded Code into the XOR Decoder Compile and Extract the Final Shellcode Connect-Back Shellcode Find-Socket Shellcode Bind-Socket Shellcode Stack Overflow Exploitation Defeating the Non-Executable Stack Exploiting rpc.ttdbserver Conclusion
262 262 266 267 267 268

xvn

269 271 286 288 300 301 302 302 303 305 306 308 308 310 310 311 312 313 314 316 317 319 320 321 322 330

Part 3

Vulnerability Discovery

331
333 334 334 334 335 335 335 335 335 336 336

Chapter 13 Establishing a Working Environment What You Need for Reference What You Need for Code
gcc gdb

NASM WinDbg. OllyDbg SoftlCE Visual C++ Pvthon

xviii Contents
What You Need for Investigation Useful Custom Scripts/Tools An Offset Finder Generic Fuzzers The Debug Trick All Platforms Unix ltrace strace fstat (BSD) tcpdump Ethereal Windows IDA Pro Disassembler What You Need to Know Paper Archives Optimizing Shellcode Development Plan the Exploit Write the Shellcode in Inline Assembler Maintain a Shellcode Library Make It Continue Nicely Make the Exploit Stable Make It Steal the Connection Conclusion Chapter 14 Fault Injection Design Overview Input Generation Manual Generation Automated Generation Live Capture "Fuzz" Generation Fault Injection Modification Engines Delimiting Logic Getting around Input Sanitization Fault Delivery Nagel Algorithm Timing Heuristics Stateless versus State-Based Protocols Fault Monitoring Using a Debugger FaultMon Putting It Together Conclusion 336 336 336 337 337 338 338 339 339 339 339 339 339 340 340 343 343 343 344 345 346 347 347 348 349 350 351 352 352 353 353 354 354 355 357 358 359 359 359 360 360 360 361 361 362

Contents
Chapter 15 The Art of Fuzzing General Theory of Fuzzing Static Analysis versus Fuzzing Fuzzing Is Scalable Weaknesses in Fuzzers Modeling Arbitrary Network Protocols Other Fuzzer Possibilities Bit Flipping Modifying Open Source Programs Fuzzing with Dynamic Analysis SPIKE What Is a Spike? Why Use the SPIKE Data Structure to Model Network Protocols? Various Programs Included with SPIKE SPIKE Example: dtlogin Other Fuzzers Conclusion Chapter 16 Source Code Auditing: Finding Vulnerabilities in C-Based Languages Tools Cscope Ctags Editors Cbrowser Automated Source Code Analysis Tools Methodology Top-Down (Specific) Approach Bottom-Up Approach Selective Approach Vulnerability Classes Generic Logic Errors (Almost) Extinct Bug Classes Format Strings Generic Incorrect Bounds-Checking Loop Constructs Off-by-One Vulnerabilities Non-Null Termination Issues Skipping Null-Termination Issues Signed Comparison Vulnerabilities Integer-Related Vulnerabilities Different-Sized Integer Conversions Double Free Vulnerabilities Out-of-Scope Memory Usage Vulnerabilities Uninitialized Variable Usage
363 363 368 368 370 371 371 371 372 372 372 373
374 374 374 381 382

xix

383 384 384 385 385 385 386 386 387 387 387 388 388 388 389 390 392 392 393 394 395 396 398 400 400 400

xx

Contents Use After Free Vulnerabilities Multithreaded Issues and Re-Entrant Safe Code Beyond Recognition: A Real Vulnerability versus a Bug Conclusion Chapter 17 Instrumented Investigation: A Manual Approach Philosophy Oracle extproc Overflow Common Architectural Failures Problems Happen at Boundaries A Process Calling into an External Process on the Same Host A Process Calling into an External, Dynamically Loaded Library A Process Calling into a Function on a Remote Host Problems Happen When Data Is Translated Problems Cluster in Areas of Asymmetry Problems Occur When Authentication and Authorization Are Confused Problems Occur in the Dumbest Places Bypassing Input Validation and Attack Detection Stripping Bad Data Using Alternate Encodings Using File-Handling Features Required String Is Present in Path Prohibited String Not Present in Path Incorrect Behavior Based on File Extension Evading Attack Signatures Defeating Length Limitations Sea Monkey Data Harmful TruncationSevering Escape Characters Multiple Attempts Context-Free Length Limits Windows 2000 SNMP DOS Finding DOS Attacks SQL-UDP Conclusion Chapter 18 Tracing for Vulnerabilities Overview A Vulnerable Program Component Design Process Injection Machine-Code Analysis Function Hooking Data Collection
401 402 402 403

405 405 406 410 410

410 410 411 411 413 414 414

415
415 415 416 416 417 417 419 419 419 420 421 421 421 422 423 424

427 428 429 431 432 432 436 439

Contents
Building VulnTrace VTInject VulnTrace.dU Using VulnTrace Advanced Techniques Fingerprint Systems More Vulnerability Classes Conclusion Chapter 19 Binary Auditing: Hacking Closed Source Software Binary versus Source-Code Auditing: The Obvious Differences IDA ProThe Tool of the Trade Features: A Quick Crash Course Debugging Symbols Binary Auditing Introduction Stack Frames Traditional BP-Based Stack Frames Functions without a Frame Pointer Non-Traditional BP-Based Stack Frames Calling Conventions The C Calling Convention The Stdcall Calling Convention Compiler-Generated Code Function Layouts If Statements For and While Loops Switch Statements memcpy-Like Code Constructs strlen-Like Code Constructs C++ Code Constructs The this Pointer Reconstructing Class Definitions vtables Quick but Useful Tidbits Manual Binary Analysis Quick Examination of Library Calls Suspicious Loops and Write Instructions Higher-Level Understanding and Logic Bugs Graphical Analysis of Binaries Manual Decompilation Binary Vulnerability Examples Microsoft SQL Server Bugs LSD's RPC-DCOM Vulnerability IIS WebDAV Vulnerability Conclusion
440 440 443 445 448 448 449

xxi

450 451 452 452

453 454 454 454 455 455 456 456 457 457 458 458 458 459 460 462 462 463 463 464 465 465 466 466 466 467 468 468 468 469 469 470 472

xxii Contents
Part 4 Advanced Materials Modifying the Program The SQL Server 3-Byte Patch The MySQL 1-Bit Patch OpenSSH RSA Authentication Patch Other Runtime Patching Ideas GPG 1.2.2 Randomness Patch Upload and Run (or Proglet Server) Syscall Proxies Problems with Syscall Proxies Conclusion
Chapter 21 Writing Exploits that Work in the Wild 473 475

Chapter 20 Alternative Payload Strategies

476 477 481 483 484 485 486 486 489 498
499

Factors in Unreliability Magic Numbers Versioning Shellcode Problems Network Related Privilege Related Configuration Related Host IDS Related Thread Related Countermeasures Preparation Brute Forcing Local Exploits OS/Application Fingerprinting Information Leaks Conclusion
Chapter 22 Attacking Database Software

499 499 500 501 501 501 502 502 502 503 504 504 505 505 507 508
509

Network Layer Attacks Application Layer Attacks Running Operating System Commands Microsoft SQL Server Oracle IBM DB2 Exploiting Overruns at the SQL Level SQL Functions Using the CHR/CHAR Function Conclusion
Chapter 23 Kernel Overflows

510 520 521 521 522 523 525 526 526 528
529

Kernel Vulnerability Types Oday Kernel Vulnerabilities OpenBSD exec_ibcs2_coff_prep_zmagic() Stack Overflow The Vulnerability

529 538 538 540

Contents xxm
Solaris vfs_getvfssw() Loadable Kernel Module Traversal Vulnerability The sysfs() System Call The mount() System Call Conclusion
Chapter 24 Exploiting Kernel vulnerabilities

544 546 547

548
549 549 554

The exec_ibcs2_coff_prep_zmagic() Vulnerability Calculating Offsets and Breakpoints Overwriting the Return Address and Redirecting Execution Locating the Process Descriptor (or the Proc Structure) Stack Lookup sysctl() Syscall Kernel Mode Payload Creation p_cred and u_cred Breaking chroot Returning Back from Kernel Payload Return to User Mode: iret Technique Return to Kernel Code: sidt Technique and _kernel_text Search Getting root (uid=0) Solaris vfs_getvfssw() Loadable Kernel Module Path Traversal Exploit Crafting the Exploit The Kernel Module to Load Getting root (uid=0) Conclusion
Index

557
558 558 558 560 560 561 561 562 564 567 574 574 576 579 580 581