Debugging Techniques in Linux
Duration: 3 days

Back to course list Home
Synopsis This course introduces debugging tools in Linux. Powerful utilities ubiquitous in Linux distributions, such as strace/ltrace, proc, and - the built-in debugger, gdb. We discuss how to develop and use debugging skills to deal with common bugs, such as memory corruptions, erroneous usage of pointers, multi-threaded code and race-conditions, socket bugs, and more. Special consideration is given to analyzing core dumps and crashes. The techniques shown are applicable across all flavors of Linux - including Android distributions (at the CLI level)
Target Audience Application developers, and/or anyone interested in obtaining deeper insights into ekeing out the maximum from the Linux OS
Prerequisites
  • Knowledge of user mode programming. Familiarity with POSIX (unistd.h) and system calls
Objectives
  • Use common Linux performance monitoring tools
  • Use Linux's built-in debugging tools - strace, ltrace, gdb, and others
  • Understand and effectively use GCC's myriad compiler optimizations
  • Effectively code multi-threaded programs
Exercises This course allocates plenty of time for hands-on practice.
The hands-on exercises include:
  • Debugging user mode crash dumps ("core" files)
  • Writing, profiling and debugging code
Modules
1. Introduction to Linux Architecture
4 hours
Introduction to the Linux architecture. We discuss the design and implementation of the Linux Kernel and its various subsystems, at a modular "black box" level, without going into the source code level. We discuss the Kernel architecture and tunable parameters that allow for performance optimization.
  • Linux Kernel features
    • Version differences - 2.6.x, 3.x and 4 subversions
      • User-Mode and Kernel-Mode Architecture
        • The Linux Task Scheduler (CFS)
          • The Linux I/O architecture:
            • I/O Schedulers
              • Buffer cache and page cache
              • The /proc filesystem
                • The /sys filesystem
                  2. Compiling, Linking and Debugging
                  2 hours
                  In-depth discussion of code at the assembly (machine code) level. Students will learn:
                  • Familiarity with assembly code
                    • GCC compiler optimizations
                      • Compiler level tricks - Preprocessor
                        • Compiler level tricks - Debugging Info
                          • Compiler level tricks - Inline Assembly
                            • Using GDB - Tips & Tricks
                              3. Processes and Threads
                              2 hours
                              The inside view of Linux "tasks" - processes and threads - and how the scheduler manages. Using process diagnostic tools effectively.
                              • Processes
                                • Daemons
                                  • Threads
                                    • Priorities and Nice
                                      • The kernel perspective - clone()
                                        • Process scheduler states
                                          • Real-Time priorities
                                            4. Process Tracing & Hooking
                                            2 hours
                                            Methods for code-injection into running processes via libraries and the ptrace(2) API.
                                            • Library calls vs. system calls - when to use which?
                                              • Libraries
                                                • Compiling a shared library
                                                  • Injecting libraries
                                                  • System call tracing with ptrace
                                                    • The ptrace(2) API
                                                      • System call level tracing with strace(1)
                                                        • Library call level tracing with ltrace(1)
                                                          • Advanced tracing with jtrace
                                                          • gprof
                                                            • valgrind

                                                              This module demonstrates the following tools:
                                                              • Strace
                                                              • LTrace
                                                              • JTrace
                                                              • Valgrind
                                                              5. Virtual Memory
                                                              3 hours
                                                              A deeper look into virtual memory :
                                                              • Process memory layout
                                                                • System memory management
                                                                  • Page/Buffer Cache maintenance
                                                                    • Kernel tunables
                                                                      • Kernel optimizations (KSM, compressed memory, etc)
                                                                        6. Memory management bugs
                                                                        3 hours
                                                                        A detailed discussion of memory allocation/management bugs, and how they manifest themselves in stack and heap corruption.
                                                                        • Stack overrun
                                                                          • Stack buffer overflows
                                                                            • Heap buffer overflows
                                                                              • Double Free bugs
                                                                                • Uninitialized memory
                                                                                  • Use after free
                                                                                    • Memory leaks

                                                                                      This module demonstrates the following tools:
                                                                                      • Val/Cal/MemGrind
                                                                                      • Mcheck
                                                                                      7. Concurrency
                                                                                      3 hours
                                                                                      A discussion of the challenges involved with writing multi-threaded and multi-process code.
                                                                                      • Concurrency
                                                                                        • Inter-Process Communication (IPC) techniques
                                                                                          • Locking techniques
                                                                                            • Types of locks
                                                                                              • Optimizing locks
                                                                                                • Deadlocks
                                                                                                • GDB and multi-threaded programs
                                                                                                  8. Kernel-Level Debugging for User-mode
                                                                                                  2 hours
                                                                                                  Basic knowledge of Kernel-level coding techniques offers a plethora of new debugging techniques. While this is not a full-fledged Linux Kernel course, we introduce the basic concepts for a quick-and-dirty module, to allow for the injection of code into the Kernel.
                                                                                                  • Why Kernel? The benefits of writing a module:
                                                                                                    • System call interception
                                                                                                      • Precise timing measurements
                                                                                                        • Faking system calls - Virtualizing or stress testing
                                                                                                        • Writing a basic module
                                                                                                          • KProbes
                                                                                                            • UProbes (3.5+)
                                                                                                              9. Kernel Network Debugging
                                                                                                              2 hours
                                                                                                              Simple, yet surprisingly effective techniques to debug networking (socket) code. We introduce socket level API hooking, and packet level hooking (via NetFilter).

                                                                                                              • struct sock - The Kernel implementation of sockets
                                                                                                                • struct sk_buff - Socket buffers
                                                                                                                  • struct proto - the implementations of TCP and UDP
                                                                                                                    • NetFilter hooks