Debugging Techniques in Linux
Duration: 3 days

Back to course list Home
Synopsis This course introduces debugging tools in Linux. Powerful utilities ubiquitous in Linux distributions, such as strace/ltrace, proc, and - the built-in debugger, gdb. We discuss how to develop and use debugging skills to deal with common bugs, such as memory corruptions, erroneous usage of pointers, multi-threaded code and race-conditions, socket bugs, and more. Special consideration is given to analyzing core dumps and crashes.
Target Audience Application developers, and/or anyone interested in obtaining deeper insights into ekeing out the maximum from the Linux OS
  • Use common Linux performance monitoring tools
  • Use Linux's built-in debugging tools - strace, ltrace, gdb, and others
  • Understand and effectively use GCC's myriad compiler optimizations
  • Effectively code multi-threaded programs
Exercises This course allocates plenty of time for hands-on practice.
The hands-on exercises include:
  • Debugging user mode crash dumps ("core" files)
  • Writing, profiling and debugging code
1. Introduction to Linux Architecture
2 hours
Introduction to the Linux architecture. We discuss the design and implementation of the Linux Kernel and its various subsystems, at a modular "black box" level, without going into the source code level. We discuss the Kernel architecture and tunable parameters that allow for performance optimization.
  • Linux Kernel features
    • Version differences - 2.6.x, 3.x and 4 subversions
      • User-Mode and Kernel-Mode Architecture
        • The Linux Task Scheduler (process manager)
          • The Linux I/O Schedulers:
            • Elevator/NoOp
              • DeadLine
                • Anticipatory
                  • Complete Fair Queueing (CFQ)
                  • The /proc filesystem
                    • The /sys filesystem
                      2a. The Basics
                      2 hours
                      This module, dealing with the basic hardware and software architectures of Intel-compatible x86 and x64, serves as an introduction and sets the baseline for the debugging workshop.

                      Note: This module may be replaced by a platform-specific discussion of non-intel architectures as well. This is mostly the case where embedded Linux, on ARM or PPC architectures, is involved.

                      • Computer Architecture
                        • The OS and the CPU
                          • Exception handling
                            • Traps
                              • Interrupt handling
                              2b. Compiling, Linking and Debugging
                              2 hours
                              In-depth discussion of code at the assembly (machine code) level. Students will learn:
                              • Familiarity with assembly code
                                • GCC compiler optimizations
                                  • Using GDB - Tips & Tricks
                                    3. Processes and Threads
                                    2 hours
                                    The inside view of Linux "tasks" - processes and threads - and how the scheduler manages. Using process diagnostic tools effectively.
                                    • Processes
                                      • Daemons
                                        • Threads
                                          • Priorities and Nice
                                            • The kernel perspective - clone()
                                              • Process scheduler states
                                                • Real-Time priorities
                                                  4. Process Tracing & Hooking
                                                  2 hours
                                                  Methods for code-injection into running processes via libraries and the ptrace(2) API.
                                                  • Library calls vs. system calls - when to use which?
                                                    • Libraries
                                                      • Compiling a shared library
                                                        • Injecting libraries
                                                        • System call tracing with ptrace
                                                          • The ptrace(2) API
                                                            • System call level tracing with strace(1)
                                                              • Library call level tracing with ltrace(1)
                                                              • gprof
                                                                • valgrind

                                                                  This module demonstrates the following tools:
                                                                  • Strace
                                                                  • LTrace
                                                                  • Valgrind
                                                                  5. Process Internals
                                                                  3 hours
                                                                  A deeper look into user mode processes - the virtual memory layout, and structure of the stack and the heap.
                                                                  • Process memory layout
                                                                    • The Stack
                                                                      • Compiler optimization
                                                                        • The Heap
                                                                          6. Memory management bugs
                                                                          4 hours
                                                                          A detailed discussion of memory allocation/management bugs, and how they manifest themselves in stack and heap corruption.
                                                                          • Stack overrun
                                                                            • Stack buffer overflows
                                                                              • Heap buffer overflows
                                                                                • Double Free bugs
                                                                                  • Uninitialized memory
                                                                                    • Use after free
                                                                                      • Memory leaks

                                                                                        This module demonstrates the following tools:
                                                                                        • Val/Cal/MemGrind
                                                                                        • Mcheck
                                                                                        7. Concurrency
                                                                                        3 hours
                                                                                        A discussion of the challenges involved with writing multi-threaded and multi-process code.
                                                                                        • Concurrency
                                                                                          • Inter-Process Communication (IPC) techniques
                                                                                            • Locking techniques
                                                                                              • Types of locks
                                                                                                • Optimizing locks
                                                                                                  • Deadlocks
                                                                                                  • GDB and multi-threaded programs
                                                                                                    8. Kernel-Level Debugging
                                                                                                    1 hours
                                                                                                    Basic knowledge of Kernel-level coding techniques offers a plethora of new debugging techniques. While this is not a full-fledged Linux Kernel course, we introduce the basic concepts for a quick-and-dirty module, to allow for the injection of code into the Kernel.
                                                                                                    • Why Kernel? The benefits of writing a module:
                                                                                                      • System call interception
                                                                                                        • Precise timing measurements
                                                                                                          • Faking system calls - Virtualizing or stress testing
                                                                                                          • Writing a basic module
                                                                                                            9. Kernel Network Debugging
                                                                                                            2 hours
                                                                                                            Simple, yet surprisingly effective techniques to debug networking (socket) code. We introduce socket level API hooking, and packet level hooking (via NetFilter).

                                                                                                            • struct sock - The Kernel implementation of sockets
                                                                                                              • struct sk_buff - Socket buffers
                                                                                                                • struct proto - the implementations of TCP and UDP
                                                                                                                  • NetFilter hooks