This thesis presents a historical and technical analysis of heap exploitation and the corresponding mitigation mechanisms in Linux systems, with a focus on the glibc allocator ptmalloc2. The goal of the thesis is to understand how heap exploitation has evolved over time and how defensive mechanisms changed the requirements for successful exploitation. We also want to show that memory vulnerabilities are still relevant today and that understanding the progression of heap exploitation and how allocator internals behave is crucial for writing safer low-level software.
The work first reviews relevant background, including common heap vulnerability classes, major exploitation techniques, automatic exploit generation, and defensive approaches at the hardware, operating system, compiler and allocator levels and approaches such as patching, runtime-protection, and dynamic-analysis levels. It then explains the internal structure of the heap and the main glibc components relevant to exploitation, such as chunks, bins, arenas, hooks, exit handlers, and the thread control block.
The core contribution of the thesis is a step-by-step analysis of how a single proof-of-concept exploit based on a use-after-free vulnerability must evolve as modern mitigations are introduced. Starting from an environment without major protections, the PoC is progressively adapted to account for non-executable memory, address space randomization and position-independent executable, safe-linking, and the removal of malloc hooks in recent glibc versions. The analysis demonstrates how exploitation techniques shifted from direct shellcode execution toward address leaking, code reuse, pointer mangling bypasses and abuse of alternative control flow targets.
The results show that although modern mitigations significantly increase the complexity of heap exploitation, they do not eliminate the risk posed by memory corruption vulnerabilities. Instead, exploitation strategies evolve together with allocator internals and defensive mechanisms, making a detailed understanding of both offensive techniques and mitigations essential for software security research.
|
