Smashing the Stack with Hydra: The Many Heads of Advanced Polymorphic Shellcode

Pratap Prabhu; Yingbo Song; Salvatore Stolfo

Smashing the Stack with Hydra: The Many Heads of Advanced Polymorphic Shellcode
Prabhu, Pratap
Song, Yingbo
Stolfo, Salvatore
Technical reports
Computer Science
Permanent URL:
Columbia University Computer Science Technical Reports
Part Number:
Department of Computer Science, Columbia University
Publisher Location:
New York
Recent work on the analysis of polymorphic shellcode engines suggests that modern obfuscation methods would soon eliminate the usefulness of signature-based network intrusion detection methods and supports growing views that the new generation of shellcode cannot be accurately and efficiently represented by the string signatures which current IDS and AV scanners rely upon. In this paper, we expand on this area of study by demonstrating never before seen concepts in advanced shellcode polymorphism with a proof-of-concept engine which we call Hydra. Hydra distinguishes itself by integrating an array of obfuscation techniques, such as recursive NOP sleds and multi-layer ciphering into one system while offering multiple improvements upon existing strategies. We also introduce never before seen attack methods such as byte-splicing statistical mimicry, safe-returns with forking shellcode and syscall-time-locking. In total, Hydra simultaneously attacks signature, statistical, disassembly, behavioral and emulation-based sensors, as well as frustrates offline forensics. This engine was developed to present an updated view of the frontier of modern polymorphic shellcode and provide an effective tool for evaluation of IDS systems, Cyber test ranges and other related security technologies.
Computer science
Item views:
text | xml
Suggested Citation:
Pratap Prabhu, Yingbo Song, Salvatore Stolfo, 2009, Smashing the Stack with Hydra: The Many Heads of Advanced Polymorphic Shellcode, Columbia University Academic Commons, http://hdl.handle.net/10022/AC:P:9311.

In Partnership with the Center for Digital Research and Scholarship at Columbia University Libraries/Information Services | Terms of Use | Copyright