Behavior-Based Modeling and Its Application to Email Analysis

Salvatore Stolfo; Shlomo Hershkop; Chia-wei Hu; Wei-Jen Li; Olivier Nimeskern; Ke Wang

Behavior-Based Modeling and Its Application to Email Analysis
Stolfo, Salvatore
Hershkop, Shlomo
Hu, Chia-wei
Li, Wei-Jen
Nimeskern, Olivier
Wang, Ke
Computer Science
Industrial Engineering and Operations Research
Persistent URL:
ACM Transactions on Internet Technology, vol. 6, no. 2 (May 2006), pp. 187-221.
The Email Mining Toolkit (EMT) is a data mining system that computes behavior profiles or models of user email accounts. These models may be used for a multitude of tasks including forensic analyses and detection tasks of value to law enforcement and intelligence agencies, as well for as other typical tasks such as virus and spam detection. To demonstrate the power of the methods, we focus on the application of these models to detect the early onset of a viral propagation without "content-base" (or signature-based) analysis in common use in virus scanners. We present several experiments using real email from 15 users with injected simulated viral emails and describe how the combination of different behavior models improves overall detection rates. The performance results vary depending upon parameter settings, approaching 99% true positive (TP) (percentage of viral emails caught) in general cases and with 0.38% false positive (FP) (percentage of emails with attachments that are mislabeled as viral). The models used for this study are based upon volume and velocity statistics of a user's email rate and an analysis of the user's (social) cliques revealed in the person's email behavior. We show by way of simulation that virus propagations are detectable since viruses may emit emails at rates different than human behavior suggests is normal, and email is directed to groups of recipients in ways that violate the users' typical communications with their social groups.
Computer science
Publisher DOI:
Item views
text | xml
Suggested Citation:
Salvatore Stolfo, Shlomo Hershkop, Chia-wei Hu, Wei-Jen Li, Olivier Nimeskern, Ke Wang, 2006, Behavior-Based Modeling and Its Application to Email Analysis, Columbia University Academic Commons, http://hdl.handle.net/10022/AC:P:8686.

Center for Digital Research and Scholarship at Columbia University Libraries | Policies