Get access

Latent variable mining with its applications to anomalous behavior detection



In this paper, we propose a new approach to anomaly detection by looking at the latent variable space to make the first step toward latent anomaly detection. Most conventional approaches to anomaly detection are concerned with tracking data which are largely deviated from the ordinary pattern. In this paper, we are instead concerned with the issue of how to track changes occurring in the latent variable space consisting of the meta information existing behind directly observed data. For example, in the case of masquerade detection, the conventional task was to detect anomalous command lines related to masqueraders' malicious behaviors. Meanwhile, we rather attempt to track changes of behavioral patterns such as writing mails, making software, etc. which are information of more abstract level than command lines. The key ideas of the proposed methods are: (i) constructing the model variation vector, which is introduced relative to the latent variable space, and (ii) the latent anomaly detection is reduced to the issue of change-point detection for the time series that the model variation vector forms. We demonstrate through the experimental results using an artificial data set and a UNIX command data set that our method has significantly enhanced the accuracy of existing anomaly detection methods. Copyright © 2009 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 2: 48-69, 2009

Get access to the full text of this article