ECML 2023 Tutorial -- Revolutionizing Anomaly Detection: Approaches and Guidelines Generation
Anomaly detection is a significant task of data mining, and also a hot research topic in various fields of artificial intelligence in recent years. It has a wide range of applications, such as extreme climate event detection, mechanical fault detection, terrorist detection, fraud detection, malicious URL detection etc.
This tutorial aims to present a comprehensive review of both shallow and deep-learning-based anomaly detection with an explanation. We first introduce the key intuitions, objective functions, underlying assumptions, and advantages and disadvantages of state-of-the-art anomaly detection methods. We also introduce several principled approaches used to provide anomaly explanations for deep detection models. Furthermore, we will discuss the connections between classic shallow and novel deep methods and provide a practical guide on how to select an outlier detector in different applications.
There are many kinds of classic shallow anomaly detection methods proposed to solve the problem of anomaly detection in various scenarios. However, the explosive growth of databases in both size and dimensionality is challenging for anomaly detection methods in two important aspects: the requirement of low computational cost and the susceptibility to high-dimensionality issues. Efficient methods are in high demand for time-critical applications ranging from network intrusion detection to credit card fraud detection.
Recently, deep learning has shown Phenomenal success in tackling these complexities in a wide range of applications, but popular deep learning techniques are inapplicable to anomaly detection due to some unique characteristics of anomalies, e.g., rarity, heterogeneity, unbounded nature, and prohibitively high cost of collecting large-scale anomaly data. A large number of studies, therefore, have been dedicated to deep methods specifically designed for anomaly detection. These studies demonstrate great success in addressing some major challenges to which shallow anomaly detection methods fail in different application contexts.
Specific goals and content
This tutorial presents a comprehensive coverage of both shallow and deep learning-based anomaly detection and explanation, including hands-on practices and a practitioner's guide on how to select suitable outlier detectors in various real-world applications.
Through this tutorial, we aim to promote the research development in algorithms, theories and evaluation of explainable shallow and deep anomaly detection in the data mining and machine learning community.
Expected background of the audience
The tutorial has no prerequisites but general knowledge of data analytics is desired. This tutorial is intended to draw the attention of students and researchers who are interested in anomaly detection and have basic machine-learning skills, and statistical and mathematic backgrounds.
Researchers and practitioners in finance, cybersecurity, and healthcare would also find the tutorial helpful in practice.
You can preview and download the tutorial slides here