eISSN: 3087-4041 editor@aimjournals.com
All Journals
Submit your article

The Pinnacle Research Journal of Scientific and Management Sciences

Open Access Peer Review International
Open Access

Enhancing Anomaly Detection in Complex Systems through Context Discovery

DOI https://doi.org/10.55640/
PDF
Dr. Elara V. Thorne 1 ,
4 Department of Data Science, Institute for Advanced Systems Research, Berlin, Germany

Abstract

Background: The increasing complexity and interconnectedness of modern systems across various domains have made effective anomaly detection a critical task. However, traditional anomaly detection techniques often operate in a context-agnostic manner, leading to sub-optimal performance characterized by high false-positive rates and an inability to detect subtle, context-dependent anomalies [1, 16]. This limitation is particularly pronounced in systems where the definition of "normal" behavior is highly dependent on situational factors such as time of day, network topology, or system state.

Objective: This study aims to address the limitations of conventional anomaly detection by proposing a novel framework that systematically discovers and integrates contextual information. The primary objective is to demonstrate that by leveraging context, detection models can achieve significantly improved accuracy and reliability in identifying deviations from normal behavior.

Methods: Our framework employs a multi-stage approach, beginning with the identification of relevant contextual features from the dataset. These features are then used to condition the anomaly detection process. The proposed model is compared against widely-used baseline models such as Isolation Forest and Local Outlier Factor (LOF) [2, 8] using a dataset derived from a complex system. Performance is evaluated using standard metrics, including precision, recall, and the F1-score.

Results: The experimental results show that the context-aware approach consistently outperforms traditional methods, achieving a higher F1-score and significantly reducing the false-positive rate. The integration of contextual data enables the model to accurately classify behaviors that would otherwise be misidentified by conventional techniques.

Conclusion: This research demonstrates the paramount importance of context discovery for effective anomaly detection. The proposed framework provides a robust and practical method for integrating contextual information, leading to more accurate, reliable, and actionable anomaly detection in complex systems.

Keywords

Anomaly Detection, Context-Aware, Machine Learning

References

📄 Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3), 1-58.
📄 Liu, F. T., Ting, K. M., & Zhou, Z. H. (2008). Isolation Forest. Proceedings of the IEEE International Conference on Data Mining (ICDM), 413-422.
📄 Ahmed, M., Mahmood, A. N., & Hu, J. (2016). A survey of network anomaly detection techniques. International Journal of Computer Science and Network Security (IJCSNS), 16(12), 258-275.
📄 Pimentel, M. A. F., et al. (2014). A review of novelty detection. Signal Processing, 99, 215-249.
📄 Eskin, E., et al. (2002). A geometric framework for unsupervised anomaly detection. Proceedings of the 17th International Conference on Machine Learning (ICML).
📄 Lin, J., & Keogh, E. (2007). A symbolic representation of time series, with implications for streaming algorithms. Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD).
📄 Xu, H., et al. (2018). Contextual anomaly detection in graphs. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 1347-1355.
📄 Breunig, M. M., et al. (2000). LOF: Identifying density-based local outliers. ACM SIGMOD Record, 29(2), 93-104.
📄 Gopalakrishnan, V., & Murthy, C. A. (2014). Context-sensitive anomaly detection in time-series data. Proceedings of the IEEE International Conference on Data Mining (ICDM).
📄 Cheng, S., & Xie, J. (2021). Contextual Outlier Detection via Conditional Density Estimation. IEEE Transactions on Neural Networks and Learning Systems, 32(5), 2069-2081.
📄 Gómez, V., et al. (2019). Time-Series Anomaly Detection with Deep Learning. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD).
📄 Zhao, Z., & Karypis, G. (2012). Context-aware anomaly detection in large-scale data. Proceedings of the 2012 SIAM International Conference on Data Mining (SDM).
📄 Xia, L., et al. (2015). A survey of context-aware anomaly detection methods for wireless sensor networks. Journal of Computer Science and Technology, 30(6), 1169-1181.
📄 Dong, Z., & Chen, H. (2017). Adaptive context-aware anomaly detection for network intrusion detection systems. Proceedings of the IEEE International Conference on Communications (ICC), 1-7.
📄 Vidal, R., et al. (2020). Context-aware deep anomaly detection for monitoring industrial systems. IEEE Transactions on Industrial Informatics, 16(5), 3204-3213.
📄 Tan, P. N., Steinbach, M., & Kumar, V. (2006). Introduction to Data Mining. Pearson.
📄 Bishop, C.M. (2006). Pattern Recognition and Machine Learning. Springer.