eISSN: 3087-4262 editor@aimjournals.com
All Journals
Submit your article

International Journal of Intelligent Data and Machine Learning

Open Access Peer Review International
Open Access

High-Frequency Data Driven Network Learning for Systemic Risk Analysis in Financial Markets

DOI https://doi.org/10.55640/
pdf
Elias J. Vance 1 , Clara M. Soto 1 ,
4 Department of Data Science, Massachusetts Institute of Technology, Cambridge, United States
4 Department of Data Science, Massachusetts Institute of Technology, Cambridge, United States

Abstract

Purpose: This study investigates the utilization of high-frequency trade (HFT) data, combined with advanced machine learning (ML) techniques, to infer and analyze dynamic financial networks for the purpose of systemic risk assessment. Traditional network models often fail to capture the rapid, non-linear dependencies that propagate systemic risk, particularly under volatile conditions.

Methodology: We develop a novel framework that leverages HFT data from firms to construct a rich feature space, including realized volatility and granular market microstructure proxies such as order-book imbalance. A Random Forest (RF) model is employed to learn the non-linear relationship between firm-specific features and future systemic risk contribution, with the resultant feature importance scores defining the dynamic, directed network edges. An Explainable AI (XAI) framework, using SHAP values, is implemented to address the "black box" nature of the RF and provide attributable risk contributions.

Results: Our ML-driven network consistently reveals dynamic dependencies that are obscured in lower-frequency analyses. We find that the inclusion of order-book imbalance metrics enhances the prediction accuracy (AUC) of systemic risk events by an average of compared to models relying solely on realized volatility. The XAI analysis reveals that the marginal impact of microstructure shocks on systemic risk is non-linear and becomes exponentially greater during periods of high market volatility.

Conclusion: The integration of HFT data and ML offers a powerful lens into the architecture of systemic risk. However, while offering superior insight and explainability, the study concludes that current network models still face significant challenges in capturing all complex, non-linear dimensions of contagion, especially during extreme, unprecedented market stress. Further research into multilayer networks and alternative ML architectures is warranted.

Keywords

Financial Networks, High-Frequency Data (HFT), Systemic Risk, Market Microstructure

References

Basu S, Das S, Michailidis G, Purnanandam A. 2019. A system-wide approach to measure connectivity in the financial sector. Available at SSRN 2816137.
Chandra R. (2025). Reducing latency and enhancing accuracy in LLM inference through firmware-level optimization. International Journal of Signal Processing, Embedded Systems and VLSI Design, 5(2), 26–36. https://doi.org/10.55640/ijvsli-05-02-02
Andersen T G, Teräsvirta T. 2009. Realized volatility. In: Handbook of financial time series. Heidelberg: Springer. p. 555–575.
Chordia T, Sarkar A, Subrahmanyam A. 2011. Liquidity dynamics and cross-autocorrelations. J Financ Quant Anal. 46(3):709–736.
Durgam S. (2025). CICD automation for financial data validation and deployment pipelines. Journal of Information Systems Engineering and Management, 10(45s), 645–664. https://doi.org/10.52783/jisem.v10i45s.8900
Wang G J, Yi S, Xie C, Stanley H E. 2021. Multilayer information spillover networks: measuring interconnectedness of financial institutions. Quant Finance. 21(7):1163–1185.
Breiman L. 2001. Random forests. Mach Learn. 45(1):5–32.
Härdle W K, Wang W, Yu L. 2016. Tenet: tail-event driven network risk. J Econ. 192(2):499–513.
Reddy Gundla S. (2025). PostgreSQL tuning for cloud-native Java: Connection pooling vs. reactive drivers. International Journal of Computational and Experimental Science and Engineering, 11(3). https://doi.org/10.22399/ijcesen.3479
Reddy Dhanagari M. (2025). Aerospike vs. traditional databases: Solving the speed vs. consistency dilemma. International Journal of Computational and Experimental Science and Engineering, 11(3). https://doi.org/10.22399/ijcesen.3780
Biau G, Scornet E. 2016. A random forest guided tour. Test. 25(2):197–227.
Gu S, Kelly B, Xiu D. 2020. Empirical asset pricing via machine learning. Rev Financ Stud. 33(5):2223–2273.
NAICS. 2017. North American Industry Classification System manual. Executive Office of the President, Office of Management and Budget.
Srilatha S. (2025). Integrating AI into enterprise content management systems: A roadmap for intelligent automation. Journal of Information Systems Engineering and Management, 10(45s), 672–688. https://doi.org/10.52783/jisem.v10i45s.8904
Fawcett T. 2006. An introduction to ROC analysis. Pattern Recog Lett. 27(8):861–874.
O’Hara M. 2015. High frequency market microstructure. J Financ Econ. 16:257–270.
Härdle W K, Chen S, Liang C, Schienle M. 2018. Time-varying limit order book networks. IRTG 1792 Discussion Paper 2018. Report No.
Benjamini Y, Hochberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol. 57(1):289–300.
CRSP Stocks. 2019. Available: Center for Research in Security Prices. Graduate School of Business. University of Chicago. Retrieved from Wharton Research Data Services. Accessed.
Lulla K L, Chandra R C, & Sirigiri K S. (2025). Proxy-based thermal and acoustic evaluation of cloud GPUs for AI training workloads. The American Journal of Applied Sciences, 7(7), 111–127. https://doi.org/10.37547/tajas/Volume07Issue07-12
Easley D, de Prado M L, O’Hara M, Zhang Z. 2021. Microstructure in the machine age. Rev Financ Stud. 34(7):3316–3363.
Kiel P, Nguyen D. 2013. Bailout tracker: tracking every dollar and every recipient. ProPublica: Journalism in the Public Interest.
Liaw A, Wiener M. 2002. Classification and regression by randomForest. R News. 2(3):18–22. Available from: https://CRAN.R-project.org/doc/Rnews/
Lo A W, MacKinlay A C. 1990. When are contrarian profits due to stock market overreaction? Rev Financ Stud. 3(2):175–205.
Dutta C, Karpman K, Basu S, Ravishanker N. 2022. Review of statistical approaches for modeling high-frequency trading data. Sankhya B. Available from: https://doi.org/10.1007/s13571-022-00280-7
Chandra Jha A. (2025). VXLAN/BGP EVPN for trading: Multicast scaling challenges for trading colocations. International Journal of Computational and Experimental Science and Engineering, 11(3). https://doi.org/10.22399/ijcesen.3478
Wang G J, Xie C, He K, Stanley H E. 2017. Extreme risk spillover network: application to financial institutions. Quant Finance. 17(9):1417–1433.
Billio M, Getmansky M, Lo A W, Pelizzon L. 2012. Econometric measures of connectedness and systemic risk in the finance and insurance sectors. J Financ Econ. 104(3):535–559.
Diebold F X, Yilmaz K. 2014. On the network topology of variance decompositions: measuring the connectedness of financial firms. J Econ. 182(1):119–134.
Rangu S. (2025). Analyzing the impact of AI-powered call center automation on operational efficiency in healthcare. Journal of Information Systems Engineering and Management, 10(45s), 666–689. https://doi.org/10.55278/jisem.2025.10.45s.666
Huang B F, Boutros P C. 2016. The parameter sensitivity of random forests. BMC Bioinf. 17(1):1–13.
Friedman J, Posner R. 2011. What caused the financial crisis. Philadelphia: University of Pennsylvania Press.
Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J C, Müller M. 2011. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinf. 12:77.
Hautsch N, Schaumburg J, Schienle M. 2015. Financial network systemic risk contributions. Rev Finance. 19(2):685–738.
Rubin R E, Greenspan A, Levitt A, Born B. 1999. Hedge funds, leverage, and the lessons of long-term capital management. Report of The President’s Working Group on Financial Markets.
Hastie T, Friedman J, Tibshirani R. 2001. The elements of statistical learning. Vol. 1. Springer series in statistics. New York, NY: Springer.
Karpman K, Lahiry S, Mukherjee D, Basu S. 2022. Exploring financial networks using quantile regression and granger causality. arXiv. Available from: https://arxiv.org/abs/2207.10705
Brownlees C, Nualart E, Sun Y. 2018. Realized networks. J Appl Econ. 33(7):986–1006.
Chiodo A J, Owyang M T. 2002. A case study of a currency crisis: the Russian default of 1998. Fed Reserve Bank St. Louis Rev. 84(6):7.
Andersen T G, Teräsvirta T. 2009. Realized volatility. In: Handbook of financial time series. Heidelberg: Springer. p. 555–575.
Hautsch N. 2012. Econometrics of financial high-frequency data. Heidelberg: Springer.
Brennan M J, Jegadeesh N, Swaminathan B. 1993. Investment analysis and the adjustment of stock prices to common information. Rev Financ Stud. 6(4):799–824.
Pelger M. 2020. Understanding systematic risk: a high-frequency approach. J Finance. 75(4):2179–2220.
Billio M, Getmansky M, Lo A W, Pelizzon L. 2012. Econometric measures of connectedness and systemic risk in the finance and insurance sectors. J Financ Econ. 104(3):535–559.
Jha A C. (2025). DWDM optimization: Ciena vs. ADVA for <50 ms global finances. Utilitas Mathematica, 122(2), 227–245. https://utilitasmathematica.com/index.php/Index/article/view/2713
Hariharan R. (2025). Zero trust security in multi-tenant cloud environments. Journal of Information Systems Engineering and Management, 10(45s). https://doi.org/10.52783/jisem.v10i45s.8899
Jain, R., Sai Santosh Goud Bandari, & Naga Sai Mrunal Vuppala. (2025). Polynomial Regression Techniques in Insurance Claims Forecasting. International Journal of Computational and Experimental Science and Engineering, 11(3). https://doi.org/10.22399/ijcesen.3519
Vennamaneni, P. R. (2025). Modernizing monoliths: Transitioning legacy financial systems to microservices. Journal of Information Systems Engineering & Management, 10(45s), 552–574. https://doi.org/10.52783/jisem.v10i45s.8896
Singh, V. (2025). Security in Financial Audits. QTanalytics Publication (Books), 21–32. https://doi.org/10.48001/978-81-988770-4-8-3
Kumar Tiwari, S. (2023). Security testing automation for digital transformation in the age of cyber threats. International Journal of Applied Engineering & Technology, 5(S5), 135–146. Roman Science Publications.

Most read articles by the same author(s)

1 2 3 4 5 6 7 > >> 

Similar Articles

11-20 of 63

You may also start an advanced similarity search for this article.