A Scalable Multi-Tenant Framework for AI-Driven Big Data Lake Management and Processing
Abstract
The rapid growth of artificial intelligence (AI), machine learning, and heterogeneous data sources has increased the need for scalable data-lake architectures capable of supporting concurrent workloads, diverse data types, and dynamically changing computational requirements. Conventional data-management approaches often struggle to provide adequate isolation, resource elasticity, governance, and intelligent workload coordination in multi-tenant environments. This research proposes a conceptual scalable multi-tenant framework for AI-driven big data lake management and processing in which tenant-aware orchestration, workload classification, resource allocation, data governance, and AI-assisted decision mechanisms operate as integrated architectural components. The framework is theoretically grounded in scalable data-lake orchestration, formal reasoning, explainable decision-making, and resource-aware computational management. Particular emphasis is placed on separating tenant-level policies from shared infrastructure while maintaining efficient utilization of storage and processing resources. The architectural rationale is informed by research on rule-based learning, formal verification, satisfiability solving, explainability, and computational reasoning. The proposed framework further incorporates principles associated with multitenant data-lake orchestration for AI workloads (Goyal, 2025). Analytical findings indicate that a policy-aware, AI-driven orchestration layer can improve workload prioritization, resource utilization, isolation, and operational transparency compared with static allocation models. The study also identifies limitations associated with governance complexity, model dependence, computational overhead, and fairness across tenants. The resulting framework provides a research foundation for scalable, intelligent, and explainable big data lake management.
Keywords
References
Similar Articles
- Agus Santoso, Siti Nurhayati, ALGORITHMIC GUARANTEES FOR HIERARCHICAL DATA GROUPING: INSIGHTS FROM AVERAGE LINKAGE, BISECTING K-MEANS, AND LOCAL SEARCH HEURISTICS , International Journal of Intelligent Data and Machine Learning: Vol. 2 No. 02 (2025): Volume 02 Issue 02
- Prof. Karan M. Bhatia, Mehul A. Rajput, HARNESSING AI FOR PROACTIVE PUBLIC RELATIONS: A FRAMEWORK FOR PREDICTING AND CAPITALIZING ON SOCIAL MEDIA TRENDS , International Journal of Intelligent Data and Machine Learning: Vol. 2 No. 10 (2025): Volume 02 Issue 10
- Dr. Jianhong Liu, Dr. Meilin Zhou, A Machine Learning–Driven Framework for Multi-Temporal Flood Inundation Mapping and Spatial Analysis in Kolhapur, India Using SAR Remote Sensing Observations , International Journal of Intelligent Data and Machine Learning: Vol. 3 No. 07 (2026): Volume 03 Issue 07
- Dr. Kwame Mensah, Dr. Abena Owusu, An Interpretable Visual Analytics Framework for Machine Learning–Based Multichannel Time Series Classification and Performance Evaluation , International Journal of Intelligent Data and Machine Learning: Vol. 3 No. 08 (2026): Volume 03 Issue 08
- Mateo Laurent Dufour, Architecting Secure and Scalable Production Machine Learning Systems: Integrating Model Management, High Performance Computing, and Cloud Native Infrastructure , International Journal of Intelligent Data and Machine Learning: Vol. 3 No. 03 (2026): Volume 03 Issue 03
- Dr. Tanay Deshpande, Dr. Kavita Sharma, ADVANCING ARTIFICIAL INTELLIGENCE: AN IN-DEPTH LOOK AT MACHINE LEARNING AND DEEP LEARNING ARCHITECTURES, METHODOLOGIES, APPLICATIONS, AND FUTURE TRENDS , International Journal of Intelligent Data and Machine Learning: Vol. 2 No. 01 (2025): Volume 02 Issue 01
- Dr. Tashi Wangchuk, Karma Lhendup, Data-Driven Model Supporting Defect Analysis through Vision Techniques in Press-Formed Vehicle Components , International Journal of Intelligent Data and Machine Learning: Vol. 3 No. 04 (2026): Volume 03 Issue 04
- Dr. Larian D. Venorth, Prof. Maevis K. Durand, The Transformative Trajectory Of Large Language Models: Societal Impact, Predictive Limitations, And The Unforeseen Geohazard Nexus , International Journal of Intelligent Data and Machine Learning: Vol. 2 No. 10 (2025): Volume 02 Issue 10
- Dr. Rohan Pillay, Dr. Ananya Naidoo, Comparative Analysis of Machine Learning Approaches for Cardiovascular Risk Assessment , International Journal of Intelligent Data and Machine Learning: Vol. 3 No. 08 (2026): Volume 03 Issue 08
- Dr. Emil Novak, Deep Learning For E‑Commerce Recommendations: Capturing Long- And Short-Term User Preferences With Cnn-Based Representation Learning , International Journal of Intelligent Data and Machine Learning: Vol. 2 No. 12 (2025): Volume 02 Issue 12
You may also start an advanced similarity search for this article.