The burgeoning landscape of e-commerce has transformed traditional marketplaces, giving rise to complex, dynamic environments where automated negotiation agents can play a pivotal role. Effective autonomous negotiation requires agents to not only understand their own objectives but also to strategically interact with and adapt to the behaviors of other participants. This article provides a comprehensive review of the synergistic integration of reinforcement learning (RL) and game theory (GT) to develop intelligent bargaining agents for digital marketplaces. We delve into how RL enables agents to learn optimal negotiation strategies through experience, even in environments with imperfect information and unknown opponents, while GT provides the theoretical foundation for rational decision-making, equilibrium analysis, and strategic interactions. By synthesizing empirical findings from various applications, including multi-issue bargaining and team formation, we illustrate the distinct advantages of combining these paradigms over purely isolated approaches. Furthermore, we address the current limitations of such hybrid frameworks and outline critical future research directions towards building more robust, adaptive, and human-like negotiation agents in e-commerce.