Q-learning cell selection for femtocell networks: Single- and multi-user case

Chaima Dhahri, Tomoaki Ohtsuki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

31 Citations (Scopus)


In this paper, we focus on user-centered handover decision making in open-access non-stationary femtocell networks. Traditionally, such handover mechanism is usually based on a measured channel/cell quality metric such as the channel capacity (between the user and the target cell). However, the throughput experienced by the user is time-varying because of the channel condition, i.e. owing to the propagation effects or receiver location. In this context, user decision can depend not only on the current state of the network, but also on the future possible states (horizon). To this end, we need to implement a learning algorithm that can predict, based on the past experience, the best performing cell in the future. We present in this paper a reinforcement learning (RL) framework as a generic solution for the cell selection problem in a non-stationary femtocell network that selects, without prior knowledge about the environment, a target cell by exploring past cells behavior and predicting their potential future state based on Q-learning algorithm. Our algorithm aims at balancing the number of handovers and the user capacity taking into account the dynamic change of the environment. Simulation results demonstrate that our solution offers an opportunistic-like capacity performance with less number of handovers.

Original languageEnglish
Title of host publication2012 IEEE Global Communications Conference, GLOBECOM 2012
Number of pages6
Publication statusPublished - 2012 Dec 1
Event2012 IEEE Global Communications Conference, GLOBECOM 2012 - Anaheim, CA, United States
Duration: 2012 Dec 32012 Dec 7

Publication series

NameGLOBECOM - IEEE Global Telecommunications Conference


Other2012 IEEE Global Communications Conference, GLOBECOM 2012
Country/TerritoryUnited States
CityAnaheim, CA

ASJC Scopus subject areas

  • Electrical and Electronic Engineering


Dive into the research topics of 'Q-learning cell selection for femtocell networks: Single- and multi-user case'. Together they form a unique fingerprint.

Cite this