Q-learning based cell selection for UE outage reduction in heterogeneous networks

Cell range expansion (CRE) is a load balancing technique that virtually expands a pico cell range by adding a bias value to the pico received power, instead of increasing transmit power of the pico base station (PBS); It can make cell-edge throughput and overall network throughput improved. CRE disperses the load of macro base stations (MBSs) on PBSs, so that it can reduce the number of UE outages. Although the configuration of the bias values of each user equipment (UE) has potential to reduce UE outages compared with the common bias value configuration among UEs, the common one is applied in the majority of related works for simplicity. In this article, we propose a scheme to select a cell by using Q-learning algorithm where each UE learns to which cell to send a service request to reduce the number of UE outages from its past experience independently. Simulation results show that the proposed scheme has the minimum number of UE outages in the system. Moreover, they show that it reduces the number of UE outages and the required memory size, compared with our previous proposed method.