Non-local processing based denoising methods obtain high quality outputs. However, they have high calculation costs due to the non-local processing. In this paper, we focus on the bilateral filter instead of non-local processing. Although bilateral filter is a traditional local processing filter which can preserve edges, it cannot remove strong noise. Therefore, we utilize the joint bilateral filter, which calculates filter coefficients using another reference image. For adaptation to single image denoising, we utilize a pre-estimated image by the hard threshold operation in 2D frequency domain. To obtain a pre-estimated noiseless image while preserving image details, we apply multi-sized hard threshold operation and aggregate their outputs. Moreover, we change the estimation of the joint bilateral filter coefficients whether a target pixel is in an edge regions or not. By these improvements, our proposed method removes noise with preserving image details in a low calculation cost and experimental results show our method has comparable quality to non-local denoising methods.