Minimum MSE based regularization for system identification in the presence of input and output noise

The corrected least squares (CLS) approach gives a consistent estimate of a system model in the presence of input and output noises. However, when the input signal is band-limited or strongly correlated, and/or a transfer function model is identified by using an overdetermined model, the CLS estimate often becomes ill-conditioned. To overcome this problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate. The asymptotic MSE can be evaluated by considering the third and fourth cross moments of the input and output noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using only accessible input-output data. The relationship between the regularization using multiple parameters and the truncation of small eigenvalues is investigated and then it is clarified that the proposed regularization scheme is also efficient to decide the order of a transfer function model.