The vision and development of blind source separation (BSS) considering the multi-dimensional components attract growing interests in the research community. The single-set BSS methods cannot address the challenge of aligning the extracted components across different datasets, because they neglect the dependence information across datasets. In this paper, we propose to apply the tensor theory to develop a novel structure for the BSS technique that allows us to discover the multi-set components and meaningful spatio-temporal structures of data. The structure is stacked of covariance matrices with temporal information, in terms of limited storage, using the tensor theory that can efficiently encapsulate and compress large-scale data into a compact format. In addition, we discuss how our results provide unique decomposition under such conditions from the theoretical perspective. The experiments designed on the signal with time variation (multiset), which is not identifiable when each mixture is considered individually. It can be solved to restate as a tensor structure of their spatial and temporal correlation matrices. Compared with the other three classical algorithms, the proposed algorithm consistently shows high accuracy.