Recent studies on speaker verification show total variability space (TVS) based approaches followed by Gaussian probabilistic linear discriminant analysis (GPLDA) are effective in dealing with convolutional noises (such as channel noise), even with additive noises. However, issues arise owing to the various types of noise that are unseen and non-stationary in real-world applications. To remove these noises, we introduce robust principal component analysis (RPCA) into a TVS modeled speaker verification system, called the RPCA-TVS. In which the noise spectrum is considered as the low-rank component and the speech spectrum as the sparse component in the short-time Fourier transform domain. The aim of this paper is to improve the robustness of speaker verification under additive noisy environments, particularly for non-stationary and unseen noises. Experimental results demonstrate that the proposed RPCA-TVS performs better than the competing methods at various signal to noise ratio levels. In particular, the RPCA-TVS reduces the equal error rate (EER) by 4.7% on the whole, compared with the multi-condition system, under the six additive noise conditions at the SNR of 5, 10, and 25 dB.