In this paper, a novel scheme for face recognition or authentication is proposed against pose, illumination, and expression (PIE) variation using modular face features. A sub-image in low-frequency sub-band is extracted by a wavelet transform (WT) to reduce the image dimensionality. It is partitioned into four parts for representing the local features and reducing the PIE effects, and the small image in a coarse scale is generated via the WT without losing the global face features. Five modular feature spaces were constructed. The most discriminative common vectors in each feature space were found, and a nearest feature space-based (NFS-based) distance was calculated for classification. Finally, a weighted summation is performed to fuse the five distances. Experiments were conducted to show that the proposed scheme is superior to other methods in terms of recognition and authentication rates.