Age estimation remains challenging because of its high dependence on small facial changes (based on individual propagation, increased wrinkles, and even racial or gender factors). In the past decade, some learning models of neural network based on image anal-ysis have been rapidly developed to overcome such limitations. In this study, we developed a novel method, namely correlation-refined convolutional neural network (CR-CNN), based on some deep learning model (AlexNet). Additional to the parameters in model, the CR-CNN model considers a specific learning network, in which the neuron parameters along with refined facial features at various field-of-view levels have determined through canonical correlation analysis (CCA). Such a novel learning strategy, called low-to-mid-dle-level-features retained transfer learning (LMLFR). Through LMLFR, the feature maps in CNN would be reorganized and join as new layer. That means the maps with high CCA values, in which neurons have high coadaptation with respect to feature-map values, are averaged and flattened; and contrarily, the maps with low CCA values are retained for the low-coadaptation neurons. All refined layers are then subjected to principal component analysis to further reduce dimensionality. At the output layer, classification is executed through support vector regressions (SVR) and Marginal Fisher Analysis (MFA) to over-come the non-Gaussian distributions of refined features on different layers. Experiments were conducted using images obtained from the well-known MORPH dataset, and the re-sults indicated that for age estimation, the proposed model outperformed commonly used methods; the error range was approximately −5 to +5, covering approximately 80% of the learning age range. The proposed model constitutes a novel approach to feature refinement and can potentially become the basis of extensive applications.