The Internet is disseminating more and more information as it continues to grow. This large amount of information, however, can cause an information overload problem for users. Recommender systems to help predict user preferences for new information can ease users’ mental loads. The model-based collaborative filtering (CF) approach and its variants for recommender systems have recently received considerable attention. Nonetheless, two issues should be carefully considered in practical applications. First, the data reliability of the rating matrix can affect the prediction performance. Second, most current models view the measurement scale of output classes as nominal instead of ordinal ratings. This study proposes a model-based CF approach that deals with both issues. Specifically, this approach uses the Hebbian learning rule to facilitate singular value decomposition in reducing noisy and redundant data, and employs support vector ordinal regression to build up the models. The results of the experiments conducted in this study show that the proposed approach outperforms other methods, especially under data of mild data sparsity and large-scale conditions. The feasibility of the proposed approach is justified accordingly.