Deep learning methods have been used to reduce the number of unnecessary breast biopsies. In this study, an accurate hybrid rule-based fuzzy system with transfer learning is developed to classify breast abnormalities as malignant or benign by calculating breast cancer risk from digital mammogram images. Our system consists of three phases: (i) data augmentation methods (e.g., traditional methods, Generative Adversarial Networks (GANs)); (ii) classification of the breast abnormalities on the BCDR-D02 and mini-MIAS databases by fine-tuning transfer learning methods with the deep learning base model Convolutional Neural Network (CNN); and (iii) calculation of breast cancer risk with a rule-based fuzzy system using the results of the second phase to improve the classification of breast abnormality results. Using our CNN baseline model and traditional extension methods, we achieve 64% and 82% accuracy for mini-MIAS and BCDR-D02, respectively. With fine-tuning the transfer learning methods, we obtain 80% and 83% with VGG-16 for mini-MIAS and BCDR-D02, respectively. Using the rule-based fuzzy system, called the risk method, we achieve the highest results for mini-MIAS (93%) and BCDR-D02 (94%). The classification results of our risk method are compared with the other transfer learning and baseline methods, and it is found that the accuracy of breast abnormality classification is improved by using a hybrid rule-based fuzzy system with transfer learning. Our study can serve as a guide that provides useful tips to researchers in the field of breast cancer classification to develop more effective and reliable studies.