In this paper, we propose a novel salient object detection approach, which aims in suppressing distractions caused by the small scale pattern in the background and foreground. First, we employ a structure extraction algorithm as a pre-processing step to smooth the textures, eliminate high frequency components and retain the image’s main structure information. Second, we segment the texture-suppressed image into perceptually homogenous regions. Third, two saliency feature maps are computed and fused according to the color contrast and center prior cues. To better exploit each pixel’s color and position information, we refine the fused saliency map. Experiments on two popular benchmark datasets demonstrate that our proposed approach achieves state-of-the-art performance compared with sixteen other state-of-the-art methods in terms of three popular evaluation measures, i.e., Precision and Recall curve, Area Under ROC Curve and F-measure value.