The research in the robot vision becomes more and more attractive since the demand for robot is growing. To detect and avoid the obstacles in the outdoor environment is an important task for a moving robot. In this paper, we propose a real-time detection of the obstacle based on the computer vision with single camera. The dense optical flow method is adopted to extract the training data for a classifier model using support vector machine (SVM). The speeded-up robust features method is used to detect the interest points to be verified as the obstacle points or not by a SVM classifier. Moreover, a measurement of the spatial weighted saliency map is proposed to highlight the pixels of the obstacle. Finally, the obstacle points and the saliency map are combined to locate the region of the obstacle. The experimental results show that the proposed algorithm can effectively detect the obstacle in the outdoor environment.