This paper proposes a real-time traffic surveillance system for the detection, recognition, and tracking of multiple vehicles in roadway images. Moving vehicles can be automatically separated from the image sequences by a moving object segmentation method. Since CCD surveillance cameras are typically mounted at some distance from roadways, occlusion is a common and vexing problem for traffic surveillance systems. The segmentation and recognition method uses the length, width, and roof size to classify vehicles as vans, utility vehicles, sedans, mini trucks, or large vehicles, even when occlusive vehicles are continuously merging from one frame to the next. The segmented objects can be recognized and counted in accordance with their varying features, via the proposed recognition and tracking methods. The system has undergone roadside tests in Hsinchu and Taipei, Taiwan. Experiments using complex road scenes under various weather conditions are discussed and demonstrate the robustness, accuracy, and responsiveness of the method.