In this paper, we develop a system for automated human body tracking and modeling based on a monocular camera. In this system, ten body parts including head, torso, arms and legs are extracted to build a 2D human body model. One way to decompose human silhouette into different parts is to generate cuts between the negative minimum curvature (NMC) points. However, due to the self-occlusion problem and left-right ambiguity, each individual body part cannot be successfully identified in every frame. Therefore, in addition to utilizing the NMC points, we design a forward and backward tracking mechanism to identify the location of head in each frame. The torso angle and size are determined by integrating multiple-frame information with the modified solution of Poisson equation. Hands and feet can then be identified correctly based on a modified star skeleton approach along with the nearest-neighbor tracking mechanism. The rest of joint points can also be located by making use of the notion “connectivity”. In the experiments, we analyze the performance of the proposed human body modeling mechanism. We also demonstrate a behavior analysis application by employing the proposed method. The experiment results verify the robustness of the proposed approach and the feasibility of the employing the proposed approach to the action recognition application.