Recently, hybrid CPU/GPU cluster has drawn much attention from the researchers of high performance computing because of amazing energy efficiency and adaptable resource exploitation. However, the programming of hybrid CPU/GPU clusters is very complex because it requires users to learn new programming interfaces such as CUDA and OpenCL, and combine them with MPI and OpenMP. To address this problem, we propose a novel OpenMP toolkit called HyCOMP (Hybrid Cluster OpenMP) for hybrid CPU/GPU clusters in this paper. This toolkit is developed based on a novel page-based distributed shared memory system called SUM (software unified memory) which is aimed at emulating a virtual shared memory space over distributed CPUs and GPUs. Compared to traditional page-based DSM systems, SUM can effectively prevent GPUs from performance degradation caused by the latency of handling an enormous number of page faults coming from host-to-device memory copies. Moreover, HyCOMP can automatically achieve load balance of heterogeneous processors. Consequently, HyCOMP dramatically reduces the programming complexity of hybrid CPU/GPU clusters while simultaneously maintains the execution performance of user programs.