This paper addresses the exploration problem for a multi-robot system subject to constraints imposed by the communication network. Our goal is to develop a decentralized approach that leads each robot towards the frontier while keeping connectivity with the neighbors. A frontier is a part of the environment, which divides explored from unexplored space. We make use of a graph theory for characterizing the communication, which employs the adjacency and Laplacian matrices of the graph and their spectral properties. The Fiedler value of the Laplacian matrix describes the communication interactions of all the robots in the system. The proposed approach has been implemented and evaluated in several simulated environments and with varying team sizes. The results demonstrate that our decentralized approach can efficiently spread the robots in the environment. As a result, the robots are able to quickly accomplish their exploration mission despite the constraints introduced by the limited connectivity.