As a hot information technique, cloud computing is an excellent choice for building a "smart city". But cloud computing couldn't well balance convenience and privacy. This greatly influences its advantage to play. With the appearance of fully homomorphic encryption (FHE), it is possible for cloud computing to be consistent with privacy. But, the efficiency of FHE schemes is still far from the actual needs. The main reason is the additional noise reduction manipulations which take lots of time. Up to now, the effective FHE schemes are mostly constructed based on the learning with error (LWE), and re-linearization technique is an essential technique to construct LWE-based FHE scheme. Meanwhile it is also the main cause of the noise expansion, which greatly affects the efficiency of LWE-based FHE, although binary decomposition is used to reduce the noise effectively by publishing the “encryptions” of key components. In this paper, we are the first to directly study how to further reduce the noise (caused by the "relinearization" technique) in a natural way. First, we verify that, according to original processing method of "relinearization" technique, the binary is optimal in performance on noise reduction, compared to other scale numbers. Then we propose a new way to use multi-band decomposition for noise reduction. If we choose a quaternary representation as our way, noise can be reduced to about half of the original "relinearization" technique for one homomorphic multiplication, which means less additional noise reduction manipulations needed for same depth of homomorphic evaluation circuit. Moreover, the larger the number of scale number we choose, the better the performance. Besides, we present an algorithm changing a somewhat LWE-based FHE into a leveled LWE-based FHE based on our optimized "relinearization" technique.