JISE


  [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]


Journal of Information Science and Engineering, Vol. 32 No. 5, pp. 1145-1159


Category Discrimination Based Feature Selection Algorithm in Chinese Text Classification


JUNKAI YI, GUANG YANG AND JING WAN 
College of Information Science and Technology 
Beijing University of Chemical Technology 
Beijing, 100029 P.R. China 
E-mail: {yijk; wanj}@mail.buct.edu.cn; jensen-yg@163.com


    How to improve the classification precision is a major issue in the field of Chinese text classification. The tf-idf algorithm is a classic and widely-used feature selection algorithm based on VSM. But the traditional tf-idf algorithm neglects the feature term¡¦s distribution inside category and among categories, which causes many unreasonable selective results. This paper makes an improvement to the traditional tf-idf algorithm through the introduction of the concept of Category Discrimination. We evaluate our algorithm with experiments, and make comparisons with other algorithms. The experimental results show that the improved tf-idf algorithm consistently has a higher precision and recall compared with the traditional tf-idf algorithm, and is superior to other algorithm as a whole. Therefore, it is a more effective feature selection algorithm in text classification field.


Keywords: text classification, text categorization, feature selection, tf-idf, category discrimination

  Retrieve PDF document (JISE_201605_02.pdf)