JISE

[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24]

Journal of Information Science and Engineering, Vol. 27 No. 3, pp. 819-834

Explicit Use of Term Occurrence Probabilities for Term Weighting in Text Categorization

ZAFER ERENEL, HAKAN ALTINCAY AND EKREM VAROGLU
Department of Computer Engineering
Eastern Mediterranean University
Famagusta, Northern Cyprus via Mersin 10, Turkey
E-mail: {zafer.erenel; hakan.altincay; ekrem.varoglu}@emu.edu.tr

In this paper, the behaviors of leading symmetric and asymmetric term weighting schemes are analyzed in the context of text categorization. This analysis includes their weighting patterns in the two dimensional term occurrence probability space and the dynamic ranges of the generated weights. Additionally, one of the newly proposed term selection schemes, multi-class odds ratio, is considered as a potential symmetric weighting scheme. Based on the findings of this study, a novel symmetric weighting scheme derived as a function of term occurrence probabilities is proposed. The experiments conducted on Reuters-21578 ModApte Top10, WebKB, 7-Sectors and CSTR2009 datasets indicate that the proposed scheme outperforms other leading schemes in terms of macro- averaged and micro-averaged F1 scores.

Keywords: text categorization, supervised term weighting, symmetric schemes, term occurrence probabilities, support vector machines

Retrieve PDF document (JISE_201103_02.pdf)