JISE


  [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]


Journal of Information Science and Engineering, Vol. 31 No. 2, pp. 641-658


Factors Influencing the Performance of Human Computation: An Empirical Study in Web Application Testing


TSUNG-JU LEE1, SHIAN-SHYONG TSENG2 AND CHIH-CHUN KUNG3,+ 
1Department of Information Management 
National Penghu University of Science and Technology 
Penghu, 880 Taiwan 2Department of Applied Informatics and Multimedia 
Asia University 
Taichung, 413 Taiwan 
3Institute of Poyang Lake Eco-economics 
Jiangxi University of Finance and Economics 
Nanchang, 330013 P.R. China


    Human computation tackles many difficult-to-automate problems. However, the involvement of human solvers in computation process leads to unstable performance, especially when solvers recruited via Internet. By considering human’s attributes and computational situation, the research model of factors of affecting the effectiveness and efficiency of human computation algorithm is proposed in this paper, together with seven proxies of human’s attributes and computational situation. For testing the proposed model, we collect data from an implemented Web application testing system using a crowdsourcing approach. There are 87 volunteers recruited via social network and the corresponding user-session data. The results show that more human solvers with lower ability level, higher engagement, more personal bias and lower difficulty level of tasks lead to shorter completion time. Human solvers with higher ability level, higher engagement, more personal bias and lower difficulty level of tasks lead to more accurate output, but number of solvers does not show significant correlation with the correctness of human computation algorithm.


Keywords: human computation, crowdsourcing, social computing, human attributes, computational situation, latent class model

  Retrieve PDF document (JISE_201502_16.pdf)