Hidden conditional random fields (HCRFs) are derived from the theory of conditional random fields with hidden-state probabilistic framework. It directly models the conditional probability of a label sequence given observations. Compared to hidden Markov models, HCRFs provide a number of benefits in the acoustic modeling of speech signals. Prior works for training on HCRFs were accomplished with gradient descent based algorithms by conditional maximum likelihood criterion. In this paper, we extend that methodology by applying minimum classification error criterion-based training technique on HCRFs. Specifically, we adopt generalized probabilistic descent (GPD)- based training algorithm with HCRF framework to improve the discrimination capabilities of acoustic models for speech and speaker recognition. Two tasks including a speaker identification and a Mandarin continuous syllable recognition are applied to evaluate the proposed approach. We present the results on the MAT2000 database and these results confirm that the HCRF/GPD approach has good capabilities for speech recognition and speaker identification regardless of the length of the test and training speech or the presence of noise. We note that the HCRF/GPD enjoys its potential for development in acoustic modeling.