The duration and spectral dynamics of speech signal are modeled as the duration highorder hidden Markov model (DHO-HMM). Both the state transition probability and output observation probabilities depend not only on the current state but also several previous states. Recursive formulas have been derived for the calculation of the log-likelihood score of optimal partial paths. The high-order state is expanded into several equivalent firstorder states and the token passing algorithm is used to implement an extended Viterbi decoding algorithm on our DHO-HMM continuous speech recognition system. Experimental results on continuous Mandarin digit recognition show that DHO-HMM can improve the recognition accuracy.