The Telecare Medicine Information System (TMIS) has brought us a lot of conveniences. However, it may also reveal patients’ privacies and other important information. So the security of TMIS can be paid much attention to, in which identity authentication plays a very important role in protecting TMIS from being illegally used. To improve the situation, TMIS needs a more secure and more efficient authentication scheme. Recently, Yan and Li et al. have proposed a secure authentication scheme for the TMIS based on biometrics, claiming that it can withstand various attacks. In this paper, we present several security problems in their scheme as follows: (a) it cannot really achieve three-factor authentication; (b) it has design flaws at the password change phase; (c) users’ biometric may be locked out; (d) it fails to achieve users’ anonymous identity. To solve these problems, a new scheme using the theory of Secure Sketch is proposed. The thorough analysis shows that our scheme can provide a stronger security than Yan-Li’s protocol, despite the little higher computation cost at client. What’s more, the proposed scheme not only can achieve anonymity preserving but also can achieve session key agreement.