The benefit of using virtual puppetry to assist narration in has been proved in current digital storytelling systems. When performing, these systems can provide players with multimode feedback and meaningful immersive learning experiences. However, despite many years of research and effort, combining with complicated HCI and systematic integration procedure, the production of interactive virtual puppetry animation still remains one of the most labour-intensive and tedious tasks. Due to its inherent functional features, a lot of complex technical problems are involved in generation and various aspects of requirements have to be handled, from the HCI technologies to the efficient management of animation data assets. In this paper, a semantic framework is constructed to provide a clear understanding of VR based interactive animation production and promote its generation, which uses ontological methods for modelling the construction of animation generation at an abstract and semantic level. To facilitate animation generation process and improve its reusability and modularity, two domain-specific ontologies are further defined as the ontological implementation of the framework. And finally, a novel digital puppetry storytelling system is designed to assist young children's storytelling, in which hand-gesture-based interactive control and ontology-based intelligent animation data management is used as case study of the proposed semantic framework.