In this paper we provide an auditory processing system with higher biological plausibility than previous studies to solve the problem of acoustic anomaly detection in household environment. First, in the proposed system the log filter bank is adopted for extracting audio features, simulating the function of the peripheral auditory system (outer, middle and inner ears to auditory nerves). Next, we use multi-layer neural networks to imitate auditory cortex in human brains, in order to extract abstract semantic contents. Then, a semantic pointer architecture model is used to imitate prefrontal cortex, basal ganglia, and thalamus, in which the anomaly is detected using symbol-like rules. Compared with other anomaly detection methods with different biological plausibility in performance, our proposed method gets the best result on the testing set, with 0.956 AUC. Meanwhile, it takes less computational time to detect the anomaly. Hence, it is suitable for detecting acoustic anomalies in real-world cases.