The recent development in deep learning allows us to develop a computer visionbased system for recognition, detection, and localization of nutrients deficiency in fruits. Due to the time constraints, it is important to use an optimized and fast system for fruit quality inspection. In this paper, the input is taken as an image. A deep learning-based method extracts low level and high-level features such as edges, geometrical, statistical, texture, intensity, etc. After validation of the system with the test data, the output is predicted by the system. The processing time is optimized by avoiding fully connected layers which further minimize the requirement of neurons in the network. The convolutional neural network extracts the features of the fruits, Rectified Linear Unit (ReLu) removes the non-fruit pixels. Pooling shrinks, the image by selecting the maximum value of the pixel. The process is repeated until the size of the image is at the desired level. The aim is to identify the objects and recognize them. The foreground region objects are of our interest and being segmented for higher-level image processing. The proposed system attains the accuracy of 99.30 % with the processing time of 3.207 sec.