Citrus fruits are a specialty of the subtropical region and are currently the most widely cultivated and produced fruits in Taiwan. They contain a wealth of vitamins and minerals, including vitamin C, potassium, magnesium, and more, offering anti-inflammatory and antioxidant benefits. Citrus plants are among the economically significant fruits in Taiwan, and there are several citrus species that share a similar appearance. We have constructed a database containing the four most commonly purchased citrus varieties in the market. This database comprises a total of 1379 original images, which have been expanded to 7584 images using six different data augmentation methods. We have chosen three Convolutional Neural Network (CNN) models that have achieved an accuracy rate exceeding 95 % in classifying the four varieties of citrus fruits.