Automated cell classification is an important yet a challenging computer vision task with significant benefits to biomedicine. In recent years, there have been several studies attempted to build an artificial intelligence-based cell classifier using label-free cellular images obtained from an optical microscope. Although these studies showed promising results, such classifiers were not able to reflect the biological diversity of different types of cell. While in terms of malignant cell, it is well-known that intracellular actin filaments are altered substantially. This is thought to be closely related to the abnormal growth features of tumor cells, their ability to invade surrounding tissues and also to metastasize. Therefore, being able to classify different types of cell based on their biological behaviors using automated technique is more advantageous. This article reveals the difference in the actin cytoskeleton structures between breast normal and cancer cells, which may provide new information regarding malignant changes and be used as additional diagnostic marker. Since the features cannot be well detected by human eyes, we proposed the application of convolutional neural network (CNN) in cell classification based on actin-labeled fluorescence microscopy images. The CNN was evaluated on a large number of actin-labeled fluorescence microscopy images of one human normal breast epithelial cell line and two types of human breast cancer cell line with different levels of aggressiveness. The study revealed that the CNN performed better in the cell classification task compared to a human expert.