New automated whole breast ultrasound (ABUS) machines have recently been developed and the ultrasound (US) volume dataset of the whole breast can be acquired in a standard manner. The purpose of this study was to develop a novel computer-aided diagnosis system for classification of breast masses in ABUS images. One hundred forty-seven cases (76 benign and 71 malignant breast masses) were obtained by a commercially available ABUS system. Because the distance of neighboring slices in ABUS images is fixed and small, these continuous slices were used for reconstruction as three-dimensional (3-D) US images. The 3-D tumor contour was segmented using the level-set segmentation method. Then, the 3-D features, including the texture, shape and ellipsoid fitting were extracted based on the segmented 3-D tumor contour to classify benign and malignant tumors based on the logistic regression model. The Student's t test, Mann-Whitney U test and receiver operating characteristic (ROC) curve analysis were used for statistical analysis. From the Az values of ROC curves, the shape features (0.9138) are better than the texture features (0.8603) and the ellipsoid fitting features (0.8496) for classification. The difference was significant between shape and ellipsoid fitting features (p = 0.0382). However, combination of ellipsoid fitting features and shape features can achieve a best performance with accuracy of 85.0% (125/147), sensitivity of 84.5% (60/71), specificity of 85.5% (65/76) and the area under the ROC curve Az of 0.9466. The results showed that ABUS images could be used for computer-aided feature extraction and classification of breast tumors.