Restorative materials have varying surface characteristics from natural tooth, which may affect oral-bacterial surface attachment/growth. This study examined 48-h Streptococcus mutans (Sm) or Actinomyces naeslundii (An) growth on various restorative materials and tooth surfaces. The quantity and viability of 48-hour-old Sm and An growth on polished (180- or 1200-grits), saliva-coated resin composite (RC), glass ionomer cements (GIC), resin-modified GIC (R-GIC), GIC containing casein phosphopeptide-amorphous calcium phosphate (3% (w/w), CPP-ACP GIC), amalgam or tooth blocks (5 × 5 × 1 mm3 ) were examined. Rough-polished (arithmetical mean deviation of the assessed surface roughness profile (Ra): 1.50-1.75 µm) material surfaces revealed relatively higher proportion of inorganic, positively charged surface components ((Si + Al)/C) and greater quantity of surface attached bacteria than smooth polished (Ra: 0.20-0.35 µm) material groups (P < 0.001). Less Sm and An were observed on tooth, and smooth polished GIC and CPP-ACP GIC surfaces than on resin-based materials (RC, R-GIC) and amalgam (P ≤ 0.003). Viability of Sm was found to be lower on amalgam surfaces (P < 0.001), whereas that of An appeared lower on both amalgam surfaces and rough CPP-ACP GIC surfaces (P ≤ 0.033). Surface roughness exerted a pronounced effect on in vitro growth/attached Sm/An quantity but may not have an impact on bacteria viability. Interestingly, despite smoother surfaces of various materials tested, fewer Sm/An were observed attaching on tooth surfaces.