OBJECTIVE The objective of this current study was to describe the status of returning to work and work ability of colorectal cancer survivors and identify the key factors associated with returning to work and work ability of Chinese colorectal cancer patients. METHODS A cross-sectional observational study was performed in 212 colorectal cancer survivors who have worked before their colorectal cancer diagnosis. We evaluated patient's return to work (Yes/No), work ability, and factors by questionnaires of the Work Ability Index (WAI), M. D. Anderson Symptom Inventory for Gastrointestinal (MDASI-GI), and the Self-Report Psychosocial Adjustment to Illness Scale (PAIS-SR). Logistic regression analysis and linear regression were used to find the potential predictors with returning to work and work ability. RESULTS Participants mostly 145 have returned to work (68.4%). Work ability and psychosocial adjustment of colorectal cancer survivors were at a moderate level. After completing treatment, the patient still had many symptoms, and these symptoms were distress to live. In the two models, survivors with higher family monthly income per capita and lower psychosocial adjustment scores were more likely to have higher work ability and return to work. Survivors with lower symptom distress were more likely to have higher work ability (r =  - 0.038, p = 0.010). Survivors with higher work ability were more likely to return to work (OR = 1.193, 95% CI = (1.116,1.274)). CONCLUSIONS This study confirmed that symptom distress and psychosocial adjustment were significantly associated with colorectal cancer survivors' returning to work and work ability, which should be considered in future intervention research.