Contact tracing is commonly recommended to control outbreaks of COVID-19, but its effectiveness is unclear. Following PRISMA guidelines, we searched four databases using a range of terms related to contact tracing effectiveness for COVID-19. We found 343 papers; 32 were included. All were observational or modelling studies. Observational studies (n = 14) provided consistent, very-low certainty evidence that contact tracing (alone or in combination with other interventions) was associated with better control of COVID-19 (e.g. in Hong Kong, only 1084 cases and four deaths were recorded in the first 4.5 months of the pandemic). Modelling studies (n = 18) provided consistent, high-certainty evidence that under assumptions of prompt and thorough tracing with effective quarantines, contact tracing could stop the spread of COVID-19 (e.g. by reducing the reproduction number from 2.2 to 0.57). A cautious interpretation indicates that to stop the spread of COVID-19, public health practitioners have 2-3 days from the time a new case develops symptoms to isolate the case and quarantine at least 80% of its contacts.