Since COVID-19 broke out, there has been renewed interest in understanding the economic and social dynamics of historical and more recent epidemics and pandemics, from the plagues of Antiquity to modern-day outbreaks like Ebola. These events can have significant impacts on the interplay between poverty and social cohesion, i.e. how different groups in society interact and cooperate to survive and prosper. To that effect, this paper provides a theory-driven overview of how social responses to past epidemics and pandemics were determined by the epidemiological and non-epidemiological characteristics of these outbreaks, with a particular focus on the conditions giving rise to scapegoating and persecution of minority groups, including migrants. We discuss existing theories as well as historical and quantitative studies, and highlight the cases where epidemics and pandemics may lead to milder or more severe forms of scapegoating. Finally, we conclude with a summary of priorities for future research on epidemics, pandemics and social conflict and discuss the possible effects and policy implications of COVID-19.