A better understanding of the anti-SARS-CoV-2 immune response is necessary to finely evaluate commercial serological assays but also to predict protection against reinfection and to help the development of vaccines. For this reason, we monitored the anti-SARS-CoV-2 antibody response in infected patients. In order to assess the time of seroconversion, we used 151 samples from 30 COVID-19 inpatients and monitored the detection kinetics of anti-S1, anti-S2, anti-RBD and anti-N antibodies with in-house ELISAs. We observed that specific antibodies were detectable in all inpatients 2 weeks post-symptom onset and that the detection of the SARS-CoV-2 Nucleocapsid and RBD was more sensitive than the detection of the S1 or S2 subunits. Using retroviral particles pseudotyped with the spike of the SARS-CoV-2, we also monitored the presence of neutralizing antibodies in these samples as well as 25 samples from asymptomatic individuals that were shown SARS-CoV-2 seropositive using commercial serological tests. Neutralizing antibodies reached a plateau 2 weeks post-symptom onset and then declined in the majority of inpatients but they were undetectable in 56% of asymptomatic patients. Our results indicate that the SARS-CoV-2 does not induce a prolonged neutralizing antibody response. They also suggest that induction of neutralizing antibodies is not the only strategy to adopt for the development of a vaccine. Finally, they imply that anti-SARS-CoV-2 neutralizing antibodies should be titrated to optimize convalescent plasma therapy.