Chitosan is obtained from the deacetylation of chitin, and it is known to possess antimicrobial activity. It has attracted attention as it may be used for treating infections caused by different types of microorganisms due to its broad spectrum. Its application in the form of micro- or nanoparticles (CM/CN) has expanded its usage, as in this form, it retains its activity, and remain stable in aqueous solutions. However, inconsistencies in the results reported by different authors have been identified. In this communication, the antimicrobial activity of CN produced from different starting materials was tested against Listeria monocytogenes. It was observed that, even though all the starting materials were reported to have a molecular weight (MW) below 200 kDa and degree of deacetylation (DD) > 75%, the size of the CNs were significantly different (263 nm vs. 607 nm). Furthermore, these differences in sizes exerted a direct effect on the antimicrobial properties of the particles, as when testing the ones with the smallest size, i.e., 263 nm, a lower Minimum Inhibitory Concentration (MIC) was achieved, i.e., 0.04 mg/mL. Even though the largest particles, i.e., 607 nm, in individual experiments were able to achieve an MIC of 0.03 mg/mL, the results with CN presented great variation among replicates and up to 0.2 mg/mL were needed in other replicates. The starting material has a critical impact on the properties of the CN, and it must be carefully characterized and selected for the intended application, and MW and DD solely do not fully account for these properties.