Ionic liquids have become a popular solvent for cellulose pretreatment in biorefineries due to their efficiency in dissolution and their reusability. Understanding the interactions between cations, anions, and cellulose is key to the development of better solvents and the improvement of pretreatment conditions. While previous studies described the interactions between ionic liquids and cellulose fibers, shedding light on the initial stages of the cellulose dissolution process, we study the end state of that process by exploring the structure and dynamics of a single cellulose decamer solvated in 1-butyl-3-methyl-imidazolium chloride (BmimCl) and in water using replica-exchange molecular dynamics. In both solvents, global structural features of the cellulose chain are similar. However, analyses of local structural properties show that cellulose explores greater conformational variability in the ionic liquid than in water. For instance, in BmimCl the cellulose intramolecular hydrogen bond O3H'···O5 is disrupted more often resulting in greater flexibility of the solute. Our results indicate that the cellulose chain is more dynamic in BmimCl than in water, which may play a role in the favorable dissolution of cellulose in the ionic liquid. Calculation of the configurational entropy of the cellulose decamer confirms its higher conformational flexibility in BmimCl than in water at elevated temperatures.