In recent years, highly oxygenated organic molecules (HOMs) derived from photochemical reactions of α-pinene, the most abundant monoterpene, have been considered as important precursors of biogenic particles. However, the specific reactions of HOMs remain largely unknown, especially the corresponding formation and nucleation mechanism in the nanoscale. In this study, we implemented quantum chemical calculations and molecular dynamics (MD) simulations to explore the mechanism of the formation of HOM monomers/dimers by ozonolysis and autoxidation of α-pinene. Furthermore, we investigated the mechanisms of HOMs with different oxygen-to‑carbon (O/C) ratios and functional groups participating in neutral and ion-induced nucleation. The results show that the formation of HOMs is hardly affected by water, sulfuric acid and ions. In the ion-induced nucleation, HOM can dominate the initial nucleation steps; however, in the neutral nucleation, HOMs are more likely to participate in the growth stage. In addition, the nucleation ability of HOM has a bearing on the O/C ratio and the types of the functional groups. The current calculations provide valuable insight into the formation mechanism of the pure organic particles at low sulfuric acid concentrations.