Individuals with subjective cognitive decline (SCD) are at high risk of developing preclinical or clinical state of Alzheimer's disease (AD). Resting state functional magnetic resonance imaging, which can indirectly reflect neuron activities by measuring the blood-oxygen-level-dependent (BOLD) signals, is promising in the early detection of SCD. This study aimed to explore whether the nonlinear complexity of BOLD signals can describe the subtle differences between SCD and normal aging, and uncover the underlying neuropsychological implications of these differences. In particular, we introduce amplitude-aware permutation entropy (AAPE) as the novel measure of brain entropy to characterize the complexity in BOLD signals in each brain region of the Brainnetome atlas. Our results demonstrate that AAPE can reflect the subtle differences between both groups, and the SCD group presented significantly decreased complexities in subregions of the superior temporal gyrus, the inferior parietal lobule, the postcentral gyrus, and the insular gyrus. Moreover, the results further reveal that lower complexity in SCD may correspond to poorer cognitive performance or even subtle cognitive impairment. Our findings demonstrated the effectiveness and sensitiveness of the novel brain entropy measured by AAPE, which may serve as the potential neuroimaging marker for exploring the subtle changes in SCD.