Physical fitness for young people is viewed as a multidimensional construct, in that it consists of specific components such as strength, mobility, balance, flexibility, and stamina. This study examined whether this structure underlying physical fitness is also relevant to older adults. A 10-item performance test, which was assumed to assess six components of physical fitness, was administered to 69 healthy volunteers ranging in age from 61 to 83 years. A covariance structure model was applied to the test data: the second-order factor was Physical Fitness, and the first-order factors were Strength, Walking, Balance, Flexibility, Stamina, and Manual Speed which were assumed to be measured based on the ten observed variables. Goodness-of-fit index (GFI) of the model was acceptable (GFI = 0.93). While four factors relating to basic motor performances (Strength, Walking, Balance, and Manual Speed) had loadings more than 0.62 to Physical Fitness, Flexibility and Stamina had less than 0.35. It was suggested for elderly people that strength, mobility, balance, and speed components of physical fitness were highly correlated and explainable by a single factor, while flexibility and cardiorespiratory endurance were to be measured by use of specific measures.