Men with congenital hypogonadotropic hypogonadism (CHH) and Kallmann syndrome (KS) have both low circulating testosterone and estradiol levels. Whether bone structure is affected remains unknown. To characterize bone geometry, volumetric density and microarchitecture in CHH/KS. This cross-sectional study, conducted at a single French tertiary academic medical center, included 51 genotyped CHH/KS patients and 40 healthy volunteers. Among CHH/KS men, 98% had received testosterone and/or combined gonadotropins. High-resolution peripheral quantitative computed tomography (HR-pQCT), dual-energy x-ray absorptiometry (DXA), and measurement of serum bone markers were used to determine volumetric bone mineral density (vBMD) and cortical and trabecular microarchitecture. CHH and controls did not differ for age, body mass index, and levels of vitamin D and PTH. Despite long-term hormonal treatment (10.8 ± 6.8 years), DXA showed lower areal bone mineral density (aBMD) in CHH/KS at lumbar spine, total hip, femoral neck, and distal radius. Consistent with persistently higher serum bone markers, HR-pQCT revealed lower cortical and trabecular vBMD as well as cortical thickness at the tibia and the radius. CHH/KS men had altered trabecular microarchitecture with a predominant decrease of trabecular thickness. Moreover, CHH/KS men exhibited lower cortical bone area, whereas total and trabecular areas were higher only at the tibia. Earlier treatment onset (before age 19 years) conferred a significant advantage for trabecular bone volume/tissue volume and trabecular vBMD at the tibia. Both vBMD and bone microarchitecture remain impaired in CHH/KS men despite long-term hormonal treatment. Treatment initiation during adolescence is associated with enhanced trabecular outcomes, highlighting the importance of early diagnosis.