Dozens of genes are associated with idiopathic hypogonadotropic hypogonadism (IHH) and an oligogenic etiology has been suggested. However, the associated genes may account for only approximately 50% cases. In addition, a genomic systematic pedigree analysis is still lacking. Here, we conducted whole exome sequencing (WES) on 18 unrelated men affected by IHH and their corresponding parents. Notably, one reported and 10 novel variants in eight known IHH causative genes (AXL, CCDC141, CHD7, DMXL2, FGFR1, PNPLA6, POLR3A, and PROKR2), nine variants in nine recently reported candidate genes (DCAF17, DCC, EGF, IGSF10, NOTCH1, PDE3A, RELN, SLIT2, and TRAPPC9), and four variants in four novel candidate genes for IHH (CCDC88C, CDON, GADL1, and SPRED3) were identified in 77.8% (14/18) of IHH cases. Among them, eight (8/18, 44.4%) cases carried more than one variant in IHH-related genes, supporting the oligogenic model. Interestingly, we found that those variants tended to be maternally inherited (maternal with n = 17 vs paternal with n = 7; P = 0.028). Our further retrospective investigation of published reports replicated the maternal bias (maternal with n = 46 vs paternal with n = 28; P = 0.024). Our study extended a variant spectrum for IHH and provided the first evidence that women are probably more tolerant to variants of IHH-related genes than men.