Immunity to influenza viruses can be long-lived, but reinfections with antigenically distinct viral strains and subtypes are common. Reinfections can boost antibody responses against viral strains first encountered in childhood through a process termed "original antigenic sin." It is unknown how initial childhood exposures affect the induction of antibodies against the hemagglutinin (HA) stalk domain of influenza viruses. This is an important consideration since broadly reactive HA stalk antibodies can protect against infection, and universal vaccine platforms are being developed to induce these antibodies. Here we show that experimentally infected ferrets and naturally infected humans establish strong "immunological imprints" against HA stalk antigens first encountered during primary influenza virus infections. We found that HA stalk antibodies are surprisingly boosted upon subsequent infections with antigenically distinct influenza A virus subtypes. Paradoxically, these heterosubtypic-boosted HA stalk antibodies do not bind efficiently to the boosting influenza virus strain. Our results demonstrate that an individual's HA stalk antibody response is dependent on the specific subtype of influenza virus that they first encounter early in life. We propose that humans are susceptible to heterosubtypic influenza virus infections later in life since these viruses boost HA stalk antibodies that do not bind efficiently to the boosting antigen.