Recombination of antibody genes in B cells can involve distant genomic loci and contribute a foreign antigen-binding element to form hybrid antibodies with broad reactivity for Plasmodium falciparum. So far, antibodies containing the extracellular domain of the LAIR1 and LILRB1 receptors represent unique examples of cross-chromosomal antibody diversification. Here, we devise a technique to profile non-VDJ elements from distant genes in antibody transcripts. Independent of the preexposure of donors to malaria parasites, non-VDJ inserts were detected in 80% of individuals at frequencies of 1 in 104 to 105 B cells. We detected insertions in heavy, but not in light chain or T cell receptor transcripts. We classify the insertions into four types depending on the insert origin and destination: 1) mitochondrial and 2) nuclear DNA inserts integrated at VDJ junctions; 3) inserts originating from telomere proximal genes; and 4) fragile sites incorporated between J-to-constant junctions. The latter class of inserts was exclusively found in memory and in in vitro activated B cells, while all other classes were already detected in naïve B cells. More than 10% of inserts preserved the reading frame, including transcripts with signs of antigen-driven affinity maturation. Collectively, our study unravels a mechanism of antibody diversification that is layered on the classical V(D)J and switch recombination.