The influence of genetic interactions (epistasis) on the genetic variance of quantitative traits is a major unresolved problem relevant to medical, agricultural, and evolutionary genetics. The additive genetic component is typically a high proportion of the total genetic variance in quantitative traits, despite that underlying genes must interact to determine phenotype. This study estimates direct and interaction effects for 11 pairs of Quantitative Trait Loci (QTLs) affecting floral traits within a single population of Mimulus guttatus. With estimates of all 9 genotypes for each QTL pair, we are able to map from QTL effects to variance components as a function of population allele frequencies, and thus predict changes in variance components as allele frequencies change. This mapping requires an analytical framework that properly accounts for bias introduced by estimation errors. We find that even with abundant interactions between QTLs, most of the genetic variance is likely to be additive. However, the strong dependency of allelic average effects on genetic background implies that epistasis is a major determinant of the additive genetic variance, and thus, the population's ability to respond to selection.