As in other socially monogamous species, pair-bonded humans commonly engage in sex with a partner other than their primary mate. For men, extrapair mating is straightforwardly explained from an adaptive perspective in terms of the reproductive benefits of multiple mates. For women, whose reproductive output is limited by their reproductive biology rather than by their number of mates, the adaptive benefits of extrapair mating are less obvious. Dominant adaptive explanations focus on women obtaining genetic benefits for their offspring by mating with high-quality extrapair partners. Non-adaptive explanations have rarely been considered in humans, but recent findings in birds suggest that females' predisposition to extrapair mating may result from indirect selection, via direct selection on males and a between-sex genetic correlation. To examine the plausibility of this non-adaptive explanation of extrapair mating in women, we used data on recent extrapair mating in 7,378 Finnish twins and their siblings. Genetic modelling showed within-sex broad-sense heritability—i.e. the percentage of variation in extrapair mating due to genetic variation—of 62% in men and 40% in women. There was no between-sex correlation in extrapair mating, making indirect selection unlikely. Based on previous animal and human findings, we also tested for association of the arginine vasopressin receptor 1A gene (AVPR1A) and oxytocin receptor gene (OXTR) with extrapair mating. We found gene-based association for AVPR1A in women but not in men, and OXTR showed no significant association in either sex. Overall, these findings confirm genetic underpinnings of extrapair mating in humans, but do not suggest that women's predisposition to extrapair mating is due to selection on men.