A twin brother can reduce his female twin's chances of having children
Women were 25% less likely to have children if their twin was male, the study, in Proceedings of the National Academy of Sciences, concluded.
Although other factors could play a part - the women were less likely to marry - the team blamed exposure in the womb to the male hormone testosterone.
Experts have agreed testosterone might potentially damage female fertility.
They said animal work supported this.
But they said more work was needed to look at human mechanisms.
Both testosterone and the female hormone oestrogen can cross the womb.
A female twin foetus is therefore exposed to a brother's testosterone and a male twin foetus to a sister's oestrogen.
However, male and female foetuses have similar oestrogen levels, so a female is more likely to be affected, according to Dr Virpi Lummaa's team.
Experts already know certain characteristics, including facial features, can be changed by exposure to sex hormones from opposite sex foetuses.
To investigate the effect testosterone might have on fertility, the researchers looked back at Finnish medical records covering the years 1734 to 1888.
They chose this pre-industrial population because they argue fertility data on modern Western societies would be skewed by advanced healthcare and assisted conception treatments, such as IVF.
Of 754 twins, females with a twin brother were 25% less likely to have children than females with a twin sister.
Women with a male twin were also 15% less likely to marry.
Dr Lummaa's team offer several explanations. Firstly, females exposed to a male twin can have masculine traits, attitudes and behaviours, therefore affecting their decision to get married or a male's attractiveness to them.
Secondly, exposure to high levels of testosterone in the womb increases the risk of diseases that compromise fertility, such as reproductive cancers.
Dr Lummaa said: "As a consequence of a male twin's influence on a female's fertility, mothers who produce opposite sex twins have fewer grandchildren and hence lower evolutionary fitness."
Dr Laurence Shaw, a fertility expert at the London Bridge Fertility Centre and spokesman for the British Fertility Society, said: "There is some evidence to support this observation.
"Exposure to testosterone in the womb in sheep recreates a similar syndrome to a condition called PCOS which is a known cause of infertility in humans.
"Work is now needed to explore what mechanisms could explain these observations in humans."
Male twins can reduce their sisters' fertility
Twin brothers make women less fertile