The (much) higher rate of sexual assault against people under 30 vs. middle-aged and elderly people suggests that the sex part of sexual assault does matter. People who want to steal seek out elderly victims as easy pickings. People who want to rape apparently have the opposite targets.
https://www.rainn.org/statistics/victims-sexual-violence
Those aged 65 and older are 92% less likely than 12-24 year olds to be a victim of rape or sexual assault, and 83% less likely than 25-49 year olds.