There’s not a wooden nickel’s worth of difference between Duke and Penn when it comes to diversity, so it isn’t a relevant factor in choosing between the two schools.
College Factual gives Duke a little better overall diversity ranking - based on ethnicity, gender, age and geographic location of origin - but I wouldn’t reject Penn based on that alone.