I went to Duke for undergrad and spent a fair amount of time at Penn over the last several years for dissertation research. As you’ve no doubt realized, the two share many similarities, and I think the same student could very easily be happy at either school.
Do you have a preference in regard to campus setting? Penn’s campus is very nice, especially for an urban school, but it’s noticeably more compact than Duke, which has a sprawling, leafy campus that appeals to some and is too spread out for others. Durham has gentrified considerably over the last couple of decades, and it’s a great town for young people, but it’s obviously much smaller than Philly.