These are two great schools. Choose the one that fits you best.
Penn is also about 50% larger (23K vs. 15k enrollment), with a top ranked medical school for both research and primary care. They also tend to accept a significant number of their own graduates into their graduate programs, if you have any interest in that.
The students at both tend to be quite sociable for elite school students.
A preference toward Northern or Southern culture may also make you lean toward one or the other.
Also, in addition to Philly, Penn is only about an hour by train from NYC.
If you are into pro sports, choose Penn. If you prefer college sports, choose Duke.