They are pretty different so you should be able to make your choice based not he differences between those two excellent but rather different schools.
Penn has its own distinct campus but it is also within a major city with a lot of offerings (theater, restaurants, bars, games, etc). Duke is more isolated but the campus is more classically beautiful.
Penn is overall ranked higher in most fields (including medicine and neuroscience), but the differences at this level are not that important.
Penn is more diverse and has a northeast vibe, Duke is more homogeneous and has a southern vibe.
I dont think the rigor or grade inflation is very different between the schools.
The med school acceptance rate is similar for both schools (high 70%s)