^ If you think that Penn (or Brown, or Dartmouth, or Cornell) is drawing students away from HYPSM in large numbers you are mistaken. Duke is interested in drawing students away from HYPSM. These students are the ones who go on to win Rhodes scholarships and cement a university’s reputation. Maybe Penn is not interested in those things but Duke has always been committed to excellence 
Also, Duke may not have Oxford’s international reputation but it is wealthier, more selective (although this is a US vs UK issue), has more highly cited researchers, and more home grown Nobel laureates in the last decade.
^ The number of other schools that can make a similar claim: 3 (Stanford, Harvard and MIT)
http://hcr.stateofinnovation.thomsonreuters.com/
http://www.nobelprize.org/nobel_prizes/lists/universities.html