It’s not just grad schools. It’s fellowships and other “merit-based” awards. Like the Rhodes and Marshall fellowship. Of the 2024 winners of these awards, 14 are from Harvard, 7 are from Yale, and 1 is from UVA. In 2023, 11 were from Harvard, 8 were from Yale and 0 were from UVA. For the Putnam competition, it’s always MIT, Princeton, Harvard. UVA does not rank.
These are the kinds of data that leads many people to rate HYP schools highest. Either you believe that these fellowships are merit-based, in which case the top students by a huge margin attend these schools, or you believe that the “elite fix is in”, in which case there is value in being on the right side of that fix.
Yes, there are many mediocre students (many of the athletic recruits and some legacies) at these schools, though Yale’s legacy preference is relatively low. OP doesn’t seem like one of those mediocre students, and it would seem that Yale gives OP the better chance to be noticed and fly high with other high fliers.