It really depends on the fields. In humanities, the Ivies are self-obsessed and insular. They recruit grad students and hire faculty almost exclusively from other Ivies. In STEM it’s generally different (though, as pointed out, Math often behaves like humanities). In STEM, graduate programs look at the reputation of a program within the field. I am in ecology, and back when I was a grad student, UC Davis had a lot more cachet in the field than UCLA. A CS undergraduate from Notre Dame would be considered far less desirable than a CS undergrad from U Washington.
Basically, admissions committees in STEM trust some programs to prepare students more than others, but that is based on the reputation of the program, not the reputation of the university as a whole.
As @bluebayou pointed out, letter of recommendation are really important, and the bigger the name, the better the letter. However, it is extremely difficult to get famous people at prestigious universities to write a personal LoR for a lowly undergraduate. Moreover, in STEM, there are often Big Names in smaller schools, because of geography or climate.
There are plenty of public universities which are high on the lists of STEM PhD programs. For engineering, UIUC, UMich, Berkeley, and GTech are just a few universities whose graduates would be welcomed into any engineering graduate school in the country.
However, admissions are Holistic, and accomplishments can compensate for undergraduate institute. So a graduate of a relatively low ranked program who has published a peer-reviewed article as an undergraduate will get attention, as will a student who worked with a big name in the field, who wrote a glowing LoR.