Depends very much on the field, and the competitiveness of the graduate admissions process at the particular program being applied to.
Here is a quote from the PhD admissions FAQ at a program consistently rated #1 in the world in it’s field with an acceptance rate of around 3%:
“Coming from a lesser-known school is not much of a handicap, if other parts of the application are strong.”
“Not much of a handicap” is still a handicap when acceptance rates are in the single digits.