Well, it depends on what you mean by academic quality. I tend to think of schools in groups/buckets instead of absolute rankings. So choosing Georgetown or Boston College over Harvard or Yale, if you prefer one of the former two, probably won’t make a huge difference career-wise. Selecting an excellent public school like Wisconsin or Purdue probably also won’t make a huge difference either. These are excellent places overall.
Where we get into differences is if we’re talking about more regional public universities or smaller private colleges - not simply lesser-known private colleges (as some of those are top quality) but rather ones that aren’t truly comparable to the top schools. And even in those cases, there might be really good reasons to go there - some of those schools have strong local ties to great economies in nearby small or large cities, some of them are specialized in certain areas/fields and have connections in that area, and some of them may provide a really low-cost option for a student who otherwise has few options.
I also think that the balance for each student is going to be different. Some students will value academic quality above all else, but some students will basically look at any good solid school across the board and prioritize other factors.