My understanding is that the graduate ranking of your undergraduate department doesn’t matter very much, as long as it offers the possibility of research opportunities and good recommendation letters.
There are lots of factors to consider if you’re trying to choose a good college for majoring in math.
Some departments focus on pure math; others focus on applied math.
Some colleges are more rigorous than others.
Some colleges offer more research opportunities than others.
Some colleges will give you transfer credit; others won’t.
Some colleges have more courses to choose from. For example, most community colleges don’t give bachelor’s degrees because they don’t have upper-level math classes.