This past year I’ve had the fortune to really get close with my general physics professor, who happened to have his phd in string theory. I’ve asked him for a lot of advice on school, and he’s given me a lot of advice from a physics perspective. If there’s even a chance you want to become a physicist, pick the better ranked school. The field is unbelievably competitive, and you need everything that’ll help you stand out that you can get. Math is also crucial. You need to take some ridiculously hard math courses, but just as electives.
Now, if you don’t want physics, or you just want enough to teach, I don’t think it’s any big deal. From how I understand it, you can get a psychology degree and work in a bank. A background in physics will open up a lot of doors. But I would take the time to look into the programs at each school, professor rankings, etc. Not to bring you down, but it should be stressful. If you want to reach high, you’re going to have to do more than just get your degree. But that’s just the opinion of a guy who got his opinion from a theoretical physicist.