OSU is a huge sleeper (i.e. underrated) for math in my opinion. I obviously don’t know your financial situation but a Stamps scholarship to OSU would be quite tempting to me.
https://reach4.college/wp-content/uploads/2023/07/IMO-Contestant-Matriculations-tiny.png - fun fact, OSU has at least one IMO gold medalist alumnus.
More relevantly, they have a rigorous honors track (Math Honors Track (without ASC Honors Contract) | Department of Mathematics) in their math department and appear quite flexibile with placement: most courses have “…or instructor permission” after the prerequisites, and that language seems to also be the case for the physics courses. While honors versions of freshman courses are common, OSU uniquely (in my experience) has honors versions of core physics courses (EM and quantum). And the math department has some relevant courses too, like 5451, 5756, 5757.
And Cambridge is an interesting option - you can study maths for the first year and then continue with Physics for the following years, or go through maths all 3/4 years - a lot of Cambridge maths courses are directly related to physics (e.g. https://www.maths.cam.ac.uk/undergrad/files/coursesIB.pdf for a list of year 2 courses ), making even the maths degree a strong option for theoretical physics. One downside is the fact that research opportunities are quite a bit scarcer for undergrads, but that’s not as big of a deal for maths as it is for physics.