There is a LOT of math in a physics major, and jobs in the field often require a degree beyond a 4 year undergrad degree. Not to discourage you from going back to school at all, but saying that you may want to have a backup major in mind in case physics doesn’t work out.