It is a long way between being a janitor and having a satisfying career with a Mathematics degree. In any case, my point is that you shouldn’t count on the job after a second degree solving the issues you have mentioned. You should probably see what you can do to make yourself more compatible with a wider variety of jobs. None of this addresses the issue of getting another degree, of course. You should definitely go for the one which interests you most.