Major change and school change?

I am currently a Junior and a math major, and last semester was the first semester that I took a lot of math related courses. I barely scraped by and throughout the whole semester I had trouble engaging myself in the material and studying. I used to think that math was for me, but after taking these classes I can’t imagine myself making this my career. I want to change my major to Forestry or possibly Environmental Science. However, in order to do that I would have to transfer schools and I’ve already transferred schools twice. Would it hurt to transfer again, in terms of financial aid or how it will look on a resume, and what’s the best technique to discover whether I should change majors or just stick out the math path? Any advice lightly related to this would help.