You’re ahead in math. And you make “sloppy” mistakes. I would stick it out in your current class and make a goal of eliminating those silly errors.
Since you like math, why not consider an online class focused on math competition, for example? You can use your ability and the extra bandwidth afforded by your current math class situation to augment, rather than accelerate, your math skills.