My bigger concern isn’t the grade but your ability to absorb - because if you get to the next level and you aren’t ready, then it’s a problem. A B is not worth dropping - but most importantly, don’t get ahead of yourself and it sounds like you might be - but if you can work with the teacher and get a B, I’d stay.
Happened to my daughter - she overshot after an A in Calc AB and had to drop Calc in college and now has a W.