I don’t think any of us can give you a fully informed opinion on what action to take but I can provide a personal anecdote (which I typically try to avoid) that seems relevant.
My kid was in a virtually identical situation to the one you describe his sophomore year. He had always been advanced in Math and never really felt challenged. As a sophomore he similarly found AP Calc AB daunting. Your dilemma and questions elicited feelings of deja vu in me.
He was entirely stressed out with the idea of getting a Fall grade of B. We discussed it far to much and similar to what I said above didn’t think there was a clear right or wrong answer. I finally asked him two questions;
Do you actually like math or are you just good at it? He responded without hesitation that he loved math.
I then asked why? He said because he thought of it as a riddle or puzzle and passionately spoke of how he enjoyed the certainty of solving problem sets and then proving himself correct. It was obvious the potential good grades or AP credit were just a form of validation but not his motivation. That discussion led him to conclude that if the grade wasn’t the motivation to take the class in the first place, it certainly shouldn’t be the reason to drop it.
After that conversation he dug in deep, worked with his teacher (and an upperclassman in his class) and managed a B in the fall. It wasn’t easy year but his final grade was an A- I think.
Fast forward 7 years and he is on track to graduate with honors from a very strong applied math program at an Ivy.
He seemingly never looked at math again as a graded class but as a set of individual intellectual challenges.
My point for you being if you have a passion for math go for it and do so without putting pressure on yourself. Work hard and in time your intellectual interest in the subject matter will win out and you should be able to enjoy the ride. If you are doing it just for the sake of doing it find a way to pivot to subjects that really peak your interest and go all in.
Don’t do what you think that others want to see, show them what really matters to you and don’t stress out if possible. Good luck!