First of all, a B does NOT represent a lack of aptitude. That’s not a bad grade at all for a non-intro math class above Calculus I and II.
Second, I would not recommend Philosophy if you have true interest in Math and Physics and the like. Perhaps a minor for personal reasons but not a major.
Why do your parents want you to do Philosophy? Good grades for med school?
Overall, it sounds like you need to figure out what you WANT to do. Do you want to be a doctor? Do you want to work in CS? Do you want to do physics research or engineering?
It sounds like all three of those are feasible options for you, you just need to pick based on your interests.
It sounds like you like math a lot, and pairing it is certainly not impossible. The track you are on seems fine. It should be noted that higher level math does not directly translate to CS: you will need to take classes in it, whether it be for a minor or in graduate work if you plan to stay in school longer.
Basically, go with what you want here of the STEM fields and if you want, minor in Philosophy (or even just take a few classes for personal interest).