I am not an expert on admissions, therefore I won’t dare to make you a suggestion on whether or not Math SL will decrease your chance of admission. But, on the other hand, getting a 5 from Math HL will not make your chance better. In our school, kids’ averages in math usually get lower in their IB years. Maybe you can get some info on math from your school’s juniors/seniors. Some high schools put the grade distrubitions on each subject for different years on the school profile (usually these are the high schools that do not rank). Those might give an idea on your survival probability in the Math HL class.
Very good ECs btw. Congrats.
Good luck.