How bad is dropping a non prereq class for med/dental school

Ok! Yes unfortunately I do like physics a lot, but it requires a lot more time and energy than I can give it while also taking hard bio and chem classes. I could handle it if it was all I was doing, but not with everything else!! And I do like chemistry. From what I’ve seen online the requirements look pretty much the same but vary more from school to school but I’ll definitely take any input and ask my advisor next week!