It doesn’t matter whether you consider it skimping or not. If you have to take physics in college to be pre-med, it will be beneficial to you take physics in high school.
If you don’t mind being introduced to forces, lenses, magnetism, and electricity for the first time in college, when almost everyone else has had at least a year of it, then fine.
I work with pre-med students at my university, so I know a bit about med school requirements.