I think you should be okay as long as you do well in your classes.
The only thing is, if the majority of the top cohort is skipping ahead, then it may affect your overall class rank (weighted), and also your load may not appear as rigorous.
Some students take these summer classes to get ahead, and fit in as many APs as possible, whereas other students do have a genuine need to accelerate because the classes are easy for them.
e.g. in Math,
Would you be able to manage Algebra 2 over the summer (6 weeks) and move on to Precalc and then Calc ?