First of all, congratulations for everything you have been able to accomplish so far. i don’t really know anything about graduate school for physics specifically, or what the graduate school competition landscape is today.
My advice is more to help you with your current state of anxiety about your future. It seems like you are trying to think about and solve all of the variables all at once rather than break things down and take one step at a time. Right now you are adjusting to your transfer and upper level courses. You don’t know whether your GPA will go down or not at this point in the term even. Try to break the process of getting to graduate school down into steps for your next few terms and get those steps accomplished. Then see where it takes you. You can’t predict at this moment how things will turn out, and that is scary. Keep working hard toward your goals, one step at a time, and see where it takes you.