There is a disconnect between your pieces.
You struggled in science class but want to do pre-med, in a school that has a reputation for creating med-school ready students by setting a tough standard.
You are anxious about going to a school that is very competitive, yet you applied to Harvard, the very definition of a competitive school.
You want to keep your summers free to have a balance between work and relaxing, yet you want to go into medicine - a field that is famously intense. The people who are going to med school spend their summers either working to earn money to pay for med school or working to get experience for med school.
It’s water under the bridge, but it seems as if you didn’t really do much homework on the colleges you applied to- how did you not know the size of Oxford? that your state U does offer your major?
What I can’t tell is if this is cold feet: you are about to leave the known for the unknown- or if you are in fact recognizing a real mis-match between you and the school. But you say you liked the feel of the school, and it is quite possible that the supportive environment of Oxford will help you figure out your path, while the maturing that happens for every single first year college student will help you find your feet.
Also, if you find the standard of work challenging, use the resources that are on offer early- there is a lot available. Do not think that you are the only one struggling or that you will be able to sort it out on your own. Neither are likely to be true 
And, one thing is for certain:
college is not high school.