If you have what it takes to get into med school, you can get there from anywhere. An Ivy institution will not give you a boost. In face, it might make it harder for you because it almost certainly will be more difficult to get the GPA.
If you get to college and decide that medicine isn’t for you, which of these places would you like to be at? And, would your parents be happy to pay Brown costs for your studies in something less directly career focused than pre med?