As a former townie, there are benefits to being local. You can show off your home town and the ‘hidden treasures’ to the new arrivals and experience it again through their eyes. I had easy access to a car. You can make a point of doing away weekends. When friends from high school were home visiting their folks, we were able to catch up easily even if our breaks didn’t match. And of course, a junior year abroad means you’ve got your ‘away experience’ - just delayed slightly. I don’t regret having been close to home - I visited my family slightly more often than those who were from out of town - but only because I wanted the car. And when I got sick, I was able to go home for the weekend to recoup.
So long as you aren’t clinging to Brown because you’re afraid of what else is out there, I’m not sure I see a problem. On the other hand, there are other of ‘Brown-ish’ schools out there, depending on what criteria you have set.