On a practical note, I live about 4 hours’ drive from my husband and we see each other 1-2 times a month. It’s definitely close enough to drive for a weekend. 3-3.5 hours is even better. You could always drive down on Friday afternoon or evening and return on Sunday night.
But, I agree that you should see a counselor - one at school if you are still in, maybe one at a community mental health clinic if you aren not. Talking it over will help you manage the distress of this situation and figure out your priorities, as well as make a decision. I think you should also discuss with your brother - see whether and how much help he feels he needs, and what you can do to support him.