I would have picked Penn State based on rank, but it sounds like your gut is telling you RIT. So go with that and don’t look back. Maybe go to some U Rochester events to meet different people and experience a different social scene.
Still I’ll note that only 20% of students at Penn State are greek, so it is not something you need to worry about. Just don’t pledge and don’t attend the parties if you don’t want to, you won’t be alone.