When I was a sophomore in the early 80’s, a student was held up at gun point in her room, and a homeless guy showered in our (women’s) bathroom. I live in the area and New Haven is far safer than it used to be, but it’s a city, and I understand the reasons for security.
And that (restricted access to inside yard) makes Yale’s Residential College setting feel safe (for me!).
Tour groups can enter the old campus and cross campus yards. I like the current setting.