I think this would depend on facts not in evidence. For example, if this is an octagonal parcel, I expect the south side would be exempt from the rule. If it’s a rectangle with no side headed due north, then the side most accurately described as SE would probably be the prohibited side. On the other hand…
I love these kinds of questions, but unfortunately they can become quite contentious.