At Columbia, changing between the College and SEAS is not unrestricted.
Harvard visual and environmental studies requires a 3.0 GPA to declare.
Cornell has various restrictions on changing divisions. Penn also.
It is true that some other schools, including many popular state flagships, have much greater restrictions on changing major or division.