A state school in which only approx 33% of students are in state (your option 2) is not really a state school. I don’t know the exact wording of UM’s charter, but I’m guessing that would bring it into question.
There are a couple of problems with your argument about unsustainability in point 3. UM is different that the other schools you mention in a couple of key ways - the most important is probably endowment. It is just way more than the others. The second is its prestige is higher, enabling it to charge more per out of state student. It is also smaller even with its current numbers.
Universities are conservative in that they don’t change fast if they can help it. UM is already changing slowly - the drop in instate students has mirrored that of the instate high school population - it hasn’t been slower. They’ve added some OOS students to take advantage of the current numbers of really good students willing to pay high OOS prices. I don’t expect any of these to change drastically.