I agree with your second paragraph and a number of your sentiments, but not with the “short change” comment.
For those with financial need, the best chance of getting fin aid, IF they can gain the admittance to such schools, is from schools that guarantee to meet need. Those tend to be the private schools. Only two state schools guarantee to meet full need for accepted US freshman during the regular admissions season.
Even among those private schools, financial aid packages can vary widely. Some expect larger student contributions, some have self help, loans in aid packages. Some allow some stacking of merit money. Some ignore or limit home equity in the fin aid calculations.
I would urge any student with a reasonable chance of getting into such schools to apply to the private schools that guarantee to meet full need, if they need financial aid and appear to qualify for it. Many of these students end up with a lower net cost at these schools than even their state schools.
“Negotiate” is not a word FA Officers want to hear. However, they will discuss awards with students and parents, and it is not unheard of for them to revise packages in favor of the family. Sometimes a little extra merit can get added to a package through admissions too. But, you are right in stating that these are all up to chance, and not something to count on.
Every student should have a school that will take them that is affordable without contingencies.