Grinnell is a hell of a school. You certainly wouldn’t be shortchanging him education wise if he went there. If you feel guilt about not spending an extra 100K To go to Middlebury…it’s not Harvard, ya know?
This whole “prestige” thing…after a while, employers don’t care. They just want someone who gets the job done, whether you went to an Ivy, or yer local State school. If your son is going to be an academic then it might be another story, but even there I’ve got some in my family and they were all over the map in where they studied.