Well pretty much everyone at Harvard back then came from a privileged background 
I think it was more about what happened AFTER Harvard. Those with high social standing did not have to worry about getting a job. Managing trusts required a certain accent and manner, not good grades. So a “gentleman” could be relaxed about grades (Not a “grade-grubber”) knowing they would still do well in life. Working harder was evidence of lower social standing, in other words. Not a meritiocracy, for sure.
I think the equivalent today might be students pretending not to work too hard because that might show superior intelligence. It is still not “cool” to be an eager beaver, socially, but for different reasons!