“recommend you based on what they have seen and observed of you while you were sitting in their classroom”
Exactly the point. And many extend this to comment about the student as they know him/her, outside class, the role in the school, activities and leadership, the respect awarded him or her. That’s more than grades and course names.
How much fo you object based on your own challenge?