Better to get an A in less demanding classes for elite college admissions?

Eventually, he and you will confront the reality that sometimes gaming the system may be more effective at some (not all) admissions gates. Does he want to become a physician or lawyer? If so, college will be a four year process of chasing A or A+ grades (and it is typically a lot harder to earn a 4.0 in college than in high school).

But also, why are you worried about math rather than some other subject? If the students whose strongest subject is math want to back off the rigor, they usually want to do that in English, foreign language, or history (where there is actually more of a risk of getting a lower grade for them), not math (which they tend to find to be an easy A even in the hardest high school math courses).