For starters you don’t simply ignore the tests. Any relatively endogamous group that can be identified is going to have test scores that are either lower or higher than other groups. That alone doesn’t indicate that the tests are “biased.” It would be a fairly simple matter to compare life outcomes of students who had roughly identical SES and test scores. I would hypothesize that such a study would find that test scores are not equally predictive for all groups, but that they are nevertheless highly predictive for all groups.
There is more to life than being “smart.” A sometime business partner of mine made the equivalent of a 21 on the ACT, and when I see things he’s written I cringe. But I admire his judgment greatly, and he’s doing just fine in life. I have another friend who never got out of remedial math in college, but he makes more money than Carter has Liver Pills. Hard work, thrift, and “people” skills do make a difference.
Likewise, I’m aware that racial pressures can make a difference, particularly for black boys. My son, who is quite bright, was explaining a low algebra test score to me when he was in seventh grade. He listed all of his friends who had made substantially lower scores. I then asked about one of his friends that he missed and he response was, “DAD, he’s Asian!” In other words, my son refused to even compete with the Asian students. Also, although my son’s best friend in high school was Asian, to the best of my knowledge my son was not invited to participate in the Asian math study group taught by one of the Asian parents (a mathematics professor). So I have personally witnessed the tendency of racial cohesion to cause students to limit their own ambitions as well as to sometimes limit opportunities.
Back to your original question. Test score differences between groups are not evidence of bias unless such scores show themselves to be unreliable in predicting academic or other outcomes. A study that refuses to consider test score data doesn’t make such data invalid or non-predictive. It just makes the study worthless.