Honestly, I agree to a point. Middle-class families often do get screwed (especially with fin aid) and being an ORM or a majority race is really not helpful sometimes, but those are factors outside of your control.
The system is not fair and it’s not meant to be that way. If it worked like it does in some Asian countries, then we would have major problems, because Western society frowns upon that level of gung-ho academic cutthroat competition. And also, our testing standards are considerably more diluted and watered down nowadays. Just look at the number of kids who get perfect scores on the SAT/ACT alone.
Colleges aren’t looking at numbers, they’re looking for people. But even if they like you, that’s no guarantee that you’ll get in, especially at a prestigious college—that’s just how competitive it is.