I was wondering if it mattered a lot. For example I didn’t take science my senior year, many kids in my class didn’t take math during their sophomore year, and my brother doesn’t plan to take math his senior year. Does this affect college admission chances by a lot?
Depends upon which colleges you and brother are targeting, the rigor of your HS schedule and your intended major.
If you’re aiming at a directional it won’t but for our flagship university, its honors college, or top 50 Lac 's / universities it’d matter.
Another aspect would be the rest of your curriculum.
Look at the required/recommended classes of the colleges you are considering applying to and be sure to meet those. Also if you are aiming for top tier colleges you want to speak with your guidance counselor and be sure you schedule is strong enough that he/she will check the box that you have taken the most rigorous schedule available at your HS.