Will not taking physics make me less competitive?

I’m currently a junior and have a 4.0 gpa, I’ve taken pretty challenging classes so far but I don’t like science and don’t really want to take physics (AP or not). Instead, I’m thinking about taking an elective closer to what I want to go into (political science). I don’t want to apply to ivies or anything, but i’m thinking about higher tier UCs and pretty selective liberal arts schools. Will not taking physics hurt my chances too much?