Will not taking Physics (school only offers academic and AP) hurt my chance at a UC school?

You probably know but just to clarify, USC is a private school and not a UC. Different requirements and different application.