Will colleges care if I hold Honors Physics until senior year?

I plan to major in a science field so I understand how important physics is. But, I really love chemistry, so here is my science schedule.

Freshman - Hon Bio
Sophomore - Hon Chem
Junior - Hon Chem 2, AP Chem
Senior - Hon Physics, (AP Physics?)

Will colleges care if I hold physics off?

Not as long as you take it. AP Physics 1 is a perfect acceptable introductory course, you don’t need Honors before it.