Does it look bad to NOT apply/respond to Honors College Invitation?

Yes. After you are accepted and have been offered an honors invitation. Not at the time of application, which is what I was referring to.

Correct.