It is only likely to affect if if it appears that you’re avoiding it/something to hide. But that would also mean there are other red flags in your application.
Given that your reason for not taking the test is logistical, it should not affect your application if the school is test optional.