I don't think that it's a disadvantage at all--if anything it could be helpful.
According to the College Board website, of the first year students, Colby has 16% in-state students and Bowdoin has 13% in-state students.
Also from the College Board Colby page:
Colby Admission requirements:
* Essay(s) required
* Required for Some: SAT Reasoning Test or ACT
* If submitting ACT, the writing section is recommended
* Required for Some: SAT Subject Tests
Very important admission factors:
* Character/Personal Qualities
* Rigor of secondary school record
Important admission factors:
* Class Rank
* Application Essay
* Extracurricular Activities
* Racial/Ethnic Status
* Recommendations
* Standardized Test Scores
* Talent/Ability
* Academic GPA
Considered:
* Alumni Relation
--> Geographical Residence
* Interview
* State Residency
* Volunteer Work
* Work Experience
* First generation college student
* Level of Applicant's Interest
(Bowdoin's says essentially the same thing)
As long as your gpa/scores/essays are good, I think you'll probably be fine