I think that you should read the “applying sideways” blog on the MIT admissions website. Of course MIT is not in the Ivy League, but it is similar in terms of overall ranking and difficulty of getting accepted and the same approach is good for the Ivy League schools.
The point as I understand it is to do do what you want to do, and do it very well. This is what I did to get into MIT (for my bachelor’s) and Stanford (for my master’s). So far it also sounds like it is what you are doing also. Even if this does not get you into Harvard or Brown, you still will have done what was right for you.
Also, try to understand the differences between the Ivy League universities and other top ranked universities, keep your budget in mind, and make sure that you apply to at least two safeties.
You might want to also think about whether you would be interested in a smaller school such as a Liberal Arts College. You should also look carefully at your local universities.
You are young. It seems to me that you have been doing very well so far.