I think often if you want to dive deep into just one subject like math or music or physics in high school, you need a bigger school (2000-3000 students) so they can offer many levels of math, or a specialty school (school of the arts, a school like TJ in Virginia). There are exceptions, of course, but harder to find.
Usually if you are the only one in the school at that level, you have to take more advanced classes at a university. Not every high school can offer math classes beyond Calc (and many can’t offer that), not every school can have a full orchestra or AP physicals (or at that level). My friend’s son was about 3 grades ahead in math and while it was easy enough for a 2nd grader to walk down the hall to take 5th grade math, it became harder for a 7th grader to get to the high school for calc. By high school, he was at the college for math, but his parents felt it was important for him to stay in hs for the social life and for classes where he wasn’t as advanced. Not perfect, but they were lucky to live in a university town so could make it work. Now with things online, it could be easier to keep moving in math.