It is not terribly unusual to see 16 year olds (particularly those soon turning 17) at the Ivies. I think younger than that is very very unusual in recent years. Many decades ago Harvard accepted a young teen who had a very difficult time and became almost a street person afterward (would probably have been diagnosed with Aspergers or autism today), you can google to find that story, and for many many years Ivies took that as a lesson not to accept kids who are too young.
There are such geniuses actually. I know one who was a 19 year old PhD student in Mathematics. I recall the bouncers at Cottage giving him a hard time "This prox is clearly fake. How can you be a graduate student when you're not even 21?"
16 can happen if you skipped a couple grades in elementary school. At least that's how the two 16-year-old college students I know went to college. How did the 28-year-old get to be a freshman? 10 gap years or applied late?
No. Probably more like 16+3 (undergrad, or 15+4) = 19 (start PhD). It is perfectly possible to start a PhD program directly out of college. Most people don't do it that way, but it's not rare at all, especially for a kid who may be something of a prodigy and is clearly headed for an academic career. Larry Summers, for example, was a PhD student at Harvard at 20, and a tenured faculty member there at 28. Kurt Godel, several generations before, received his PhD in mathematics at age 24.
Spending exactly four years in graduate school and then entering a PhD program sounds very unorthodox to me.
The only kind of non-PhD graduate programs I can think of that should take exactly 4 years are like medical and dentistry degrees, and once you graduate from those, you ought to be entering residencies, not becoming a PhD candidate.
I think starting that early is just wrong... Wouldn't you feel like an outsider being so much younger than everyone else. Plus... Super geniuses tend to be lacking in the social department. Why set them back even more developmentally by starting at a younger age. I think you'd be missing out on a huge part of you life.