To me there are 3 levels of cold. The first is just “cold”. The second is “do I need to worry about frostbite?” (maybe below 10 degrees). Third (the worst) is “it hurts to breathe” (around 20 below).
AFAIK 2 and especially 3 are really not that common in most of New England. I have only encountered #3 a few times in my life.
The “Jump in the freezing water” fundraiser seems to be a thing around here, and probably in lots of cold weather states.