You can decide not to go there right up until the day you start, as long as you haven’t paid the tuition and fees yet. You will lose the deposit. You might lose tuition and fees if you’ve paid.
It sounds like you’re talking about a waitlist situation. Or maybe a late acceptance if some non-selective colleges accept apps on a rolling basis right until the start of the term.