What exactly are the consequences of breaking an Early Decision agreement?

I know that the family is usually involved. The point is that it’s not the college (for good reasons), so right now it seems like the student and family can avoid repercussions by saying that the college didn’t provide enough aid.