Normally, you only pay for the semesters that you attend, although I have heard of private colleges and universities that charge for all four years even though a student finishes a semester early.
If you have student loans and you take time off, then when your grace period ends you will have to start re-paying your loans. Because you have to file for financial aid each year, if your family’s financial picture changes during your time off, your aid can change. If you were awarded a merit-based scholarship when you first enrolled, you need to make certain that that scholarship can be renewed if you take time off - sometimes there are restrictions on how long a leave can be, or the reasons for the leave.
How hard is it to go back? That depends on the student, the major, the institution, etc. Many students do much better when they return because they are older, wiser, more self-disciplined. Some do worse because they’ve forgotten material that they learned in introductory courses that is necessary for advanced courses.