Yes, you close back down. Ideally, a State is monitoring the numbers closely enough so even a small spike over required metrics will be immediately addressed. If done on a regional basis and quickly, you only need to close that one back down (or go back one phase.) This method calls for constant monitoring of the data and vigorous testing and tracing methods. The problem now is so many states did not have any of these systems in place when they reopened willy nilly.
In addition, Governors of these states need to be aware of what is causing upswings in virus cases across the country - so they can adjust their plans before a spike occurs. I’m sure that is why NJ and hopefully NY will not allow indoor dining now.