In our experience, it’s very unlikely that the city will force a homeowner to tear down an existing structure. At most, the home owner could face a fine if it’s proven they built without a permit.
(My H was on the zoning board of appeals in our last community).