ABET does accredit CS majors, and can be looked at as a sign that the major meets a decent minimum standard (could be better, of course), but it is not generally considered essential for a quality CS major (although non-ABET-accredited CS majors can range from poor to excellent). It is useful for the patent exam.
It is true that most decent CS majors require calculus and linear algebra, but the most important math course for CS is discrete math (and the upper division CS theory courses are very much like math courses). Physics requirements tend to be at engineering-based CS majors, or those at schools where physics is a general education requirement (e.g. MIT). ABET accredited CS majors do require some non-CS science, and 1/4 of the degree program in math and non-CS science.