It is possible that the sample of students may be biasing the perception of difficulty.
Math 53 in the fall may be taken by new frosh, who are adjusting to college while doing so, which may cause them to think it is harder, especially if their knowledge of calculus from high school AP courses is not as strong as it should be. It may also have sophomores who started in Math 1A and 1B in frosh year, who may not be as strong in math as students who started in a more advanced math course.
Math 53 in the spring is likely filled with second semester frosh, who have adjusted to college but also are strong enough in math to have started in Math 1B in their first semester (and not done badly enough to change major plans to a major that does not require more advanced math). I.e. strong-in-math students who are less likely to stumble on adjustment-to-college issues.