No. Why would it be unusual that someone disagrees with the opinions of an ancestor?
Even with more recent ancestors, many people disagree with the opinions of their parents or grandparents on various topics (including racism related ones).
John Calhoun had lots of children. Presumably, lots of people are his descendants. But probably very few (if any) of those alive today are pro-slavery.