Taking University Physics in college without ever taking a single Physics course in my life time.

Not necessarily. Some list the high school science as a recommended prerequisite.

Having had a high school physics course may be helpful when taking physics in college, but may not be strictly necessary. But physics in college will likely be more difficult if you have not seen any physics even at the high school level.

Also, having only “barely passed” the calculus prerequisite for physics in college will likely mean that it will be more difficult than if you know the calculus inside and out.