I know nothing about fencing.
But New Paltz has a music scene. Toured the campus 2 or 3 years ago and the dorm we visited was littered with instruments. Nice small town to walk to. It’s Woodstock country. And they will be close on price.
Back in my day the biggest college music scene was Athens, Georgia, home to University of Georgia. Out of state cost is low to mid $40’s.
Mary Washington is a match for academics, cost, and culture, although might be smaller than you want.