You don't have to join Greek life to have fun; however, it's a big deal. As for academics, it depends on which program(s) you're interested in pursuing. For example, University of Mississippi's Center for Manufacturing Excellence (CME) program is pretty high quality and most students love the community aspect of it. (CME includes Electrical Engineering, Biomedical Engineering, etc., but not General Engineering.) Also, the Croft Institute for International Studies is great. I don't know if they allow transfer students to join Croft, but I'm sure the information is on the website. If you're entering your junior year, I'd recommend applying for the Honors College. Great community, free printing, access to full kitchen, etc. and the honors classes requirements are way relaxed for junior entry. Patterson School of Accountancy is another strong undergraduate program at Ole Miss. It's currently ranked in the top 10 nationwide.
As previous users have mentioned, Oxford isn't a cultural hub by any means, but Memphis, TN is only an hour drive and it's known for music, food, entertainment. In general, while Ole Miss isn't a very academic institution, it is more diverse and inclusive than most would imagine. While you will meet many many people from small-town Mississippi, you will also frequently encounter students from Ohio, Florida, Arkansas, Illinois, etc. From your post, it sounds like you might enjoy attending college in an urban setting. Since you're keen on the South, I would recommend looking into colleges in Florida, University of Alabama-Huntsville, University of Alabama-Birmingham, University of Memphis, as well as schools in Texas like UT-Dallas.
I agree with other users who suggested visiting Oxford and University of Mississippi for yourself. Best of luck