Math-CS is not a CS degree: it’s what it sounds like, a math degree in which you also learn CS. UCSD’s math advisers do a great job of describing it here: http://talk.collegeconfidential.com/discussion/comment/18292151/#Comment_18292151
Anecdotally, I’ve heard that Math-CS majors aren’t at a disadvantage when finding CS jobs at all. In particular, if you complete projects and get internships early on, you’re usually as prepared and hireable as a regular CSE major. In fact, you might have an advantage because apparently some employers think Math-CS refers to a dual degree. All of the regular CS and Math clubs, activities and research opportunities are open to you: there is a slight difference in that CSE majors have priority enrollment for CSE courses, but even that usually isn’t a huge issue. As for transferring to CS, you can try but it’s very competitive: you usually need at least a 3.8 in the prerequisite courses, and spring quarter only a perfect 4.0 got you into the major.
Cal Poly undoubtedly has a better social scene and more school spirit, although the latter might change if UCSD successfully votes to move up to Division II next week. While UCSD also doesn’t have a traditional “college town” like SLO, San Diego is one of the most beautiful, exciting cities in the country and definitely has enough to keep you interested for four years. Our school’s proximity to the beach and beautiful scenery of La Jolla is great, and University City is rapidly developing into a combination student ghetto and second downtown. Additionally, once you turn 21 you’ll have access to some of the best nightlife in Downtown and Pacific Beach. Also, before you graduate, they’ll probably finish the trolley connecting campus to Downtown SD and Tijuana 
Class sizes, particularly in lower division courses, are pretty large but probably comparable to Cal Poly’s (and there’s no 1,000 person lectures like you hear about at Berkeley, I’d say they average 200-250). Even then, the classes feel smaller because of discussion sections with the grad student TAs. For CS in particular, there are a ton of resources and tutors to help you with assignments, and a lot of the assigned projects are group work: expect to spend a lot of time grinding in the CS basement. Academically, UCSD has much more diverse research going on (as you would expect of a PhD granting institution). That’s true for both CS and advanced mathematics. Similarly, you’re correct that UCSD is stronger in many non-engineering fields (particularly cognitive science, biology, oceanography, economics, political science, international relations and linguistics). What college were you admitted to? I can try to tell you a little more about its GEs and residential atmosphere.
Overall, I’d say UCSD is the stronger choice. You’ll receive an extensive, thorough education that will certainly prepare you better for grad school and will almost certainly afford you similar or better opportunities in finding internships and careers. The networking and conference opportunities you’ll have will be more numerous, and you’ll be part of a thriving CS community of motivated students that does cool stuff like the Entrepreneurial Challenge, Startup.UCSD and SDHacks. You’ll also get to enjoy beautiful La Jolla, a short drive from San Diego’s best attractions (and closer to LA than SLO is). Feel free to ask if you have any follow up questions and let us know what you decide!