You have a few things to consider: What do you want to do after graduation, if you know? RSA will give you a very different education and a very different career path than engineering, no matter where you go to school. If you are interested in architecture enough to want to study that in a six-year program, then perhaps RSA is your answer.
I was personally not impressed by Swarthmore engineering. I loved the school - the size, campus beauty, the people, etc. But that engineering department did not excite me. I felt limited compared to the other schools I had visited. I didn't know what type of engineering I would end up in...what if I discovered my passion was something that Swarthmore didn't offer? But I absolutely loved the size and overall vibe of Swarthmore. It was really hard to let go of.
The residential college system at Rice provides the same sort of small community feel as Swarthmore (well, an even smaller more connected community). It's everything I wanted and a lot more. And here at Rice, I can be a part of that small community, while taking classes in a larger school of engineering...with all of the resources you could possibly need or want. Individual engineering departments at Rice are small enough that you still get plenty of attention from professors and can easily do research as an undergraduate.
Keep in mind that academic schools are not binding at Rice. If you come in architecture and change your mind, you can always switch into engineering. (Not so the other way, since architecture is separate admissions.) Depending on when you make that change of course, you might need to take classes over the summer/overload your schedule/stay a fifth year to graduate.
Hope that helps!