I think the others have commented accurately on the influence of the lower test scores.
I wanted to point out that you’ll still need to be taking math and science courses at MIT should you be accepted. The GIR’s still require a full year of math (calculus) and physics plus a semester each of bio and chem. All science classes cannot be fulfilled with just math classes.
Link to the registrar’s office with info on the General Institute Requirements is below. Going in thinking you won’t have to take any science classes will be a mistake.