Yes, prestige absolutely matters for academia. Always has.
OTOH, few econ majors are aiming for PhD programs.