Prestige is only important for certain types of positions / employers.
Do not let others decide what is best for you.