Which is irrelevant to what you told us the teacher said the first time: “if we did better on our next exam he would drop our previous one.”
I’m not saying it’s totally “fair”, but he certainly followed the rules he set up. Make sure you’re aware of that when you try to make your case with the administration.