As in many fields, it gets complicated when a department is ranked primarily for its research/PhD program, and you are choosing an undergrad.
I think standard good advice for kids with an interest in fields like Philosophy is start by looking for generally good reading/writing/humanities sorts of colleges. You might also look at the relative popularity of the Philosophy major (or majors), taking into account the size of the Arts & Science program (like a relatively popular Philosophy major at a college with 2000 A&S students may still have less students than at a college with 10000 A&S students).
Then if you are looking at two colleges that both fit your other needs and also do well in these ways, you can look at departmental rankings as a tiebreaker sort of thing.
At the end of the day, quantifying how much difference this will actually make is basically impossible, although just because something can’t be easily quantified doesn’t mean it isn’t real.
And if nothing else, I think there is an independent virtue to being excited about wherever you are going to college. So if you are looking at comfortably affordable and generally suitable colleges but would be more excited about one for this reason, good enough for me!