I easily get carried away buying lens and so currently have :-
EF-S 10-22, 17-55 f/2.8 IS, 18-200 IS (it came with the 50d)

EF 24-105 L IS, 70-200 L F/4 IS + Extender 1.4x
Primes, Sigma 30 f/1.4, Canon EF 50 f/1.8 & 85 f/1.8
I have also have had two 17-85 IS, one that came with my old 40d and another I bought again after I sold it for a 24-105 as I simply found I was lost in some occasions without it.
My absolute favourite and most used lens is 24-105 by a long way, what I do find is that even though most shots are within 24-50, in many occasions I need to put on the 17-55 or 18-200 as it is just not wide enough, things line a recent childrens birthday party.
In my opinion on the times when I have the wrong lens on, if it is a lack of zoom, I can always crop (obviously loose IQ) if need be to get me by and I get the shot, problem is if the lens isn't wide enough, you are stuffed and the only thing that will fix it is a 17 or 18mm wide or even sometimes my lovely 10-22.
Forget the fact that my 18-200 IS is not as good image quality as my 24-105, the fact is with the 18-200 I can get the shot, with the 24-105 I'd have missed it simple as that.
Don't get hung up on IQ and don't substitute IQ for range. The most important thing here is just taking some photos. At the end of the day if you absolutely know the range and if you main priority is IQ then get a prime or primes. If IQ is not your 100% factor and you want to make sure you get the picture of your family get a zoom in the range you need, but add a little extra on te wide end if your budget can allow as nothing can get you out of a lens not being wide enough situation other than either you or your family moving backwards!
Problem is in most situations the lens you choose will always be a compromise between range, IQ & speed and there is not a magic perfect lens. Different people's priorities are more over one of these factors. I'm still waiting for that magic 10-500mm f/1.4 L IS that weighs only 300g!