Yes, got it in one. Thing is, speakers give you the correct flavour ( if you can use that expression ) or tone of sound. The amp then gives the presentation, so that a Naim NAP 250 partnered with a pre will have fantastic grip on the speakers, giving driving solid base, but missing out on the ultra lows and highs. A big A class like a musical fidelity will give a warmer sound with more bottom end bass and usualy a sweeter top end. A older quad would give a very neutral sound, good for listening to the tone of different speakers. If you try Rogers LS range BBC ref speakers, you can listen to what an amp is doing ( struggling if it is low powered ).
Get the speakers right, then you can play around with different amp combinations. If your speakers dont sound right now, no matter how many amp combos you try, you will never get the sound you want. Believe me, Ive been there and got the tee shirt.
In general, smaller speakers give better imaging and usualy give better timing.
Bigger speakers usually miss out a bit on timing but give greater amounts of low bass. This can get excessive in smaller rooms.
Alternatives to all this are quad electrostatics, miss out on slamming bass, but the openess and imaging are fantastic.
Hope this helps