The underpowered amp can damage speakers by "clipping".
The signal going to the speakers from the amp is AC (Alternating Current).
(the voltage is alternating too, but they have to call it something

).
This means that to play say a 100Hz tone, the amp will alternate the output voltage (and hence current) between say +12V and -12V (or +1.5A and -1.5A) 100 times a second (the actual voltage/current level depends on the volume you select and the impedance of the speaker, which I've assumed to be 8ohms in the above example).
Now, a powerful amp may be able to deliver a maximum of +/-100V across a speaker, or +/-12.5A into it, where a low powered amp can only deliver +/-40V or +/-5A.
What happens if the low powered amp tries to deliver +/-80V or +/-10A?
The tips of the voltage or current wave are simply "cut-off" or "clipped" - for the duration of the "clip", the voltage/current is steady, which means it's DC (Direct Current - ie not alternating). This is very bad for speakers, especially tweeters (you may think the series capacitor in the crossover would protect the tweeter from DC - it won't in this case, as essentially what you have is pulsed DC, which a capacitor will pass).
You may think that the manufacturers would only allow the volume knob to go round far enough so that the maximum ouput voltage could only be +/-40V, and hence max current would only be +/-5A into an 8ohm load. They may well setup like this, using a dummy 8ohm load (but the dummy load is a constant impedance)
In reality they can't be so precise, as a speaker is a complex load. The 8ohms impedance quoted is a nominal value - it's quite common for this to vary between 4-16ohms at varying frequencies throughout the audio range (and can rise to 40ohms or more low down, due to the resonance of the bass driver). Forgetting the resonace, this means that at +/-40V output, the current will be +/-5A for the time the speaker is at 8ohms, but will rise (or attempt to rise) to 10A when the speaker is at 4ohms - the amp can't deliver it, so it's clipped.
Following the above example, it becomes clear that when the speaker is actually at 4ohms, the amp will start to clip the current at just +/-20V (as this is the point where max current delivery is reached).
Using RMS figures, this is at just 1/4 of the rated output power.
This is also why low powered amps generally can't tolerate low impedance (nom. 4ohm) speakers. It's also why many manufacturers will attempt to give a guide for suitable powered amps, of say 40-100W. The lower figure is their estimate of the lowest powered amp you should use in order to avoid the problems above in average usage, while the upper figure is to avoid the problems you stated yourself - ie there is an upper limit as well, beyond which you'll damage your speakers.
However, the guide is just that - due to the way that some manufacturers quote their power figures. It's possible to take an amplifier and quote it's output at anything between 20W and 400W or more, and still technically not be lying!
(I believe that in the US, (not sure if it's just certain states though) they have now legislated on this issue, forcing manufacturers to quote their figures the same way - no idea if anyone actually checks them though).
One of the more common "tricks" at the moment though is to quote say 6x100W for a multichannel receiver. This doesn't necessarily actually mean all six channels driven simultaneously though - do that and the output can fall to 6x35W in some cases.
Imagine a sine wave, and simply slice off the tips - you'll have what's beginning to look like a square wave - the flat bits now at the tops and bottoms of the wave are the DC component.
DC can burn the voice coil out (if long enough in duration) - and can cause it (and hence the cone) to move too far, which can cause mechanical damage to the speakers motor assembly.
Sorry it's a long post - but you did ask
HTH