not necessarily. if you send it too much power at the limits of its frequency response, then yes, you will over exert the speaker. but take say a 5.25" mid-range speaker and set the high pass at 150+ hz and the low pass at 3.5K hz, you can over power it all day long without damaging it. but if you were to try and run the same power to the same speaker but at 60 or 70 hz, yeah, it'll kill it in a very short time. my buddy ran 150 watts to each side of his alpine type R 6.5" components, but was being stupid and tried to get more mid bass out of then than they could handle. about 5 minutes after finishing the install he cooked them. but i'm using an active crossover (audio control 4XS) which feeds into my 4 channel autotek amp that puts out 100x4 at 4 ohms rms. i have 100 watts going to each 5.25" mid range, and 100 watts going to each tweeter. i've had it setup like that for a few months now with no problem. and my setup gets way louder than my buddy's did for that 5 minutes. properly setting things up is the key to overpowering. running 400 watts to a set of components that are rated at 40 watts isn't bad at all(headbang)
but yeah, there is nothing wrong with underpowering a speaker, as long as it's done properly. if you take a 300 watt amp and hook it to a sub that will take well over 500, and clip the hell out of it, you are gonna have problems. for a sub amp you shouldn't go over 9 dB's worth of clipping.