I fly Piper Archer/Warriors and the POH states "Maximum Flaps Extended Speed (VFE) – Do not exceed this speed with the flaps extended." I try not to but I occasionally may exceed Vfe with one notch of flaps during a descent. Could there be structural damage to the flaps in time or do aircraft designers factor in such potential pilot errors?
VFE is calculated for full flap extension, so the loads with just one notch of flaps are much smaller than the limit loads which determine VFE. It is very unlikely that this results in structural damage. Having said that, there is really no reason to exceed VFE with flaps down.
There are two types of damage that could result from flying too fast with flaps extended. The flaps or their attachments could be overloaded, or the wing loads in a vertical gust might exceed the strength of the wing. Both should be avoided.