To elaborate on a previous answer, large changes in wind direction on short timescales tend to be caused by thermal updrafts passing through the area, tending to "suck" the wind toward their centers.
When the overall windspeed is high, the (relatively) small changes in wind velocity (speed and direction) caused by this phenomenon tend to be dwarfed by the overall prevailing wind.
In addition, strong wind tends to suppress thermal convection, at least near the ground.
On a windy day, there still can be some significant changes in wind speed, caused by turbulent "rotors" (tumbling masses of air) intermittently mixing the stronger wind aloft down to the surface, but these tend not to involve extreme changes in wind direction. This intermittent turbulent mixing will be more pronounced on a sunny day (or with an unstable airmass) than on a cloudy day (or with a stable airmass).
It is useful to know that even on a windy day, there can be some significant change in wind direction as the turbulent "rotors" make the wind speed increase and decrease, and it typically happens in a systematic way-- due to a phenomenon called the Eckman spiral, the wind aloft tends to be more westerly (in the northern hemisphere, and more southerly in the southern hemisphere) than the wind at the surface. So as the wind gusts up to a stronger value, it typically "veers" in the direction that is aligned with the wind higher up. This is good to know if you are a pilot-- and is absolutely essential to know if you are a racing sailor.
Similarly, as your aircraft rises up through the air after take-off, it will typically (in the northern hemisphere) tend to "feel" a temporary increase in apparent wind from the right (or a temporary decrease in apparent wind from the left), which may have some noticeable affect on handling. And the reverse on landing. And all the reverse in the southern hemisphere. And all disappearing or being greatly reduced near the equator. Since these effects are dependent on a marked change in wind direction with altitude, they tend to be more pronounced on a cloudy day or with a stable airmass, and less pronounced on a sunny day or with an unstable airmass, when the airmass tends to be more "churned up" by turbulent mixing.