To help you understand this, I feel like we should discuss how Va is determined.
14 CFR 23.337 Limit maneuvering load factors.
(a) The positive limit maneuvering load factor n may not be less than—
(1) 2.1 + (24,000 ÷ (W + 10,000)) for normal and commuter category airplanes, where W = design maximum takeoff weight, except that n need not be more than 3.8;
(2) 4.4 for utility category airplanes; or
(3) 6.0 for acrobatic category airplanes.
(b) The negative limit maneuvering load factor may not be less than—
(1) 0.4 times the positive load factor for the normal utility and commuter categories; or
(2) 0.5 times the positive load factor for the acrobatic category.
(c) Maneuvering load factors lower than those specified in this section may be used if the airplane has design features that make it impossible to exceed these values in flight.
Based on that regulation, the light single-engine aircraft I've flown have a +3.8/-1.52 load factor limit in normal category loading. The aircraft is designed to structurally handle those load factors up to max gross weight.
A pilot could exceed those limits by making abrupt control inputs or "pulling G's" because by pitching up abruptly, for example, the pilot is increasing the angle of attack and accelerating the aircraft.
At speeds above Va, the wings are capable of producing enough force to damage the structure with positive or negative loading.
This is a V-G diagram (Source)
The curved line represents the load factor at which the aircraft will stall at a given indicated airspeed.
Va is the speed at which the aircraft would reach its maximum positive load factor limit at the same time it would experience an aerodynamic stall.
An angle of attack less than or greater than the critical angle of attack will cause the wing to generate less lift than it would at the critical angle of attack. Therefore, the stall point is used to determine where the maximum load factor would occur for a given speed.
Maneuvering speed, as you can see by the chart, has no safety buffer above it, which is why Dave's answer has some details about inadvertently increasing speed.
Flying below maneuvering speed would be a better guideline to follow if you feel that there is are hazardous levels of turbulence.
Let's say you are flying well below Va, and you are hit by a strong vertical gust. The vertical, upward, gust will increase the angle of attack by momentarily changing the relative wind, and therefore increase lift. If that change in angle of attack exceeds the critical angle of attack, then the aircraft will aerodynamically stall rather than continue to produce greater lift. In that case, the stalled condition would protect the structure of the aircraft by acting as a limit on structure loading.
Now to answer your specific question:
It is unlikely that severe turbulence will come as a surprise during flight in a small single-engine aircraft. I have never personally experienced turbulence where I felt the need to decrease my speed.
It is a judgement call that needs to be made by the PIC, and I would slow down whenever I am experiencing consistent turbulence that is causing me to physically perceive unusually high G-forces.
If you find yourself in that scenario, you have likely made a poor go/no go decision regarding the flight. If you need a quantifier, "severe turbulence" would be the classification for which this matters.
I still need to edit this for grammar, but I am out of time at the moment.