[![enter image description here][1]][1]<br>
https://www.eucass.eu/doi/EUCASS2017-318.pdf

Above shows a configuration where the flaps and spoilers are deployed. The blue line shows the vertical velocity of the wing's wake, which is indicative of the lift being generated. Since the effect of the flaps lessens outboard, that causes the downforce shown above near the end of the wing due to the spoilers.

Since spoilers are thin and are allowed to move slightly, and the precision of their symmetrical deployment is not as high, and since this downforce is far out with a greater moment arm, any slight [asymmetric] disturbance will result in a roll.

The effect is lessened with a smaller flaps deflection, since the inboard part of the wing will create less lift. And in the case where the speed brakes are used on their own, there is a uniform downforce distribution over the wing without the abrupt shift shown above.


  [1]: https://i.sstatic.net/xOFsz.png