@Lnafziger is correct that absolutely ZERO contact between aircraft flying in formation is acceptable, with Aerial Refueling (AR) being the sole exception to this -- but in that case the AR equipment is designed to allow for what's going on.
The problem with contact IS the significant risk of damage to the aircraft involved, so the assumption in "assuming no damage is caused, how much impact..." is outside of typical experience. IF you could assure no damage, then all sorts of things might be possible, but even pretty stoutly built fighter jets still aren't "bumper cars" and shouldn't be treated as such. Even if a collision didn't cause an immediate structural failure, the risk of damage to the internal structure of the wing/tail/whatever would be enough that the aircraft would have to be taken apart & xray'd (or another non-destructive inspection method) to ensure that there wasn't damage that would later result in structural failure.
The one exception to all of this that I can recall was a technique that the RAF used in WW II to defeat the V-1 "buzz bomb" cruise missiles. While the Spitfires could shoot them down, the small size of the missile meant that you had to get so close that you were probably flying through a cloud of debris right after you killed it (i.e. the warhead would detonate and the shrapnel might bring down the Spitfire as well). They discovered that the gyroscopes in the V-1 guidance (i.e. the primitive autopilot) would tumble if they were inverted, so some pilots would fly formation with the V-1 then intentionally make contact with its wing with their own, flipping the buzz bomb onto its back. At this point, the autopilot lost its bearings and the V-1 went down into (presumably!) empty countryside, crashing far from its London target.
Other than this, formation flight is carefully planned out so as to minimize all risk of contact between the aircraft. Even in the (very cool!) video of the Blue Angels, you can see that there is always space between the wingtips -- more space for the more dynamic maneuvers, with the least space during a fairly brief and reasonably stable pass. Remember, these pilots are extremely good & highly experienced in this sort of formation flying, and while there IS oscillation in pitch & roll, the formation is designed so that this can safely go on. There just isn't much risk there of a force that would move one aircraft directly to the left or right of its position. Turbulence might produce a rolling or pitching motion that could LEAD TO a collision, but the pilots have enough control authority to correct these sorts of forces.
That's why they're able to do what they do, safely.