A swept wing would most likely adversely affect the handling of a U2 during a typical mission profile, and would adversely affect the structure required (weight) of the aircraft.
A swept wing adversely impacts the stall characteristics and stability when flying near a stall.
It should be noted that while a swept wing helps manage effects due to the compressibility of air near mach speed. The primary effect is a swept wing delays the near mach shock wave development. This results in reduced drag near transonic speeds. The U-2 is designed to operate just below transonic speeds, so in theory, there could be a benefit of a swept wing on the U-2.
The benefits of a swept wing comes at a price. In addition to requiring a beefier structure to handle the torsional load stresses of a swept wing, the stall characteristics of the wing are different.
The resultant stall characteristics can become more pronounced and more difficult to recover from. In the case of the U-2, which at higher altitudes can fly with as little as a 2.5 to 7.5 knot gap between stall and MMO, the pilot must be sensitive to the pre-stall buffet.
A swept wing would make the sensing of that buffet potentially a little more difficult, but more importantly, it would make stall recovery much more difficult. In many aircraft that would be manageable, however, in the case of the U-2 stalls are not docile, and have been known to cause tail structural failures and separation.
The risks of a swept wing on the U-2 outweigh any benefit of reduced aerodynamic drag from a swept wing, given that there was a decision to have the U-2 be a subsonic aircraft. The additional weight of the structure needed for the swept wing could also be construed as to further close the stall to MMO gap.
Therefore, not implementing a swept wing on the U-2 helps control the stall characteristics, and should the onset of a stall occur, it is easier and with less risk, to recover from a stall.