Above 100,000 feet, there is almost no air at all, so no oxygen for air breathing engines to burn, and no air to produce lift or for control surfaces to react against.
In the 1960's, the USAF had two research aircraft that could exceed 100,000 feet. Might have had more, but these are the two that I remember:
The NF-104 could go slightly over 100,000 feet using a rocket engine above 70,000 feet for propulsion, and small reaction rockets to provide attitude control. It was built as a low cost X-15 trainer as its high flights simulated the X-15's operating characteristics.
The X-15 could go considerably over 100,000 feet, using a rocket engine at all times (after being launched from a B52), and small reaction rockets to provide attitude control.
Point being - by the time those aircraft exceeded 100,000 feet, they weren't flying. They were following a ballistic arc, propelled by a rocket, and kept aloft entirely by inertia, not lift.
So - why didn't the U2 and SR71 fly higher? One reason is - they would need rocket engines with oxydiser on board for propulsion, which is fairly short ranged, and very tempermental.
The requirement for rocket propulsion would negate the primary advantage of both aircraft: the ability to stay at high altitude for long periods of time.