What would have happened if Joe Kittinger, Felix Baumgartner and Alan Eustace had opened their parachute right after jumping off the capsule (respectively releasing oneself from the balloon in case of Eustace)? At first it probably wouldn't matter as the chutes would be needless in the near-vacuum the balloonists jumped from, but how exactly would an open parachute have altered their falls? And would something happen to the parachute itself?
tl;dr Opening too high will kill you.
As you note, the parachute would just hang next to them immediately after leaving the balloon -- there's too little air that high to even drag the shroud lines out straight.
Later on, there would come a point when the canopy would fill, probably while the jumper was near Mach 1: the air would become thick enough to drag the parachute out to the end of the shrouds and pressure inside the canopy would build enough to shape the parachute. Assuming it doesn't tangle and fail in one of several ways (this is a real hazard if the lines are slack at any point), there's still the danger of the canopy opening abruptly. Even in the thin air above 40 km (25 mi), a canopy sized for a man in a pressure suit is capable of generating an opening shock -- the G force from the sudden deceleration as the canopy shapes and its drag force becomes measurable in tons -- sufficient to deal severe injury. Historically, this has caused broken necks and backs even with parachutes opening at a mere 100-150 m/s (330-490 ft/s) in denser air.
This is why it's normal even with a bail-out at jet altitude to free fall to within a couple kilometers of ground level -- because the lower you get, the slower you fall. Rising air density reduces your terminal velocity from, in the case of Baumgartner, at least, near Mach 1 to only 50-60 m/s (160-200 ft/s).
Of course, with opening shock that high, there's also the possibility (nay, likelihood) that some part of the parachute system will fail under loads tens of times what they were designed to take. Shroud lines can snap, canopy fabric rip, etc. This can lead to an open parachute that doesn't provide the low fall speed it should, or a streamer -- potentially with a badly injured jumper who isn't functional enough to cut away and open his reserve.
Bottom line, these men had a very carefully thought out system to get them down safely: their drogue to stabilize their fall when there was barely enough air to make a self-inflating drogue pull its line straight, and their long, long fall under drogue before opening their main low enough that the opening shock wouldn't break anything.
From comments, there's a question whether there's a midway opening altitude where the parachute can fill, but the air is too thin to produce injurious opening force. I don't know that this has been studied in detail, but there are altitudes within the envelope of various aircraft (high altitude interceptors and the SR-71 they were designed to try to intercept) at which it might be possible to open safely -- perhaps between about 18 and 20 or so kilometers (59-65,000 ft) -- but doing so would be a bad idea for another reason: there's a limited supply of breathing oxygen in the "bail-out bottle" attached to a pilot's pressure suit, and a parachute descent from above some altitude (guessing, but it's probably lower than 15 km (49,000 ft)) will result in depletion of this oxygen reserve and hypoxia well before descending into air thick enough to breathe directly.