Would a homemade lawn chair balloon be visible on ATC and collision avoidance radar?
It might be visible to ATC, but, at best, only as one of several inconspicuous unidentified red dots.
It wouldn't appear on aircraft collision avoidance displays (TCAS) and would not trigger any TCAS alerts.
Radar cross section
Balloons, and clusters of balloons are mostly air (or similarly insubstantial gas). There is a thin layer of plastic but many plastic materials (e.g. mylar) are transparent to radar, or only weakly reflective.
It is for this reason that aerostats and high altitude balloons carry radar-reflectors when they need to be detected by radar.
Whether the chair itself has a detectable radar cross section depends partly on what materials it is constructed from. I suspect most available lawn chairs are not constructed from solid sheets of metal fixed at right angles but, at best, from a relatively tiny amount of relatively small-gauge metal tubing.
I'd be surprised if any improvised lawn-chair balloon had the same radar cross-section as a glider - and those are allegedly hard to see on primary radar.
Some helium balloons are made from metallised mylar (because it keeps the helium in for longer). These may be much more reflective to radar. The Calgary aviator pictured was said to have used "oversized party balloons" - they don't look like the metallised balloons that are commonplace where I live.
A perfectly conducting metal sphere of 1.15m diameter has a radar cross section of 1 m²
Some other radar cross section values:
Human: 1 m²
small combat aircraft: 2–3 m²
Cargo aircraft: up to 100 m²
This suggests that military radar might display a return to an operator. A civilian primary radar might be less likely to display a return to an operator.
Primary radar
So far as I can tell, most civilian ATC and powered aircraft (TCAS etc) don't really make much use of primary radar to detect other airborne vehicles. They rely on transponder returns (i.e. secondary radar)
Even a large metal object like a Boeing 777 can disappear from the view of ATC and other aircraft when it turns off its transponder.
The Guardian newspaper reported
Last month, the Swedish government complained that a Russian military aircraft had been flying near its airspace with its transponders turned off to avoid being spotted by civilian radar, and nearly collided with a passenger jet.
I'd be surprised if an improvised lawn-chair balloon was detectable where a Tu-95 was not.
What ATC see
This is from what is said to be a radar recording of flight 77 (at 4:18) probably at Dulles after controllers were aware of the prior 9/11 attacks.

Flight 77 had turned off its transponder, it is one of the small red dots at the left. Here's a closer look at that

Flight 77 is the red dot about 20% in and up from bottom left. There are other red dots that come and go but this one tracks steadily from left to right. I'd guess an especially alert controller who is not very busy would notice this sort of dot. Even if the radar display doesn't filter it out, unannounced slow-moving lawn-chair balloons appearing from the suburbs might not always be noticed (though see James' comment to Peter's answer which suggests they often do notice and act on such targets).
Transponders
Radios and transponders designed for hot-air balloons are available (example) and I've read that
Transponders are mandatory for flights in some types of airspace in many parts of Europe from March 2008 onwards. This includes balloons and other light / recreational aviation and aircraft.
So I suspect that authorities will expect aviators in lighter-than air vehicles will take appropriate measures to be in communication with ATC/other aircraft and perhaps to be visible to secondary radar in airspace where they may come into conflict with other aircraft.
Kevin Walsh
Heres a news report from The Milwaukee Sentinel - Jul 19, 1984

Snopes reports
He had wrapped his tether lines in foil in the hope that they would show up on radar, and, sure enough, he was picked up on the screens of controllers at Boston's Logan airport
So you probably need to take specific measures to make yourself more likely to show up on primary radar.
Conclusion
When attaching balloons to your lawn chair, first scrutinize the intertubes for designs for large improvised radar reflectors you can dangle from your vehicle. Hope that nearby on-duty ATC controllers have their displays set to show primary radar returns and not just secondary radar returns.