I've been thinking of getting an ultralight as a low cost way to pursue flying. And I've been wondering, when my instructor finally signs me off for a solo flight, am I allowed to do that in an ultralight? And, if so, is there equipment I would be required to have that isn't standard on an ultralight?
Your solo time must be done in the category and class of aircraft for which you are seeking a pilot certificate (the FAA doesn't come out and say it in those words, but that's what the "aeronautical experience" requirements basically boil down to). You must also pass a pre-solo knowledge exam and receive instruction in the particular make and model of aircraft you will be solo-ing in (the second of which is impossible in an ultralight: You cannot receive dual instruction in that aircraft as by regulation it can only have one seat).
As you don't need a pilot certificate to fly an ultralight you could just get one, get some basic instruction on flying it (from someone who has flown them before), and go fly. Some ultralights are pretty fine basic aircraft thanks to modern materials and construction techniques.
Your private pilot training & solo experience would be helpful in flying certain types of ultralight aircraft, but it's not required as long as you abide by the regulations applicable to ultralights.
Similarly experience in the more "airplane-like" ultralights may help you out in the rest of your private pilot training, but that experience isn't required (and not really "loggable" toward your training, through you can put it in your logbook if you wish -- just log the time as "Ultralight Flying" or something similar.)