Where noted, the information in this answer is from Advisory Circular 103-6, which was issued in 1983, but which apparently is not actually obsolete.
Is it possible to get the equivalent of flight following or at least call up ATC to let them know you are there, if you are planning on flying at higher altitudes? I am used to doing this for a D airspace if I am flying only just over or near it if they are not busy.
It's certainly legal to call up ATC while you're flying an ultralight, whether it's Part 103 or not. The relevant FCC regulation, 47 CFR § 87.18(b), says that aircraft are "licensed by rule and [do] not need an individual license," and ultralights are aircraft, so an airband radio on an ultralight counts as being licensed.
Advisory Circular 103-6 also recommends that ultralight pilots contact ATC using the ordinary radio frequencies (but I believe it was written when all airband radio users needed individual licenses).
I don't know whether or not ATC gives flight following to ultralights. Advisory Circular 103-6 says that separation services "will not be available to ultralight pilots." But there's no harm in asking.
If so, what would the phrasing of that sound like when you don't have a callsign?
There's the tricky bit.
The regulation governing this is 47 CFR § 87.107(a), which states that you should identify using one of the following four options: your "aircraft radio station call sign"; the aircraft type and registration number; your radiotelephony designator and flight number; or a temporary identifier for "organized flying activity."
However, if you're flying a Part 103 ultralight, then in all likelihood, you don't have any of those.
Advisory Circular 103-6 states that "ultralight operators should state the word 'ultralight' followed by the call letters assigned by the F.C.C. on your radio license, i.e., 'Ultralight 12593U.'" Assuming you don't have a radio license, it seems like the best you can to is to simply use the word "ultralight" as your call sign.
14 CFR 103.17 says that I require prior authorization to enter class B, C or D airspace. What does this look like? Can I take a handheld radio and ask for permission on the tower frequency? Or is this something I'd need to get through written permission from a FSDO or the tower directly?
Advisory Circular 103-6 says, "Requests for authorization to operate an ultralight vehicle into [controlled airspace] should be made by writing, telephoning, or visiting the air traffic control facility having jurisdiction over the airspace in which you wish to operate. Requests for such authorization via air traffic control radio communication frequencies will normally not be accepted, since it may interfere with the separation of aircraft."
That said, I'm not aware of any special legal meaning of the phrase "prior authorization." So on the off chance that ATC does tell you that you're allowed to enter the airspace, presumably you can do it.