Let's say I'm over a city. For the sake of being specific, I'm not talking about downtown areas with skyscrapers, I'm referring to residential / commercial mixed areas, typical "suburbia". If the city is 15 miles in diameter and densely populated, I would think (erring on the side of caution) I would consider my glide distance when determining a MSA.
Yes, that's what the FAA expects you to do per 91.119(a): Consider your aircraft's characteristics (glide ratio, landing distance required), your altitude, and the availability of landing areas. Select an altitude that lets you land (ditch, crash, whatever) with minimum risk to persons and property below you.
If I was directly over the center of this city, that would easily make my MSA 4,000ft AGL (using a Piper Cherokee 140 in this example).
That is not an unreasonable determination if the area you're overflying is heavily congested (streets are full of cars/people, no parking lots, fields, or parks to dead-stick into without hitting a person/car/building).
Note, however, that your emergency landing doesn't have to be a pretty one: Light planes routinely fly through the Hudson River SFRA at low altitudes - well below 4,000 feet - and the West Side Highway or Hudson River Park are not particularly hospitable landing sites (you would be hard-pressed to execute an emergency landing without endangering persons or property). That means your "emergency landing" strategy is going to be the same as US Airways Flight 1549: You're ditching in the river. You can safely do that in a Cherokee or a Cessna from 1000 feet or less most of the time since the surface of the river is not as congested as the streets and parks of Manhattan: You can turn out over the river and be clear of people and vessels.
To address your specific bullet point questions:
What would be considered a reasonable landing spot over a congested area?
One in which you can set your aircraft down without undue hazard to persons or property. (Yes, I realize that's very close to a circular definition.)
You don't have to make the landing pretty: you may damage the plane in the process, and you might damage some property or even kill a person on the ground in the process - sometimes you just can't foresee all the problems - but you should be looking out for landing areas that minimize the risk of injuring or killing people (priority #1) or damaging other stuff.
What would be considered "undue hazard?"
"Undue Hazard" is a legal concept, and it will be up to your lawyer and the FAA's lawyers to determine if the hazard posed by an emergency landing you make is "undue" (excessive, disproportionate to what operating at a "normal" altitude would have imposed).
For a common-sense definition look at your operation and imagine you were standing on the ground: If you would be asking yourself "What the heck is that pilot thinking flying so low?!" then there's a good chance your altitude is low enough that an emergency landing would constitute "undue hazard" to the folks on the ground.
What other elements come into justifying an MSA?
We already talked about the big ones: Aircraft performance & what kind of area you're flying over.
You also need to consider at least two other factors:
Pilot Skill
You know what the best short-field performance of your plane is from
the POH, but can YOU reliably execute a landing and hit those
numbers? Would you bet someone else's life on it?
Weather Conditions
What if your only landing area has a tailwind?
What if you have to fly into a headwind to reach your landing area - will you make it given your best glide speed and glide ratio?
