No, you do not need a ferry permit to fly without the transponder[*]. If you wish to fly inside the Mode C veil, it is sufficient to reach out to ATC and ask for permission. For a 10 minute flight that stays low to the ground, I'm sure they'll be happy to help. It makes it easier on them if you can fly when they aren't slammed, but since you're not in their airspace, they are pretty understanding.

You might be surprised by the number of transponder-less aircraft flying under the Mode C veil: lots of old planes which never had electrical systems, lots of new ultralights, and there could even be gliders around. You won't be the only one up there without ADSB-Out.

FWIW, I expect that your plane can fly quite some distance with everything off but the transponder. They typically consume 10-20W, so you should have several hours' worth of energy.

[*] In general flight without an alternator might not be allowed, though. This depends on the Comprehensive Equipment List, or Minimum Equipment List, or Kinds of Operation Equipment List, or basic certification standards in effect when the plane was certified. If the alternator for your specific plane is on one of those lists as required equipment then the plane would be considered not airworthy and would require a ferry permit.