No, you do not need a ferry permit. If you wish to fly inside the Mode C veil, it is sufficient to reach out to ATC and ask for permission. For a 10 minute flight that stays low to the ground, I'm sure they'll be happy to help. It makes it easier on them if you can fly when they aren't slammed, but since you're not in their airspace, they are pretty understanding.
You might be surprised by the number of transponder-less aircraft flying under the Mode C veil: lots of old planes which never had electrical systems, lots of new ultralights, and there could even be gliders around. You won't be the only one up there without ADSB-Out.
FWIW, I expect that your plane can fly quite some distance with everything off but the transponder. They typically consume 10-20W, so you should have several hours' worth of energy.