This is a good question, and I don't feel the other answers get at the essential part which is:
Is it optimal to climb with the flaps deployed?
As with any optimal question, the answer relies on what it is we wish to optimize. It's worth examining two goal states:
- To climb to altitude as quickly and efficiently as possible. For instance:
- the winds aloft are so good that the pilot wants to minimize the time spent down low
- the pilot would like altitude for aerobatics
- the pilot would like to do high altitude testing
- To go from point A to point B while employing best safety practices all while reasonably minimizing time and/or fuel costs.
Why use the flaps in the first place?
In general, getting off the ground and clearing any obstacles while remaining close to the airport is generally considered best practice, even if it means burning a little more gas. Depending on a number of design choices, the climb angle with flaps can be much improved, which means that obstacle clearance is better and in the event of a takeoff emergency the plane has a lot more runway in front of it, or it's not so far from the airport.
The advantages of taking off with flaps down:
- Wheels leave the ground at a lower airspeed, eliminating rolling resistance. (It's surprising how much drag comes from those tires, esp. in grass and soft fields.)
- Climb angle is better
- From the DA40 manual, pgs. 5-14 and 5-16, STP climb rate with flaps is 9.7deg (1160fpm @ 67kts) and without is 7.8deg (1050fpm @ 76kts).
- Climb rate might be better
So what are the disadvantages?
Increased drag at higher airspeeds
As @ROIMaison shows in this answer, for a Clark Y airfoil the L/D ratio with flaps deployed isn't even remotely close to the normal airfoil. At higher airspeeds this loss of efficiency is acutely felt.
Of course, the DA40 has a much more advanced airfoil and so the spread might be much closer together. Diamond's airfoils come from gliders, and gliders use flaps in low-speed flight in order to turn more quickly. As you might imagine, gliders are optimized for efficiency, so it's fair to reason that the flapped L/D ratio for DA40's airfoil is potentially much better than the venerable Clark Y's.
In case the link between drag and climb rate is not immediately obvious, the more drag there is the less surplus energy is available to increase the plane's potential energy, i.e. to climb.
The engine cowling is designed to provide appropriate cooling at relatively high airspeeds. There's a certain thermal inertia which protects the engine for a minute or so, but after that temperatures start reaching critical points. It's important to nose over and pick up airspeed in order to improve cooling.
Propeller inefficiency at slower speeds[*]
For a fixed-pitch cruise prop, efficiency suffers quite significantly at lower airspeeds. Speeding up will gain some extra propeller and engine performance.
[*] Note that this doesn't apply to constant-speed props.
With the above in mind, we can see that the best practices of getting up off the ground quickly, with plenty of runway to spare, and with enhanced obstacle clearance is a Good Thing (TM). These goals are largely met by 500', so this is an opportunity to reevaluate our optimal process. Do we still want to climb, or do we want to go somewhere as well?
Unfortunately, I don't have any basic sense of whether it's generally optimal to continue to climb with the flaps deployed. It might even depend on the particular plane model whether absolute climb rate is better with flaps up or down. If the climb rate is worse with the flaps down, then the answer is clearly to get them up as soon as practical.
Supposing climb rate is better with flaps deployed, the situation doesn't really become clear. If the winds aloft are favorable, then getting to altitude quickly is valuable. But more valuable than proper engine cooling? Hmmm...