The questioner seems to have noted that the basic wing with flaps retracted provides a high ratio of L/D (or Cl/Cd). Where L denotes lift, Cl denotes lift coefficient, D denotes drag, etc.
We can certainly scale up the basic unflapped wing to provide as low a landing speed as we wish, although landing will be tricky due to the flat glide path. Flaps help with landings by increasing the drag coefficient as well as the lift coefficient, making the glide path steeper.
The main problem with this approach is that for cruising flight, not only do we wish to achieve a high L/D ratio, we wish to achieve it at a high airspeed. Lift is proportional to lift coefficient times airspeed squared, and in cruising flight, lift cannot be larger than weight. If the wing is too large, it will be optimized to deliver its peak L/D ratio at a much lower airspeed than we wish to cruise at. In such a case, at our intended cruise speed, if we increased the angle-of-attack to the max L/D angle-of-attack, the wing would be making way too much lift, and we'd pitch up into the start of a loop. To keep the flight path level, we'd have to trim for an angle-of-attack far to the right side of the peak L/D ratio, as portrayed on the polar curve of L/D ratio versus airspeed. In other words, we'd have to trim to an angle-of-attack much lower than the angle-of-attack that delivers the max L/D ratio. We'd end up with more drag than we'd have if the wing were smaller.
The situation is not unlike that of a glider pilot wishing to achieve a flat glide at a high airspeed. The glider gets a lower sink rate and a better glide ratio at that high airspeed when the wing loading is high than when the wing loading is low, because the wing may be operated at the angle-of-attack that yields the best L/D ratio, rather than at some much lower angle-of-attack. So water ballast is carried.
In powered flight, the equations are a bit different, and simply adding weight to the aircraft never improves high-speed cruising performance. But if the wing were designed to be large enough to give an acceptably low landing speed without using flaps, then scaling down that wing to a smaller size certainly would improve high-speed cruising performance. And that's why it's worth carrying around the weight and internal volume of a complicated flap system-- because we can make the wing smaller, so that when the flaps are retracted, it is optimized to deliver its peak L/D ratio at a high airspeed.
The basic thrust of this answer remains the same regardless of whether we are trying to achieve a low landing speed by scaling up the wing in all dimensions, or by only increasing the chord. In general, a high peak L/D ratio is associated with a high aspect ratio, and therefore a small wing chord. However, if in cruising flight we know we must fly our scaled-up wing at some airspeed that is much higher than its maximum L/D airspeed, it's possible that we'll have a better L/D ratio at that high airspeed if we've scaled up the wing by expanding the chord alone than by expanding all dimensions. Because the curve of L/D versus airspeed may be less "peaky" with the lower aspect ratio than with the higher aspect ratio. But the better solution is to keep the high aspect ratio, and keep the wing small enough so that it can actually be flown at its maximum L/D ratio at the intended cruising speed. Then we "scale up" the wing for landing by extending the flaps.
Of course, extending the flaps does much more than just "scale up" the wing. At full extension, the designer's goal is to minimize the stall speed, so maximizing the lift coefficient is the priority. He or she is free to choose a configuration that maximizes the lift coefficient, with no concern for minimizing the drag coefficient to maximize the L/D ratio. As noted above, an increase in drag is actually helpful during final approach-- it is easier to guide the aircraft to the intended touchdown zone if the power-off glide path is not too flat. (And no, despite the questioner's suggestion, we cannot eliminate the extra drag created by the extended flaps in cruising flight simply by "closing the slots".) The flaps are designed purely to make the wing's lift coefficient as high as possible, while the airfoil of the clean wing is designed to optimize the ratio of L/D or Cl/Cd. Therefore, to achieve the same low stall speed simply by scaling up the unflapped wing -- either chordwise, or in all dimensions -- would require a much greater increase in wing area than the area that is actually added by the deployed flaps.
Related --
(Q) Why would a glider have water ballast? If it is trying to stay aloft without an engine, wouldn't it be better to be as light as possible?
(A) For a large commercial plane on landing, does the L/D ratio increase, decrease, or not change much?
(A) For a large commercial plane on landing, does the L/D ratio increase, decrease, or not change much?
(A) Why and when to use flaps?