Maximum speed is not the best speed to fight. Two major factors in a fighter must be taken into account:
- Wing loading.
- Thrust-to-weight ratio (T/W ratio).
Wing loading is the weight of the airplane distributed across the total area of the wing. This tells the aircraft's ability to turn sustained.
The T/W ratio is the amount of thrust developed by the engine divided by the weight of the aircraft, and this will tell the acceleration the aircraft is able to achieve in level flight. When both things are taken into account, you get a second parameter that is the aircraft ability to climb (climb rate).
When air war developed in the WW1/2, one rule of thumb for fighter pilots was that speed/altitude was life. If you examine the aircraft as a physical system of energy, you have two kinds of energy at your disposal: Kinetic energy (the actual rate of aircraft movement) and potential energy, the altitude of the aircraft. When you dive, you convert potential energy into kinetic energy. When you climb you turn kinetic energy into potential energy, until the point where your engine is the only thing that is inputting energy into the system (so you reach the stage where instantaneous climb rate ends and you start to see the sustained aircraft climb rate).
When your aircraft turns, it will be slowed a little bit, and the increase in induced drag generated by the turn will decrease your speed slowed. In the fighter pilot jargon, you are burning energy. There are two (again) kinds of turn, instantaneous and sustained.
The fighting styles where divided into two kinds: Turn and Burn and Boom and Zoom.
Turn and burn means that both fighters try to outturn each order in order to get into their six o'clock position in order to fire, it's a horizontal fight in circles. Boom and Zoom is a climb fight where the higher positioned aircraft dives into the other in order to make a pass, fire and then climb again in order to not lose airspeed (and thus energy).
Quickly the boom and zoom style was adopted. Experienced pilots only entered combat when sure to have higher altitude than their enemies, because altitude was a reservoir of energy to be exploited in combat.
A turn and burn styled fighter/pilot would quickly slow down and be unable to either attack or flee from a higher positioned aircraft.
So far so good.
After the war, all efforts went on to produce ever faster aircraft and aircraft able to reach high altitudes quickly. This has dual advantage of giving a better high altitude position and allowing such fighters to engage incoming bombers.
All changed when better missiles and onboard radars became available. Not that current fighters don't need to be able to dogfight, but because the missile became the major combat element. Two kinds of missiles come to my head at this stage. AAMs and SAMs.
The increasingly powerful radars (even onboard) meant that aircraft might very well attack things that are above it, meaning that the boom and zoom tactics might go wrong. Secondly, a radar allows one to see the enemy way before visual contact is made, giving more time to climb to a similar altitude, etc.
If you are flying very high, you will be seen on enemy radar far away, and missiles directed by those radars will come from various directions. To enter enemy airspace, you need to stay low, where air resistance is bigger and most aircraft can't reach the stated figures of speed. (See F-111 or Su-24 for example).
Bombers won't be flying at stratospheric levels like B-29 anymore, due to the same reasons. They need to enter enemy airspace at low altitude in order to have a chance of surviving. If they use standoff weapons, those will fly low too. So, if you are escorting a bunch of bombers in a mission, or you are trying to intercept those, they will not be very high.
When a onboard radar tries to lock against a low flying aircraft, there is a problem. Earlier radars where not able to do so at all. More modern versions (called look-down/shot-down) are able to lock a target against terrain, but not with the same performance as when they are looking up. Clutter (false echoes) is removed via software. Infrared detectors work better at high altitude (where air is colder and a hot fighter is more visible), than against the hot ground below, and so on.
So, all in all, fighter combat in intensive theaters will be going to be done at middle or low level, where speed is slower due to higher drag. Even then, all fighter aircraft have something called corner velocity, where the aircraft is able to sustain their most tight turns, and those speeds are usually far from being top speed.
Another factor is engine consumption. Only recently did supercruising aircraft came to be operational. Forth-generation aircraft where not able to break the sound barrier without using afterburns, which both increase aircraft detectability (due to increased heat signature) and fuel consumption.
The same aircraft that might travel 1000 km without using afterburners, might very well be unable to do more than 300 km while afterburning all along.
All things taken into account, an aircraft with more missiles, better radar and longer loiter time is preferred above fighters like the British Lightning or the F-104 which had a very high speed and climb rate, but were short-legged and lugged only a pair of missiles.