There is no special certification that gives out the title of a "military aircraft". Any aircraft that is owned and operated by the military is, thus, a military aircraft.
Even the fastest aircraft can be easily shot down by modern missiles. For fighters, missile evasion is more about turn rate than speed, and for large non-combat aircraft, countermeasures are used. Speed is dictated by mission requirements instead.
The Antonov An-2 is operated by at least 50 national militaries around the world - more than the F-16 - which makes it one of the most-widespread military aircraft. It's popular with militaries for its ruggedness and rough field capabilities, and thousands have been built for military use.
It also happens to hold a solid standing on lists of the slowest production airplanes, with a top speed of just 140 knots and no specified stall speed. Few factory-built fixed-wing aircraft are slower, and they're all considerably smaller.
Being manned, fixed-wing, powered, mass-produced, and employed operationally, not for training, it ticks all the boxes to count as a military transport aircraft, without any small print. That should be example enough to make the answer clear: there isn't. Military aircraft can be as fast or as slow as the mission requires.