The delta wing was, in fact, very widely used in fighter designs of the mid-late 50s, 60s, and early 70s. The Mirage series (even the latest models) and their derivatives, the MiG-21, the F-106, and the A-4 are some of the more iconic designs to have used the wing.
However, as time went on, shifting philosophies on aircraft design, along with technological and manufacturing advancement meant that the speed and acceleration attainable with Delta wings could be reached by other means and manufacturers were free to work with other types of wings that don't suffer from the Delta's weaknesses.