Yes, both pilots are required to see and avoid at all times when operating in VMC conditions. Typically the military pilots have radar used to pickup other aircraft that can help them, but ultimately it still comes down to looking outside.
We have military training routes charted here in the US that are used by military aircraft when operating at high speeds, and pilot should be especially vigilant when operating in the vicinity of one of these routes. Use all available information: Get flight following and contact the frequency on the chart for current operational information, and above all keep an eye out for them!
The Aeronautical Chart Users Guide shows an example of how they chart it on VFR Charts:
Military Training Routes (MTRs) are shown on Sectionals and TACs.
They are identified by the route designator: 
. Route
designators are shown in solid black on the route centerline,
positioned along the route for continuity. The designator IR or VR is
not repeated when two or more routes are established over the same
airspace, e.g., IR201- 205-227. Routes numbered 001 to 099 are shown
as IR1 or VR99, eliminating the initial zeros. Direction of flight
along the route is indicated by small arrowheads adjacent to and in
conjunction with each route designator.
The following note appears on Sectionals and TACs covering the
conterminous United States.
There are IFR (IR) and VFR (VR) routes as follows: Route
identification: a. Routes at or below 1500’ AGL (with no segment
above 1500’) are identified by four-digit numbers; e.g., VR1007, etc.
These routes are generally developed for flight under Visual Flight
Rules. b. Routes above 1500’ AGL (some segments of these routes may
be below 1500’) are identified by three or fewer digit numbers; e.g.,
IR21, VR302, etc. These routes are developed for flight under
Instrument Flight Rules.
MTRs can vary in width from 4 to 16 miles. Detailed route width
information is available in the Flight Information Publication (FLIP)
AP/1B (a DoD publication), or in the Digital Aeronautical Chart
Supplement (DACS) produced by AeroNav Products. Special Military
Activity areas are indicated on the Sectionals by a boxed note in
black type. The note contains radio frequency information for
obtaining area activity status.
On IFR charts, they are similar:
MILITARY TRAINING ROUTES (MTRs)
Military Training Routes (MTRs) are routes established for the
conduct of low-altitude, high-speed military flight training
(generally below 10,000 feet MSL at airspeeds in excess of 250 knots
Indicated Air Speed). These routes are depicted in brown on Enroute
Low Altitude Charts, and are not shown on inset charts or on IFR
Enroute High Altitude Charts. Enroute Low Altitude Charts depict all
IR (IFR Military Training Route) and VR (VFR Military Training Route)
routes, except those VRs that are entirely at or below 1,500 feet
AGL. Military Training Routes are identified by designators (IR-107,
VR-134) which are shown in brown on the route centerline. Arrows are
shown to indicate the direction of flight along the route. The width
of the route determines the width of the line that is plotted on the
chart:
Route segments with a width of 5 NM or less, both sides of the
centerline, are shown by a .02” line.
Route segments with a width greater than 5 NM, either or both sides
of the centerline, are shown by a .035” line.
MTRs for particular chart pairs (ex. L1/2, etc.) are alphabetically,
then numerically tabulated. The tabulation includes MTR type and
unique ident and altitude range.
As far as who is at fault, it is a joint responsibility so everybody would be in most cases.
