Short answer: you're expected to follow the rules you mentioned unless it's an emergency.
The regulations are in 91.185 and they say (among other things):
Unless otherwise authorized by ATC, each pilot who has two-way radio
communications failure when operating under IFR shall comply with the
rules of this section.
If the failure occurs in IFR
conditions, or if paragraph (b) of this section cannot be complied
with [i.e. the aircraft doesn't enter VMC], each pilot shall continue the
flight according to the following: [route, altitude, clearance limit
As written, that seems clear: if you aren't in VMC and don't enter VMC, you "shall" continue your flight according to the usual rules. The AIM 6-4-1(c) says the same thing:
In the event of two-way radio communications failure, ATC service will
be provided on the basis that the pilot is operating in accordance
with 14 CFR Section 91.185. A pilot experiencing two-way
communications failure should (unless emergency authority is
exercised) comply with 14 CFR Section 91.185
The whole point of 91.185 is that you don't want to fly around literally blind in IMC with no way to know what other aircraft are out there. And especially in busy airspace, ATC and other pilots certainly don't want an aircraft they can't see behaving unpredictably. If the comms failure also took out your transponder, for example, then ATC could only track you on primary radar, which is quite limited.
Having said that, the AIM is clear that if the PIC decides that the situation is an emergency, then he can do whatever seems best. Comms failure by itself isn't an emergency (see the AIM 6-4-1(b)), but every situation is different and if other equipment fails or you have other safety issues to deal with then diverting might be the best decision. There are just too many possible scenarios to have a strict yes/no rule.