Generally speaking airliners are always able to be in contact with ATC, either by VHF (regular radio), HF (usable over longer distances, e.g. on oceanic crossings), or satellite communication (possibly relayed through their company dispatch). It is also possible for one aircraft to relay messages for another (for example if one aircraft's HF radio has failed they may contact a nearby aircraft on VHF to relay messages to ATC).
Because they can usually talk to ATC even when they are out of radar contact airliners can and will provide position reports, and ATC updates their position information the old fashioned way (which we talked about over here). The controllers can then assign altitudes, speeds, and headings/routes in such a way that no two aircraft attempt to occupy the same piece of sky at the same time. (This is generally the way oceanic routes are handled. There are also air-to-air frequencies for aircraft flying these routes so they can coordinate with each other if necessary.)
In the event an aircraft is out of contact with ATC - whether through a radio blind spot, an equipment failure, or for some other reason - they rely on a combination of lost communication procedures (here's a quick reference from the FAA) and "see-and-avoid" (if you see another aircraft avoid hitting it) until contact can be re-established.
The use of standard lost communication procedures also allow the controllers to continue updating the aircraft's position in a non-radar environment (they can estimate progress based on the standard procedures and the aircraft's prior position reports/speed/wind data). A larger block of airspace will be reserved in such situations to account for any error.