who operates the hoist in the Coast Guard helicopter SAR teams?
This depends on country and other factors, perhaps aircraft type etc.
In general, the winchman descends on the end of the winch cable and does not operate the winch.
In the UK, helicopter search and rescue were, until recently, mostly provided by the RAF. HM Coastguard HMCG (now part of Maritime and CoastGuard Agency - MCA) operate SAR helicopters too and will mostly take over the SAR role.
The standard SAR crew is made up of four members: two pilots, one of whom is the aircraft captain, a radar operator who acts as the winch operator at the rescue scene and a winchman, normally trained to paramedic standard, who will supply immediate first-aid and recovery services at the rescue site.
From RAF Sea King
Sometimes the role is taken by aircrew normally responsible for loading/unloading:
If you are part of aircrew, you would be responsible for loading and unloading aircraft payloads, ... You would also take on extra duties according to the aircraft type and its operational role. For example, ... you might train as a winch operator for search-and-rescue operations.
From RAF non-commissioned aircrew