Would I just file an IFR flight plan to the airport in question, and expect to be cleared for the visual approach if there are no instrument approaches at the destination airport?
Yes. If the airport in question has no operating control tower you need to cancel your ifr flight plan after landing if you choose to not do it in flight. Once you land you can cancel with air traffic contol, if you are able to establish radio communication, or cancel with a flight service station. It is usually going to be easiest to cancel with the local approach controller in flight, if there is no operating tower. Remember all ifr traffic, landing and departing, will not be able to receive their approach clearance or ifr release until you cancel.