I have read in several sources that you are supposed to log instrument time when you cannot orientate the airplane by outside, visual cues alone. This seems clear enough. But I've also read that roughly 10% of flight time should be on instruments. Now, the instrument rating requires 40 hrs on instruments or in poor visibility conditions, so can we infer that pilots would receive their rating after approximately 40x400 = 16,000 hrs? Is this estimate approximately correct? Also, does all of this time require an instructor?
I think you are conflating two different things. Most IFR rated pilots have very little actual instrument time. Even airline pilots who fly in all kinds of weather have a low percentage of their time in the clouds. A lot of it has to do with where they fly—at over 30,000' they are often above the clouds. I would imagine that it varies a lot with the type of flying that the commercial pilot does. A freight hauler in the Northeast probably has a lot more instrument time than one in Arizona. Likewise, someone who flies their small aircraft a lot on business might have relatively more actual instrument time simply because they will be flying in the weather rather than above it.
If you look at the instrument requirements in CFR Part 61, the training requirements for the Instrument say "…hours of instrument training using a view-limiting device…". So it is possible to get your instrument rating with 40 hours of flying "under the hood" and never fly in actual instrument conditions. (You can also get some hours in a flight simulator.) And when you take your practical test for these ratings, you will be flying under the hood and not in actual instrument conditions.
The same language is used in the Commercial rating as well. In fact, you could have a CFI who has never flown in actual instrument conditions teach you how to fly IFR!
To get an instrument rating you need 40 hours of hood time. Only 15 hours of this is required to be with an instructor. Lots of people save money on their rating by flying with a pilot friend or fellow student.
Once you have your instrument rating, you need to maintain currency by flying 6 approaches, holding patterns, and tracking courses in actual instrument conditions or with a view-limiting device. If you listen to the radio on a nice weekend, you will hear lots of pilots requesting practice approaches to maintain currency. Usually they are flying with a friend rather than an instructor.
Instrument time is, as you say, when flight is conducted solely by reference to instruments. This can further be broken down into actual and simulated time. Actual time is when you are flying in cloud or visibility below VFR minimums and cannot navigate by visual references. Simulated time is when you artificially limit your outside view and requires a safety pilot.
The 10% rule is a rule of thumb of what instrument time you may see in the normal course of flying. It is not applicable to training. During training you may fly 95% of your time as simulated instrument (everything but taxi, takeoff and landing). For this time you need only another rated pilot who can be a safety pilot. It does not need to be an instructor.
If you have a private pilot rating and nothing more you can get an instrument rating in tens of hours, not hundreds or thousands of hours.