Most people can function at 12,000 ft. Pilots do as well and don't need oxygen there.
Above that altitude, however, it gets worse very quickly. By 14,000 ft (where oxygen is always required for crew) most people will be noticeably short of breath and between 16,000 – 17,000 ft they may pass out.
I was once that high in the mountains. We were hiking, so some acclimatisation was taking place, though we were climbing faster then would be ideal. We first noticed being weakened as we neared the 13,000 ft. Then we spent two days there, so it got a bit better, but as we climbed above 15,000 ft, one (of the 15 or so) got altitude sickness and somewhere below 16,000 ft one had acute symptoms of hypoxia and had to turn back I think two of us turned back as we were barely able to walk. That matches the altitudes where pressurisation or supplemental oxygen is needed quite well.
Now if you get acclimatised, you will be able to function normally at those altitudes. That, however, requires spending days to weeks at the altitude. You'll never manage to get acclimatised just by flying.
Pilots acclimatised because they live at higher altitudes do handle the altitude better. But the regulations need to be simple and work for the worst case with some safety margin. Also, Carlo is right that brain requires a lot of oxygen, so concentration suffers even if you are not doing any physical exercise and don't feel physical weakness.