Short answer: you can fly in Yosemite Valley at 500ft AGL (most likely); there's no special regulation applicable there; the wording on the sectional is a non-regulatory 'encouragement' only.
By regulation, the minimum altitudes under VFR are in 14 CFR 19.119:
(a) Anywhere. An altitude allowing, if a power unit fails, an
emergency landing without undue hazard to persons or property on the
surface.
(b) Over congested areas. Over any congested area of a city, town,
or settlement, or over any open air assembly of persons, an altitude
of 1,000 feet above the highest obstacle within a horizontal radius of
2,000 feet of the aircraft.
(c) Over other than congested areas. An altitude of 500 feet above
the surface, except over open water or sparsely populated areas. In
those cases, the aircraft may not be operated closer than 500 feet to
any person, vessel, vehicle, or structure.
I don't know what Yosemite Valley looks like and whether it's "congested" or not (see this question), but it's very likely that you could operate legally at 500ft AGL there, if not lower.
What the sectional says comes from AC 91-36, Visual Flight Rules (VFR) Flight Near Noise-Sensitive Areas. But as the AC itself says, it only encourages pilots to fly above 2000ft AGL (emphasis mine):
This Advisory Circular (AC) encourages pilots making VFR flights near
noise sensitive areas to fly at altitudes higher than the minimum
permitted by regulation
If you read the whole thing, you can see the not-so-hidden message:
The Federal Aviation Administration (FAA) receives complaints
concerning low flying aircraft over noise sensitive areas such as
National Parks, National Wildlife Refuges, Waterfowl Production Areas
and Wilderness Areas. Congress addressed aircraft flights over Grand
Canyon National Park in
Public Law 100-91 and commercial air tour operations over other units
of the National Park System (and tribal lands within or abutting such
units) in the National Parks Air Tour Management Act of 2000.
[...]
Adherence to these practices is a practical indication of pilot
concern for the environment, which will build support for aviation and
alleviate the need for any additional statutory or regulatory actions.
In other words, 'please be responsible and considerate because if you aren't, Congress might force us to introduce more regulations like we had to do with the Grand Canyon'. Practically speaking, I guess that if you fly down Yosemite Valley at 500ft AGL every day, even if it's completely legal you can still expect a call from the FAA asking why you're doing it and 'encouraging' you ("cough 91.13 cough") to fly higher for 'safety' reasons.