For a large class B complex, the same approach control frequency may be working aircraft both inside and outside the B, with some of those outside the B in a class C or D.
Establishing communication with ATC is sufficient to enter the C or D unless the controller says no, but you still have to keep out of the B unless they say yes. If the rule were the same for all 3 (i.e. you may enter the B unless specifically told no), then most initial contacts would get a reply to the effect of "N123AB, radar contact, remain clear of Bravo." Lots of words used for lots of aircraft that may never intend to enter the class B.
The class C and D don't tend to need as much tight control as the class B, so "just" establishing communication is sufficient. But just because the controller is talking to you, doesn't mean that they're okay with you entering the class B.
If we think about it in terms of computer logic, it'd be simple for the controller's first reply packet to include bits to indicate 'cleared into class B', 'cleared into class C', and 'cleared into class D', all of which are set as desired and any of which could later be changed as necessary. But voice communication has nuances that digital processing lacks, and the amount of fast-moving (generally, airline) traffic in class B makes it desirable to have a more explicit clearance into that airspace required, and a stronger "stay out unless you have permission" requirement in place.