I was thinking of contacting Clearance/Ground/Tower/approach and omitting the airport name to shorten the transmission. If the freqs are not used by any other nearby airports, I figure there would be no confusion which airport I'm talking to. Is there any other concerns that make this not a good idea?
When every transmission to the tower begins with "Executive Tower...." that is the keyword that makes the controller's ears perk up and start taking notes.
If you don't say the magic words, your controller may be a step or two behind as you rattle off your request.
Also, you think you're on the right frequency, and you think you know who you're talking to, but what if you mis-tuned the frequency knob by a digit or two, or forgot to hit the COM1/COM2 button? You could be talking to the wrong receiver and not know it. That is how airplanes sometimes land at the wrong airport.
Every initial transmission should start with the intended receiver; that is part of the rules and just common sense. Don't try to shorten transmissions and end up causing more work for everyone.
From the Aeronautical Information Manual: AIM (emphasis is mine)
4 − 2 − 3. Contact Procedures
a. Initial Contact.
1. The terms initial contact or callup means the first radio call you make to a given facility or the first call to a different controller or FSS specialist within a facility. Use the following format:
(a) Name of the facility being called;
c. Subsequent Contacts and Responses to Callup from a Ground Facility.
Use the same format as used for the initial contact except you should state your message or request with the callup in one transmission. The ground station name and the word “Over” may be omitted if the message requires an obvious reply and there is no possibility for misunderstandings.
Also, as in the answer above, if you mistakenly set the wrong frequency by even one digit on your radio, you may be talking to a different facility, setting up the opportunity for a real problem to occur.
You didn't add any jurisdiction, and I don't know if this applies to the US (though I suspect it does at least to some extent), but in Europe, it's also pretty common for the controller to shorten call signs.
So your initial call on a small European airport might be as Springfield Tower, AB-XYZ, only to have the controller come back to you with A-YZ, Springfield. In that case, you're expected to keep using the abbreviated call signs when communicating with that station or facility, so your response will be addressed as Springfield, A-YZ for the duration of your contact with Springfield Tower. When you leave their frequency or otherwise stop communicating with them, then call them up again later, you start over with Springfield Tower, AB-XYZ to avoid ambiguities.
The same principle applies also with controllers talking to many more aircraft; at least in the case of Sweden, all the way up to Sweden Control.
The controller (hopefully) knows which stations (such as aircraft) they are communicating with at the time, and are in a much better position than you are to determine how much a given call sign can be shortened without risk of conflict. Rather than making guesses about this yourself, let them take the lead. You're much less likely to make a mistake that way.
Besides, it has to be a very long airport name if it takes more than a second or two to rattle off, so it's not like you're saving all that much for the risks you get by omitting it.
Given that the overall aim of radio communication in aviation is to increase flight safety, a good rule of thumb is to simply ask yourself if doing something, or doing it in some particular way, is conducive to improving flight safety (either in the air or on the ground) compared to not doing it at all, or doing it in some other way. If it doesn't improve flight safety, then as a rule of thumb you should probably refrain.