Yes, it is very possible. As you say, it may be against the rules, but it is possible. ADS-B Out may be integrated in a "standard" 4096-code Mode A/C transponder (which can be disabled), or it may be wired in to another system on the aircraft, such as the anti-collision strobe (which can also be disabled). A standalone ADS-B device may even be plugged into a 12V accessory outlet, though I believe that's more common for an ADS-B In setup (where various data, most importantly nearby traffic, is fed in to a display device)—I'm not sure if that counts as a valid Out setup.
In any case, as the answers at the linked question describe, every electrical device on an aircraft should have the capability to be powered off. In an emergency, especially a fire onboard, the immediate safety of the aircraft and its occupants is prioritized above the generally enhanced safety situation afforded by ADS-B.
But there are other reasons you may not see an aircraft on a flight-tracking website:
- Pilots can make a request, both with the FAA and with the website, to have their aircraft hidden. This is very common. However, this will not be the reason you do not see an aircraft on ADS-B Exchange; they are very clear that they do not filter or block any aircraft from their data.
- The website may not have any data feeds nearby. Official ATC displays use ADS-B receiver antennas that are installed and maintained by the government or other appropriate organization; third-party sites may have an arrangement to access some of this data, but if not they must rely on a network of volunteers with privately-owned receivers. There may not be any such volunteers within range of the aircraft.
- In particular, in the United States, many ADS-B transponders broadcast on a frequency of 1090 MHz, the same as in the rest of the world. But smaller aircraft that will not operate above 18,000' MSL nor outside of the United States may use a cheaper "Universal Access Transceiver" which broadcasts on 978 MHz. Fewer aircraft use this technology which means fewer volunteer receivers are set up for it; if your aircraft of interest has an ADS-B Out device that uses 978 instead of 1090, it is less likely to appear on third-party sites.
- The aircraft may not have ADS-B Out installed at all, and that may be quite legal. In the United States, 14 CFR 91.225 lists the airspace where ADS-B Out is required, and it is not "all airspace" by any means (though it does include the airspace in and above busier airports, as well as all airspace above 10,000' MSL that is at least 2,500' above the ground).
And yes, if the the aircraft's ADS-B is turned off the only way to track them would be by using their transponder (secondary radar) or listening for an echo bouncing off the physical aircraft (primary radar). And if the pilot turned off the transponder as well, primary radar would be the only automated means of tracking the aircraft. This would still be quite possible, as long as the aircraft was high enough; one of the benefits of ADS-B is that the cheaper receiver sites allow for greatly increased low-altitude coverage, which the expensive primary radar antennas do not always provide (see: Can someone actually "fly under the radar"?).
Note that even in non-radar airspace known aircraft can be "tracked" if the pilot reports over a known location at a known time, and ideally provides an estimate of when they will be over the next location.