If a civil aircraft's transponder fails in RVSM airspace, can it still be allowed to fly in RVSM according to ICAO?
And could it be allowed to fly in class A airspace, according to ICAO?
Note: I assume bare ICAO requirements in this answer. Most states that have RVSM implementations, as well as operator approvals for RVSM, will have additional stricter rules that will change the answer for specific cases.
There are no requirements in the ICAO Annexes and SARPS that require and aircraft the leave the RVSM airspace if the transponder fails.
Note that RVSM airspace is not necessarily airspace where ATS Surveillance services are offered, so for ATC purposes, a functioning transponder is not strictly necessary in RVSM airspace.
The only other use of the transponder that could prevent flying in RVSM airspace (under bare ICAO regulations) is the Airborne Collision Avoidance System (ACAS/TCAS). There are no requirements to have a functioning ACAS/TCAS in ICAO RVSM requirements, so no transponder is needed in support of such requirement.
Even if a transponder would be necessary to enter RVSM airspace (which is the case in for example EASA regulations), a transponder failure while being in RVSM airspace does (under ICAO rules) does not necessarily mean that the flight has to leave that airspace ASAP.
The following is from ICAO Doc 4444 (PANS ATM).
8.8.3.3 AIRCRAFT TRANSPONDER FAILURE IN AREAS WHERE THE CARRIAGE OF A FUNCTIONING TRANSPONDER IS MANDATORY
8.8.3.3.1 When an aircraft experiencing transponder failure after departure is operating or expected to operate in an area where the carriage of a functioning transponder with specified capabilities is mandatory, the ATC units concerned should endeavour to provide for continuation of the flight to the aerodrome of first intended landing in accordance with the flight plan. However, in certain traffic situations, either in terminal areas or en-route, continuation of the flight may not be possible, particularly when failure is detected shortly after take-off. The aircraft may then be required to return to the departure aerodrome or to land at the nearest suitable aerodrome acceptable to the operator concerned and to ATC.
...
Flight in Class A airspace without transponder is possible under ICAO rules, as long as the airspace is not surveillance airspace. Procedural air traffic control does not require surveillance, and a transponder is therefor not required.
Normally, a transponder failure will mean that you will be required to exit RVSM airspace according to ATC instructions, and either climb or descend out of RVSM without levelling out at any altitude inside RVSM airspace. It is possible to fly in RVSM without a transponder but it requires an altitude reservation not less than 4 or more than 12 hours or more before the flight, and you have to have a reason to ask for it, such as a ferry flight, air ambulance flight, etc.
If it fails in Class A airspace ATC will generally make an effort to let you fly to your destination, but the traffic situation and other factors may prohibit that, in which case you may have to return to your departure airport, land at the closest airport where a repair can be made, or be directed out of Class A as directly as possible for you to then continue your navigation to your destination.