Try to see it this way: ALPHA FLOOR is a protection function of the AUTO THRUST SYSTEM, and it protects the aircraft from excessive speed decay. It does not primarily protect the aircraft from stalling. That can of course a possible outcome, if you don't increase thrust to accelerate.
Stall protection is primarily done by the low speed protection (high AOA protection of normal law), where the aircraft commands nose down so it does not decelerate below V_ALPHA_PROT (thus maintaining actual AOA <= ALPHA_PROT). And that protection is working above M.53.
One example for ALPHA FLOOR: you are in approach with Flaps Full, Idle Thrust (Manual Thrust) and nearing V_APP. Environmental influences (loss of headwind, increase of tailwind, gusts) or a shallow descent or even level-off could even further let your speed drop. That is where ALPHA FLOOR kicks in, because the PRIMs calculate, that you are in a low-energy state. To prevent the aircraft from automatically pitching down, the auto thrust commands TOGA. If everything goes well (as the engineers intended the protection to work), the speed decay is prevented just in time, so that the low speed protection does not need to be activated. In this case, alpha floor makes sense.
Other example: you are in cruise flight, near MAX REC ALT and near green dot speed. It does not take a lot (sudden change in wind, turbulence, excessive pitch input in manual flight (which you should not do at cruising altitude)) to let the speed drop to V_ALPHA_PROT (which is very close to green dot in this thin air near max altitude). If you are flying in alternate law (which we do in the simulator for stall training), the stall warning will activate instantly. Maybe the Auto Flight System disengages, because the speed drops below V_LS minus 5. In both cases, it would do more harm than good to have TOGA thrust, which can cause a pitch up moment, that is hard to control manually at these high altitudes. The only thing to do to save the airplane is to pitch down (nose down to unload, g-load wise), recover speed, increase thrust smoothly and recover the flight path.
These are two edge cases. And somewhere in between, there is a point where alpha floor does not make sense anymore as the best way of protecting the aircraft in all possible scenarios. And apparently, the engineers determined M.53 to be this limit.