I know that for turbofan engines there will be Gyroscopic effects while pitching, rolling and yawing maneuvre. But if the aircraft is cruising at an altitude or is simply idling on the ground, what would be the torque loads acting on the aircraft because of the rotation of fan blades? and what would be the direction of the moment if there's any?
There is no net torque on turbofan engines
According to Newtonian physics, in order for the engine to transfer a torque onto the aircraft, it must impart some angular momentum (in the opposite direction) onto the air particles. More simply, the engine must make the exhaust gases swirl in one direction for it to transfer a torque onto the aircraft in the other direction.
However, the exhaust guide vanes exist precisely to prevent this - they ensure that the exhaust gases exit with as little angular momentum as possible. This is desirable since the EGV's are actually converting that angular momentum into linear momentum, producing some additional thrust in the process.
Image source. In this image of a GEnx, you can see the stator assembly for both the core exhaust and fan exhaust
Propeller driven aircraft do not have a stator assembly as such, and so air ends up exiting in a swirling motion, and hence the torque.