Perhaps yes, but it's not easy to qualify objectively.
Simulating a light airplane may require a different approach in some areas, but it's not inherently difficult compared to 'heavy' ones.
However, the truth is, nobody cares that much, so there is much less investment in such simulators. A decent FTD (flight training device, something a step lower than what is legally called a 'simulator' (FFS)) will cost easily several times more than an ultralight, while the risk benefit is arguably lower (compared to real flight).
Secondly, type-rated simulators are rarity even on the 'proper' GA market; typically FTDs are 'generics' that aim to simulate a representative 'class' of aircraft. This makes fair comparison difficult, especially by real pilots who have most experience on a single/few real types.
Then, there are technical features that are even more important for light (and GA) airplanes than for big ones. One is a good visual system: in real visual flying, you obviously use it more, and having a single monitor stuck in front of you will hamper you more than you might think.
Second is control loading: on light airplanes you get airspeed and trim feedback right at your fingertips. There are no cheap solutions to this. You may argue this is not 'physics', but it actually is, this is all part of a feedback between the airplane and you being the part of control system. Lack or poor quality of these things (which is more than common on the simulators used for light planes) will substantially affect subjective and objective qualification of the flying qualities/handling of the simulator.
There are areas where even expensive FFS suffer. A typical example is ground handling. Or, speaking about 'loss of control', many regimes that result from it, notably stall and spin, are actually difficult to simulate well, and are often not required for certification.
That all said, it's an overstatement to say that simulators are 'useless for preventing accidents'. Flying is more about thinking ahead, planning and situation awareness than any specific piloting skills, and even a game simulator can teach you some of that.