From what can be gleaned from the Net, it was instability. The AvorcarAvrocar had plenty of power but was unstable. The resulting behavior made it uncontrollable when leaving ground effect. In this publication I found an illustration which tries to explain what went on:
Any asymmetry in flow would shift the center of pressure, resulting in a wobbling motion called hubcapping (because it resembled the motion of a hubcap when dropped to the ground).
From the linked document:
On September 29, the first attempt of sustained flight was made with the Avrocar pinned to the ground by cables. After that the vehicle took off, an uncontrollable oscillation occurred with each wheel alternately bouncing on the ground. […] These first captive flight[s] revealed then a new problem, called “hubcapping” which has never been fully resolved. The hubcapping was rapid and unpredictable swings in pitch and roll axes.