In the early 1930's aircraft were breaking records each week because the performance needed was quite low. As technology advanced the performance needed to break a record also became higher. Which causes us to ask if it was easier to break records early in the century, or later with the aid of technology.
Howard Hughes built many airplanes specifically for breaking records, yet even today the Russian and American fighter manufactures often strip down their latest fighter prototype and set new world records.
Addendum: There seems to be some confusion over what the question is asking... "most" means "total" as "aeroalias" has supplied below. The basis for the question was a YouTube video that stated the F4 Phantom had broken the most records of any aircraft, but the video was circa 1980 without specifics and it seemed there was a high probability that other aircraft had superseded it. (I would have guessed some aircraft in the 30's, therefore my keen interest in the answer)
At this point "aeroalias" is right on track...
So, what aircraft broke the most (total) world records?