The Canadian Flight Training Manual stated this when talking about maximum endurance:
For reciprocating engines, maximum endurance is achieved at sea level.
The book does not explain further as to why. Air density is my first guess, when the engine gets a better mixture of air to burn with its fuel, and so we get the most power for the same amount of fuel.
But if that is true then it sort of contradicts why maximum range is at higher altitude.