The correct answer seems to be, "Nobody knows for sure" but here's some data.
The earliest citation in the Oxford English Dictionary for the phrase "black box" referring specifically to a flight data recorder isn't until 1964, from the UK Daily Telegraph: "The flight recorder is an indestructible 'black box' which automatically records the key functions in the aircraft..."
There are earlier uses of the phrase, with different meanings. Since 1932, it has been used to mean a device whose internal workings are unclear but which is specified by its inputs and outputs; since 1945, it has been used in the Royal Air Force to refer to various navigational devices which, according to Wikipedia sometimes were housed in literal boxes that were black.
My interpretation/thoughts/speculation/whatever word you want to use:
The use of quote marks in the Daily Telegraph seems significant. First, it suggests that we're not talking about a literal box which is black and you might imagine they'd say something like " 'black box' (which is now actually orange)" if they were formally literal black boxes. Second, "a 'black box' " suggests that the FDR might be just one of several devices on a plane that could be described as "black boxes". Unfortunately, that seems consistent with both of the other given definitions: you could perfectly well imagine it being RAF-style slang for "This electronic box of tricks does navigation, this one holds the autopilot and this one records flight data" or the input-output version of "I don't know exactly how the autopilot works, but I know what it does; ditto the navigation system; ditto the data recorder."