IATA and ICAO has different methods of specifying flight designators. For IATA:
A flight designator is the concatenation of the airline designator, xx(a), and the numeric flight number, n(n)(n)(n), plus an optional one-letter "operational suffix" (a). Therefore, the full format of a flight designator is xx(a)n(n)(n)(n)(a).
ICAO on the other hand, calls for:
the ICAO designator for the aircraft operating agency followed by the flight identification... when in radio telephony the call sign to be used by the aircraft will consist of the ICAO telephony designator for the operating agency followed by the flight identification
There is no rule that I'm aware of that says that their identifiers should be the same. Flight plans use ICAO codes for flight identification, while IATA codes are mostly used for ticketing and other commercial purposes. It is entirely possible that the same aircraft has different codes, usually to avoid callsign confusion, as the ICAO callsign is sued for ATC purposes.
That being said, this seems to be mostly used in Eurpoe, rather than US (all the flights you've given as examples are from Europe). From this forum:
... In North America the airlines only add a letter designator to a flight number if a flight with the same number is in the air at the same time; which is a rare occournance.
On the other hand; if you heard this in Europe then its a different set of rules. I hear from a friend in London the regulatory authorities (such as National Air Traffic Services in the UK) allow the airlines to only use certian numbers at any one time and they often add a bunch of letters to the end just so they can't get the flights mixed up.
Using different IATA (passenger/luggage) and ICAO (flightplans/atc) flight numbers rarely happens in north america, but is a common occurrence with European airlines....
From this book:
In order to reduce the possibility of two callsigns on one frequency at any time sounding too similar, a number of airlines, particularly in Europe, have started using alphanumeric callsigns that are not based on flight numbers.
Also see the answers here, here and here