Revised answer
The ARINC 429 frame is composed of 4 bytes (32 bits numbered by convention 1 to 32 from right to left). Bits 11 to 29, which are the payload of the frame, can represent data using different formats, e.g. binary or BCD data. The label is used to indicate the content of the data field, and which format is used, and is necessary to convert data bits into useful information.

ARINC 429 frame used in BCD format (source)
The label is usually spelled as an octal quantity (rather than decimal or hexadecimal), e.g. 01010110
2 will be spelled as 1268 (01 010 110
) rather than 5616 (0101 0110
) or 8610.
The way bits are numbered somehow makes the label the "least significant byte" and bit 1 the "least significant bit". However:
The format above is for naming and ordering information, but doesn't tell how bits are really stored in components memory.
In digital engineering, bits are usually numbered from 0 to 31, as the number actually indicates the weight of the bit: 0 is 20 (1), 4 is 24 (16). In this convention bit 0 is indeed less significant than bit 31.
The ARINC 429 data bus is composed of 2 wires and the ground reference. When an ARINC frame is transmitted on the bus, it must be serialized, which means sending bits one at a time. The transmission order is:
- The label byte transmitted first, "MSB" first: 8-7-6-5-4-3-2-1.
- The other bits are then transmitted "LSB" first: 9-10-11 ... 30-31-32.
So, back to your question, why not using the left to right order for the whole frame? Or at least the left to right order for the label if it's sent first?
From Wikipedia:
Like CAN Protocol Identifier Fields, ARINC 429 label fields are
transmitted most significant bit first. [...] This notional reversal
also reflects historical implementation details.
So let's assume there is no reason to reverse the label bits, other than historical implementations that wanted to mimic the behavior of CAN. In the French version of this Wikipedia article, it is stated that this reversal helped to read the label when looking at a frame using an oscilloscope. The MSB arriving first also appears on the left of the display (the signal on an oscilloscope generally enters on the right of the screen and scroll towards the left side as a function of the time):

ARINC 429 bit stream on a datascope (source)
I'm a bit baffled by this explanation, as engineers working with binary representations are used to read them is both directions quite easily after a short time.
Still according to Wikipedia, hardware used to serialize and de-serialize the ARINC word were actually insensitive to the order of the label bits:
ARINC 429 transceivers have been implemented with 32-bit shift
registers. Parallel access to that shift register is often
octet-oriented. As such, the bit order of the octet access is the bit
order of the accessing device, which is usually LSB 0; and serial
transmission is arranged such that the least significant bit of each
octet is transmitted first. So, in common practice, the accessing
device wrote or read a "reversed label" (for example, to transmit a
Label 2138 [or 8B16] the bit-reversed value D116 is written to the
Label octet). Newer or "enhanced" transceivers may be configured to
reverse the Label field bit order "in hardware."
About odd parity
It is usual in digital serial communication to use data value changes (0 to 1 or 1 to 0) to derive a clocking clue in order to ensure the receiver samples the line voltage at the right time. This is less expensive than transmitting the clock signal on a separate wire. As each bit lasts the same time, the receiver uses a local clock that is roughly in sync with the transmitting clock. To actually maintain the synchronization in the long term, the receiver uses the raising and falling edges of the signal to detect bit borders and phase-align its clock.

Resynchronization of receiver clock on front and rear edges of the signal
That works well if the delay between edges is short. This means avoiding sending long series of 0 or 1. The parity bit can help. For data containing only 0 bits (or 1 bits) the odd parity bit will be 1 (or 0), preventing long edgeless streams to occur.
In some uses of ARINC 429, the parity bit is not transmitted, and is used for data, but nevertheless it seems superfluous for sync in this case since the link protocol is 3-level with return to zero, ensuring two edges for each bit of data.

Return to zero coding (source)