There is actually quite a lot of difference between ARINC-429 and MIL-STD-1553. They are different because they were developed by two different groups with different needs.
Both started in the 1970's. Before then there was almost no digital data of any kind on a an aircraft. It was almost all analog or discrete. The biggest issue was noise and interference of the signals. The introduction of microprocessors in the early 70's opened up the option for reasonably priced digital bus technology.
The MIL-STD-1553 bus was a major component of the F-16. Like most military programs, this was a push for maximum performance. ARINC-429 came out of the Boeing 757/767 development effort. Civilian aircraft certification doesn't like a lot of risk and the simpler bus architecture made it easier to design, analyze, and certify.
Since then there have been a number of new buses introduced into aviation. Some have lasted longer than others. B777 introduced ARINC-629 and Fibrechannel. B787 uses ARINC 664 for the main buses with a large mix of ARINC-429 and ARINC-825 (CAN bus) as well. There are a number of data concentrators/converters on the aircraft to get data where it needs to go.
The primary reason for moving away from ARINC-429 is that it is a single transmitter/multiple receiver architecture. This results in a large amount of physical wire. That adds weight and requires maintenance. A bus such as ARINC-664, which is based on Ethernet, can significantly reduce wire. The down side is that it costs a lot more and isn't very efficient when all you need is a small, low rate connection to an odd corner of the aircraft. So you end up with a mix of buses.
And B787 opted for fiber optic cable for the ARINC-664 network to reduce interference from high-intensity radio frequency (HIRF) and lightning.
MIL-STD-1553 is a powerful bus, but it's not as good a fit as many of the newer buses available in the commercial market. The military keeps using it because they have a huge installed base of equipment, it's flexible enough to support their newer functional needs, and commonality reduces their logistics costs.
ARINC-429 sticks around because it's cheap and it has its own large installed base as well.