I've observed this on many flight trackers; the time taken to fly from A to B is not necessarily the same as the time taken to fly back from B to A. Why is this so and what external factors besides prevailing winds (headwind going one way, tailwind the other) create this difference in time taken?

Example: late May 2015, per American Airlines, PHL to MCO is about 15 minutes longer going south than going north, but times MIA to BDL are essentially identical in either direction. Prevailing winds would affect both routes similarly, but the difference in times northbound vs southbound isn't the same. Why not?