As Robbie's answer implies, the answer is wind - Trade Winds govern what happens on or near the surface and were historically important for sailing ships, and today in aviation we deal with the [the Winds Aloft](http://aviationweather.gov/adds/winds/) at whatever altitude your plane is flying.

As an example, consider two hypothetical flights at 30,000 feet with the winds shown below:  
[![Winds Aloft at 30,000 ft][1]][2]

With these winds a flight from A to B (East-to-West) will be fighting headwinds the whole way. My handy flight planning program tells me that at 500 knots (airspeed) you'd be in the air for about 5 hours 30 minutes.  

Traveling in the opposite direction from B to A you'd have a tailwind, and with the same 500 knot airspeed you would make the trip in roughly 4 hours.

***

The winds aloft at higher altitudes are fairly consistent in speed and direction (there are seasonal variations, but for the most part as far as jet traffic and passengers are concerned the time differences based on direction of flight are pretty consistent).  

Pilots and flight dispatchers will often review the wind data prior to flight and try to select an altitude that affords a "good ride" (free of turbulence) and favorable winds (either a tailwind or the lowest headwind they can find).


  [1]: https://i.sstatic.net/6p4B1m.gif
  [2]: https://i.sstatic.net/6p4B1.gif