I see two good answers, but would like to add a bit to that from another perspective. I don't know if it's the reason, but it definitely is taken into account.
Stress
Stress is the force per area. It's used for analysis of structures. The most important thing is that too much stress can lead to (fatigue) failure.
Aircraft cabins are pressurised. The result of this is that the entire outer shell of the cabin is under stress. You can split this into an axial and a hoop stress. The equation for these are:


It's clear the axial stress is significantly lower. Keep this in mind.
Stress concentrations
First, a simple sample calculation:
Let's say you have a plate of 1 x 1 m with a thickness of 1 mm. You can place a uniform load of 1 kN/m on one side. This means the plate is supporting a load of 1 kN. The stress is the load (1 kN) divided by the area (1m x 1mm) = 1 MPa.
When you have a plate with a (small) circular hole in it you might think this only has a very small effect, but the effect is quite large. Just at the side of the hole, the stress is raised to 3 MPa*.
Adding the hole increased the stress by a factor 3 under the same load. Effectively the whole reduced the maximum loading capacity by a factor 3.
This effect is due to stress concentrations. A stress concentration factor is a multiplier for the maximum stress in a plate due to holes or other geometric properties. The shape of the hole has a large effect on the stress concentration factor. For an oval shaped hole, the stress concentration factor is:

2A is the width of the hole, 2B is the length of the hole. It should be clear that increasing the height of the hole along the axis with the most stress reduces the maximum stress, making the plate stronger!
Removing area
The other effect of adding a hole is that you remove area over which the load is spread out. If your hole has a diameter of 0.9 m, you can imagine that the maximum stress would increase much more than if it were 0.00001 m.
For more information on these interactions, you can check out
Putting it all together
The stress in the hoop direction is much higher than that in the axial direction. By elongating the holes (windows) along this axis you can reduce the stress concentrations. This decreases the maximum stresses, resulting in a stronger aircraft.
You can also more easily increase this length as you can more easily afford to lose material along this axis.
Additional factors
The fuselage is also subjected to bending forces due to the fuselage tube being supported only at the wings. However, these forces only cause large stresses at the top and bottom of the cabin and much less so at the position of the windows which are located near the centerline.
Furthermore, according to @Therac most of the load is caried by the frame members, which also reduces the importance of the stress concentrations around the windows.
I personally am not certain about that as apparently windows add about 200 kg for an aircraft seating 150 passengers. There are also suggestions for window-less passenger aircraft with wraparound screens.
*slightly higher due to the plate also losing surface area, but I will ignore that effect at this point