Pretty much the only thing that is going to change season to season is the weather. Depending on where you are, there may be more aircraft traffic around a particular time of year, but I can't answer that for sure.
Obviously, the further from the equator, the more extreme the seasons. If you are close to the equator, it probably doesn't matter what time of year you start, since the weather is pretty much the same year-round.
If you are not at the equator, you need to consider the pros and cons of each weather season. Additionally, at extremes from the equator (near the poles), summer and winter months have extremely long days and long nights, respectively, which can make night and day flying difficult.
- In the winter, the aircraft has better performance (airplanes like colder weather). However:
- You have to deal with ice and snow, which is especially dangerous to light airplanes, which are often not as prepared to deal with them. It is dangerous in some degree to all airplanes
- You have to be careful of low temperatures when starting the airplane before you take off
- In the summer, the aircraft has worse performance (they don't like hot weather).
- Many small airplanes have poor airflow inside the cockpit, which means they can get rather hot (it is easier to run a heater inside a small cockpit than cool it)
- You have to be more careful to avoid overheating the engine
- Some places may form thunderstorms constantly
- In the rainy season, which is often in the spring, you will have degraded performance. Even if it is not rainy, the air will likely be more humid, which decreases aircraft performance
Depending on your latitude, other weather conditions may be more prevelant at a certain time of year (hurricanes/cyclones, high winds, tornados, sudden storms). This paper discusses wind differences based on season and latitude. In particular, page 7 of the document shows that summer months had slightly stronger winds at southern latitudes (southern hemisphere), and winter months had significantly stronger winds at northern latitudes. Some geographical areas may have changes in prevailing wind directions, which could also affect training depending on routes and runways too.
TL;DR: It's mainly a weather concern, which can affect your comfort (temperature inside, possibly air turbulence), aircraft performance, as well as grounding weather. Additionally, the most obvious concern is also length of the day - summer months will have more daylight, a particular concern if you can only fly in VFR conditions/do not hold a night rating.