I know that when an aircraft starts rolling, it's load factor increases. But can anybody explain how does load factor vary when the aircraft pitches up/down? Does it increase/decrease?
Think not about the rate of pitching but rather about the skyward or earthward curvature in the flight path (trajectory). This curvature is an acceleration, and requires a force to make it happen.
G-load (i.e. load factor) is essentially just the aerodynamic force generated by the aircraft, divided by weight.
In straight-and-level flight the G-load is +1.
In the case where the aircraft is wings-level and in a level upright pitch attitude (not inverted), we can observe--
If the flight path is curving skyward, the G-load is greater than +1
If the flight path is curving earthward, the G-load is less than +1. It may even be zero or negative.
This answer could be expanded to consider other flight attitudes. Think about a loop. At any instant in time, if the flight path is curving, some force is causing that. The main forces that could be contributing to this curvature are gravity, and the wings' lift force. Of these two forces, only the latter one determines the G-loading, i.e. the load factor. In the plane's own reference frame, an "upward" aerodynamic force (towards the canopy) creates a positive G-loading, and a "downward" aerodynamic force (toward the belly) creates a negative G-loading.
Knowing this, you can compute the G-load at any instant, as long as you know the aircraft's attitude in space, velocity, mass, and the radius of curvature of the flight path.
Starting from level upright flight, if you pull back on the stick or yoke, you'll feel heavier. If you push forward on the stick or yoke, you'll feel lighter.
Note: this answer assumes we are measuring the G-force near the center of pitch rotation of the aircraft. A pilot seated far forward of the center of pitch rotation will feel a greater increase in G-load when pitching up, and a greater shift toward a negative G-load when pitching down, than if he were seated at the center of pitch rotation. The same is true of the load factor imposed on any part of the aircraft structure.