Location
The general design is to group elements that need to be driven by the engine around one "accessory gearbox" which is located away from the engine centerline. A driving axial shaft in therefore required. In some cases, there are multiple accessory gearboxes.
Usually the accessories are located at the bottom of the engine fan case, making them and the gearbox away from airflow and more accessible for maintenance.

CFM56-3, source
Here a view of an accessory gearbox with its accessories in place:

Source
When this is not possible to put the accessories here, because the clearance between the engine nacelle and the ground is too limited, they are moved sideway as in the B737 with a CFM56-7. This gives the nacelle of the B737 its characteristics squared profile:

The arrow points at the accessory gearbox (source: B737 AMM)
Indeed putting obstacles to the airflow is not good for the engine efficiency, so the radial driving shaft location is chosen to minimize aerodynamic effects, usually within, or behind, one of the struts of the fan frame.
A view of the driving shaft whole gearbox without the accessories:

CFM56 Gearbox (without accessories) and radial drive shaft, source
Accessories
Accessories are driven by a radial shaft, or a set of shafts with angles and gearboxes as required. Energy is taken from the HP spool, or the IP (intermediate pressure in engines with 3 spools, like in many Rolls Royce engines).

Source: The Jet Engine, Rolls Royce
The generator can be seen on the right of the accessory box.
The starter is another accessory. When starting the engine, the radial driving shaft is used in the other direction, the accessory box drives the engine shaft. Sometimes the starter and the generator are a single component, an electric engine being reversible. See How is a turbine engine rotated for starting and inspections?
By design, all accessories turn with the engine.