(Daytime flying only. Peripheral and night vision is different and I have zero practice of night flying.)
The highest resolution part of the eye is the fovea, and it's surprisingly narrow. If a small, slow-moving object is outside this area it probably won't be seen. If you scan from say 20 degrees left of an object to 20 degrees right of it you will also miss, because the eye tends to jump in steps and we only see a small fraction of what is actually in front of us.
So I was always told to gaze steadily, in ten degree steps, for a few seconds at each area. A grid of search areas both above and below the horizon should be checked. You'll only recognise an object when your fovea is virtually on it.
- Only look for a second or two at each are because any longer and the vision "bleaches" and you are even less likely to recognise anything in that region; just keep moving on to the next.
- The narrow 10 degree steps are that size because that's the region
of acceptably sharp vision. (I suspect it differs between individuals)
- This was what I was taught for general lookout, though I suppose it should apply to a specific search area given by ATC too.
Apparently these tips were all derived from RAF training for this sort of thing, but I don't have any notes on that. In any case I suspect it's more practice than bookwork anyway.
One of my gliding instructors liked the spotting game. One point for every aircraft I spot before him. He had one dodgy eye and of course still won by an appalling margin... (I've also heard some play this game for beers, though frankly none of my instructors would ever be able to drive home.)