In Mendenhall's book about the Gee Bee racers, he describes that on the Models Z, R-1, and R-2, there was a strong tendency for the engine cowl to get pulled forward and into the prop during races, requiring reinforcement of the cowl attachment points to the engine & airframe- which puzzles me to no end. Can someone please explain the pressure distribution right behind the prop that would cause this?
You are looking for 'cowl lip suction' or 'lip thrust'. This is a phenemona that also happens on jet engine cowls when they have lots of spillage.
When the cowl flaps are (nearly) closed, the engine bay approaches stagnation pressure. Many of the streamlines that would otherwise go into the front of the cowl instead are diverted around the outside -- this is a situation similar to engine spillage (when mass flow can't go through the engine, it goes around).
This air is forced to rapidly accelerate around the cowl lip -- this region of high acceleration leads to high velocities which lead to a local suction.
Here is a panel code solution demonstrating this.
The magenta represents the inside of a blocked engine -- the pressure there reaches the stagnation pressure for the flow.
The green and blue around the cowl lip represent high velocities and strong suction pressure -- some of that area has a forward-pointing outward normal vector. This means that the local suction will resolve into a thrust.
If the cowl were truncated somewhere on the constant-diameter section of this nacelle (and you can imagine the entire engine compartment is at stagnation pressure (magenta) then you can see how the cowl would have a net forward force on it.