Is there certain gear that people consider "must-haves" for a private pilot? Specifically, a new pilot or student pilot.
Get a good headset and a decent mobile device that can run a charting app. I use my Moto4 with Avare.
Download The Airplane Flying Handbook -and- The Pilot's Handbook of Aeronautical Knowledge from the FAA site (both free). Also download the Airman Certification Standards (ACS) for private pilot airplane.
The only books that I usually recommend people actually purchase are the aircraft manual and a written test study guide from ASA, Jeppesen, Gleim, or King.
Flight bag. Sectional for the area to be flown in. Headsets usually, to protect hearing, if not to hear ATC better. POH if not in the plane already. Kneepad for taking notes, something to write with. Water. That's a bare minimum list, for a casual flight in a known area.