Is there certain gear that people consider "must-haves" for a private pilot? Specifically, a new pilot or student pilot.
Get a good headset and a decent mobile device that can run a charting app. I use my Moto4 with Avare.
Download The Airplane Flying Handbook -and- The Pilot's Handbook of Aeronautical Knowledge from the FAA site (both free). Also download the Airman Certification Standards (ACS) for private pilot airplane.
The only books that I usually recommend people actually purchase are the aircraft manual and a written test study guide from ASA, Jeppesen, Gleim, or King.