CASA (the Civil Aviation Safety Authority) have a number of training publications but they are largely oriented at existing pilots, be they recreational or commercial. For someone interested in becoming a pilot, probably the single most useful CASA publication (other than the regulations themselves) is the Visual Flight Rules Guide which gives an overview of key VFR matters and practices. A lot of the information is also provided at http://vfrg.casa.gov.au/ which is worth a look for the wannabe pilot.
AirServices also publishes charts and supplemental material and it is almost certainly worthwhile getting a VTC chart to become familiar with the area in which you will do your flight training.
However, whether you plan to get a Recreational Pilot Licence or a Private Pilot Licence, the goto book is the RPL Study Guide by Bob Tait. As shown, it comes with a supplement for recent changes. Later books cover the more extensive knowledge required by the PPL and CPL but the RPL is the first step.
The book unfortunately is not cheap but it covers a lot of material in the nearly 400 pages within and there is certainly a lot for the student pilot to get into. It incorporates Q&A at the end of each chapter so you can self-study your way through quite a lot of it.
I did my original training for my FAA PPL with what was the bible at the time - the Jeppesen-Sanderson Private Pilot Manual
Comparing that to the Bob Tait book is interesting and IMHO the Bob Tait content is probably more readable as it doesn't present as simply a technical treatise and it is laced with some very good analogies. For example, apparent wind is presented with pictures of a motorcycle rider which makes it instantly obvious to anyone who has ridden a motorcycle. And the descriptions of moment, force and arm are easier to digest than I have seen elsewhere.
It also uses metric measurements/hectopascals and references what measurement systems the US uses, whereas an FAA-oriented book will never use a metric reference and pressure will be in inches of mercury. And of course a US book will reference US geographic locations & meteorology whereas an Australia book will reference Australian locations and met.
So buy the book for the county where you are training. That said, the used Jeppesen books are less than $10 on Amazon so you might want to pick one up as a supplemental reference.
Update
With CASA blurring the lines between the RPL and the PPL, the RPL Study Guide has now been renamed to RPL/PPL Study Guide - Volume 1 (The PPL Study Guide is now the RPL/PPL Study Guide - Volume 2).