You should definitely not attempt flight if there is frost on the windows that would affect visibility, or on any of the wings or flight control surfaces. Frost accumulation on such surfaces can reduce the lift generated by your wings, possibly causing a crash on takeoff.
According to the NTSB, frost the size of a grain of salt, distributed as sparsely as one per square centimeter over a wing's upper surface, can destroy enough lift to prevent a plane from taking off.
The cheapest solution is to try to re-position the plane on the ramp so that the sunlight has a chance to melt it away.
You can also temporarily move the plane into a heated hangar until it melts, or ask the FBO to do so for you. If the forecast is predicting a chance of such conditions the night before, you can usually call the FBO on the night before and ask them to move your plane into their hangar just for the night, in advance of your morning flight.
If you're really anxious to go, you can ask the FBO to apply a heated deicing solution to your plane, but be prepared to pay for it (it's priced by the gallon and the amount needed will depend on the size and amount of frost of your plane). Not all FBOs will have deicing capabilities, particularly if the region does not commonly experience such conditions.