tl;dr The basic differences between IFR and VFR involve weather, required equipment, filing flight plans and getting a clearance, and the altitudes flown.
Your question indicates that you may not know that there is a difference between Visual Meteorological Conditions (VMC) and Visual Flight Rules (VFR). Likewise between Instrument Meteorological Conditions (IMC) and Instrument Flight Rules (IFR). You can be flying under Instrument Flight Rules in VMC—the airlines do this on most flights. It is not wise to fly VFR into IMC—a huge percentage of general aviation accidents happen because of that.
One source of the definitions is the FAA publication Pilot Controller Glossary, although they sort of presume you already know the concept.
INSTRUMENT FLIGHT RULES− Rules governing the procedures for conducting instrument flight. Also a term used by pilots and controllers to indicate type of flight plan.
INSTRUMENT FLIGHT RULES[ICAO]−A set of rules governing the conduct of flight under instrument meteorological conditions.
INSTRUMENT METEOROLOGICAL CONDITIONS− Meteorological conditions expressed in terms of visibility, distance from cloud, and ceiling less than the minima specified for visual meteorological conditions.
VISUAL FLIGHT RULES− Rules that govern the procedures for conducting flight under visual conditions. The term “VFR” is also used in the United States to indicate weather conditions that are equal to or greater than minimum VFR requirements. In addition, it is used by pilots and controllers to indicate type of flight plan.
VISUAL METEOROLOGICAL CONDITIONS− Meteorological conditions expressed in terms of visibility, distance from cloud, and ceiling equal to or better than specified minima.
The FARs only indirectly define IFR but you can get the rest of the definitions from the FARs.
§1.1 General definitions.
IFR conditions means weather conditions below the minimum for flight under visual flight rules.
§91.155 Basic VFR weather minimums. (a) Except as provided in paragraph (b) of this section and §91.157, no person may operate an
aircraft under VFR when the flight visibility is less, or at a
distance from clouds that is less, than that prescribed for the
corresponding altitude and class of airspace in the following table:
The table lists the visibility and clearance from clouds in different kinds of airspace that you need in order to fly under visual flight rules. If the weather is less than the minimums, then you need to fly under Instrument Flight Rules.
You can get an idea of the differences by reading CFR PART 91—GENERAL OPERATING AND FLIGHT RULES Subpart B—Flight Rules