During flutter, the airfoil oscillates with a large amplitude, which generates vortices at the wake. In vortex induced vibration (VIV) the oscillations of the airfoil are caused by the shedding of vortices. Both flutter and VIV have airfoil oscillations and vortex shedding. How are they different other than the cause and effect relation? And also how to distinguish which one is the cause and which one is the effect for a case where we don't know beforehand whether it is flutter or VIV?
Flutter and vortex induced vibrations (VIV) are both dynamic instabilities arising from the interaction of unsteady aerodynamic, inertial and elastic forces. They both extract energy from the air stream to persist.
Instead of perceiving the interaction as aerodynamic forces causing structural motion or structural motion generating aerodynamic forces, it is more meaningful (and realistic) to describe it as a coupling, e.g., typically described as coupled differential equations, and observing the differences in how the aerodynamic forces are generated and its components.
For VIV, the wake instability that generates the alternating shed vortices (von Karman vortex street) in turn generates an oscillatory lift coupled with the structural oscillation. So, the wake instability and unsteady effects of flow separation dominate the oscillatory aerodynamic forces.
Flutter does not have to describe a dynamic instability associated with a fluctuating wake, e.g., panel flutter is the dynamic interaction and instability arising out of the out-of-plane motion of an aircraft panel in relation to surface fluctuating pressures. However, stall flutter does arise from flow separation and its resulting unsteady wake.
Wing flutter is the dynamical instability arising from the unsteady aerodynamic forces (including the shape of unsteady wake) coupled with the structural dynamics, however wing flutter does not require massive oscillatory flow separation as with VIV. For attached flows (unstalled), wing flutter could exhibit flutter (at a high enough freestream velocity, i.e., flutter speed) due to the resultant phase difference between the generated aerodynamic forces and structural oscillation creating a positive feedback. The time-varying shape of the wake still plays a role for wing flutter, e.g., the role of the Theodorsen function for unsteady aerodynamic lift generated by structural motion.