There seems to be some kind of stubborn myth that unstable systems are impossible to control by humans, and you need some high tech computer magic to do it. This is not true.
I invite you to take a broomstick, tennis racket or baseball bat, and balance it on one end. If you succeeded: congratulations! You just stabilized an unstable system! Now continue doing this while reading an after-takeoff checklist. You will likely drop the broomstick in the process. Alternatively, try and balance a pencil instead of a broomstick. You will find this much more difficult or indeed impossible.
There are two things to take away from this. Unstable systems are simply any system that 'falls over' when you do nothing to correct for it, and secondly, unstable systems can be stabilized using a feedback controller. This controller (whether it is you or a computer) looks at the error (difference between the actual position and the desired position), and calculates a suitable actuator output.
The above example however demonstrates that humans do not make great feedback controllers. They can't multitask well, which is detrimental for unstable systems: when you let go of the controls for just a second, any disturbance will exacerbate itself; and secondly, for systems with a very short time constant (like a small pencil), the reaction time (in control terms: phase lag) of humans is too large to stabilize the system.
This is where the computer comes in. Based on a model of the aircraft dynamics, a controller is designed which stabilizes the system (and while this is perhaps not very easy, it is perfectly possible to design a controller 'offline', i.e., without the actual aircraft, only with a mathematical model). Depending on the model accuracy and the control design, this stabilizing controller may have some limitations. In control engineering, you can generally choose between either a quick reaction time to a (pilot) input, or a very stable system, but there is always a bit of a trade-off in 'agility' versus 'twitchiness'. See @alexh's answer for how they solve this: they put pilots in a simulator (sometimes, this is an actual aircraft modified to behave like the target aircraft), and determine the best controller. A key point here is that the pilot is also part of the whole control loop; minor deficiencies in the computer controller can be corrected by the test pilot.
Secondly, there is a concept called 'region of attraction': for unstable systems, your controller may only work in normal conditions. If your broomstick were upside-down (hanging from your hand), it's a non-trivial task to upright it again. Similarly, if your aircraft is in a flat spin, you may need to revert to another controller to regain control. Note that pilots also revert to this kind of feed-forward control in a spin: kick down the rudder, and don't let go until you're out of the spin again. For unstable aircraft, there may not always be a way to regain control (for example, if the aircraft is very stable flying in reverse), which is why bailing out is an important ability for test pilots in unstable aircraft.
Finally, there is the point of mechanical linkage. In a simple aircraft, the controls are directly linked to the control surfaces. In an unstable aircraft, this is undesirable as the pilot may cause pilot induced oscillations (which they are prone to do even in stable aircraft!). Instead, the pilot's input are fed into the flight computer, which makes the aircraft feel stable as far as the pilot is concerned. This means that these aircraft cannot be flown without a computer, not necessarily because it is difficult, but simply because the pilot does not directly control the aircraft. Test flights are still possible if the computer is programmed with a marginally stable controller (whether they are successful, history will tell...)
