I would like to discuss the stability argument in a bit more detail. Since it is correct that static longitudinal stability is the main reason why these aircraft are not often developed.
However the reasoning given in the other posts is incomplete/not completely correct.
First of all, a flying wing indeed has a very small stability margin. This can be solved by either some unconventional wing designs: this has the problem of defeating by large the efficiency gain of using a flying wing configuration.
The other method, employed by the B2 spirit is to use an active controller to control control surfaces. This has the drawback of increasing complexity of the aircraft and passing regulations tests is even harder. some reference.
Static longitudinal stability
I'm going to explain the static longitudinal stability in detail a bit more. First we define stability: to be stable means that whenever a small excitation is applied to the object, the object will "recover" itself.
Longitudinal stability means that an excitation in the longitudinal direction, thus a change in pitch/angle of attack ($\alpha$), needs to be countered by "some" moment. Since an aircraft during cruise in equilibrium, an increase in angle of attack, should lead to a negative moment. - A reduction of angle of attack should lead to a positive response moment.
Or in a mathematical way: (definition)
$$\frac{\partial M}{\partial\alpha} < 0$$
A simple wing
Now let us first look at a simple configuration: just a wing. Since lift generated from a wing is due to a distributed force, a wing will always have both a Lifting force, and a lifting moment (except at a single point where the moment is zero, however this point changes with flying conditions). - In aviation we remove the units for simplicity's sake. So we have a force $C_L$ and a moment $C_M$.
On an airfoil there is also a point where the factor between $C_L$ and $C_M$ doesn't change with angle of attack. This point is called the aerodynamic center and is a static point given by the airfoil shape: it is hence used to calculate.
So (by definition):
$$\left( \frac{dC_m}{dC_l} = 0 \right)_{a.c.} $$
Now since a wing always generates more lift under a higher angle of attack, and actually we consider the C_L - \alpha
curve to be linear. (For stability we consider small changes in angle of attack) the following holds:
$$ \frac{d C_L}{d \alpha} = C_{L_\alpha} > 0 $$
Together with the earlier equation:
$$ \frac{d C_M}{d \alpha} = C_{M_\alpha} > 0 $$
conventional aircraft
I first wish to address the stability of conventional aircraft in this point, as there seems to be a lot of contradicting information.
For this consider the following configuration (notice that the points where the lift "attaches" to the wing & tail are defined to be the aerodynamic center for these calculations - we could use any point, but using ac reduces complexity a lot).

From the static equilibrium equations:
$$W = L_W + L_t$$
$$L_W = \frac{1}{2}\rho V^2 S_w \frac{dC_L}{d\alpha}(\alpha - \alpha_0)$$
(above is just the lift equation, which defines $C_L$)
The lift due to trim in the tailplane is more complex (due to the non negligible down wash of the main wing on the airflow at the tail (${\epsilon}$). ($C_l$ = lifting coefficient of tail section)). - Simplifying, we consider the horizontal tailplane to be a symmetric airfoil, so lift at $\eta=0$ is zero. (of the tailplane).
$$L_t = \frac{1}{2}\rho V^2 S_t \left( \frac{d C_l}{d \alpha} \left( \alpha - \frac{d \epsilon}{d \alpha} \right) + \frac{d C_l}{d\eta}\eta \right)$$
Similarly the moment equation can be written:
$$M = L_Wx_g - (l_t - x_g) L_t$$
Now from the very first equation again, the partial differential of the moment equation with respect to the angle of attack needs to be negative:
$$\frac{\partial M}{\partial \alpha} = x_g \frac{\partial L_w}{\partial \alpha} - (l_t - x_g) \frac{\partial L_t} {\partial \alpha}$$
Now there is a final definition that needs to be made, a distance $h$ from the center of gravity so that for the total wing the moment equation can be written as:
$$M = h(L_w + L_t)$$
Solving all equations (see wikipedia for details) leads to:
$$h = \frac{x_g}{c} - \left( 1 - \frac{\partial\epsilon}{d \alpha} \right) \frac{C_{l_\alpha}}{C_{L_alpha}} \frac{l_t S_t}{c S_w}$$
With $c$ being the main aerodynamic chord of the main wing. (Introduced once again to reduce the amount of units we work with). For stability (since $C_{M_\alpha}$ needs to be negative) $h$ needs to be negative. Let's analyze above result:
$$\frac{l_t S_t}{c S_w} = V_t$$
This part, called the "tail volume", consists of geometric definitions of an aircraft and won't change.
$$1 - \frac{\partial\epsilon}{d \alpha} $$ are the stability derivatives and difficult to calculate, but typically found to be at least $0.5$.
So this allows us to define the stability margin as:
$$h = x_g - 0.5cV_t$$
Note that since the second term is always positive, having a negative $x_g$, or (see image above) having the center of gravity in front of the aerodynamic center of the main wing. will always give a stable configuration. And remember that aerodynamic center does not change with angle of attack. (Center of gravity can shift during cruise due to fuel consumption, but this is typically mitigated in practice by pumps, and shifting center of gravity forward will always give a more stable aircraft).
neutral point
Now finally we are at the neutral point, which was used in another answer incorrectly consistently. The neutral point is, by definition, the point at which an aircraft is "just" stable: $h=0$
$$x_g = 0.5cV_t$$
From this it follows that the "range" between which the center of gravity can change is between nose of the aircraft (negative $x_g$) and a point given by mainly the tail volume. The tail volume is most easily influenced by changing either the tail surface or distance between main wing and tail.
Flying wing configuration
Finally back to the original point, the flying wing configuration. A flying wing, by definition, has no tail behind the main wing. Thus the tail volume is zero.
Hence the neutral point of a flying wing is exactly at the aerodynamic center. Which is for a conventional wing design about 1/4th of the chord distance.
thus a flying wing has, without modifications, an unusable small stability margin
Delta wing and canard
I'd also wish to quickly sidestep to the delta wing and canard configuration such as for the concorde or f16. These designs are driven by another parameter (shockwave drag/something else, like more efficient control due to no downwash).
However the stability for such aircraft is a lot different: while the picture above can still be used, we need to consider that $l_t$ is, by design, negative. This changes the location of the neutral point to always be in front of the main wing. And many of those designs also have active control surfaces and are inherently unstable.
(The name "canard" even came from this: when the brother wright created the first powered aircraft, in France people didn't believe it. They called it what we would call today "fake news". The term for fake news was "canard" in France, so they called the design a "canard").