For every aircraft, a weight and balance form is provided in that aircraft's flight manual and pilot operations handbook (POH). It provides a method for determining the aircraft's weight and position of center-of-gravity. It is often represented as a some sort of graph where the person making the calculations can rather quickly establish the results.
The load-sheet can be prepared by an operations office, ground handling agent or the pilot, but it should always be present in the cockpit and verified by the pilot before commencement of flight.
To determine the overall mass of passengers and cargo, it is often unfeasible to weigh everybody and every piece of cargo, so often standard masses are used. They are average "close enough" values to be used in calculations. If it is obvious to the crew that standard masses are not applicable for a given flight, a different method should be used - like weighing.
The above method may sound rather inaccurate, which it is. Payload makes only about 10-20% of total aircraft weight, so even gross errors in determining mass of passengers and bags usually does not compromise safety.
Usually the reason behind these kind of accidents is poor safety culture within the airline where load-sheets are not prepared at all or they do not even try to represent actual conditions. It is easy to slip in this kind of behavior if the passenger load is always the same like it could be in a chartered business jet with only a few seats. Then comes the day when the plane is full and bags are heavy. In smaller aircraft the cargo hold is usually in the tail, sometimes in the nose. If it is stacked with heavy bags you are most likely outside approved operational envelope.
Also there are several known cases where VIP passengers have put pilots in tough situation and demand that their gear is loaded even if it is obvious that aircraft is overweight. This can be reinforced by the bosses who want to keep their high-paying, high-profile customers happy.