Why do airlines use SIDs and STARs when flying IFR. Wouldn’t it be easier and make more sense for them to use direct routes (for example KDFW.KLAX instead of KDFW.L4D.ZIBIT.J5N.BUBIN.P3K.KLAX (not a real route, just an example)?
You assume that airlines always fly the published SID or STAR. That's not true though. SIDs and STARS are used in procedural control to separate arriving and departing flights. When you have radar, then you don't necessarily need the procedures to do that.
When a flight is cleared for takeoff they are assigned a SID that is relevant to its destination. After takeoff and when the flight is handed off to approach (1) controller, the controller will give a vector (heading) to the flight or direct to a point on the TMA boundary. If for any reason (see below) they intend to keep the flight on the SID the controller says
proceed as cleared(2) on first contact.
For approach, the flight is assigned a STAR and when it reaches the TMA boundary, the approach controller will give them a vector and the flight will deviate from the assigned STAR. See this video from NATS and watch the arrivals to Gatwick. Compare the trajectories to what you see in the instrument procedure maps from their website. They don't seem to follow a route looking like that
source: UK AIP chart AD 2-EGKK-7-5
They seem to go direct to the holding stacks. And after exiting holding they don't seem to follow any of these routes
source: UK AIP chart AD 2-EGKK-7-12
Compare what the charts depict with what the actual vectoring looks like.
Still frame from the NATS video on Vimeo.
Now when will a flight actually do the SID or STAR you will ask. There are several cases like for example TMAs without radar service. Normally the control is procedural in that case and that includes flying the assigned SID or STAR. Another case is during unusual circumstances like for example equipment failure (radar, communications etc.) that will make radar vectoring harder or even impossible. Noise abatement can be another reason to leave a flight on the instrument departure instead of vectoring it as these routes are designed to reduce the noise impact on nearby residents and deviating from them defeats the purpose. (Thanks Bianfable!)
It's worth noting that it's common for pilots to ask for shortcuts if that can be done legally. For example if the weather conditions are VMC an arriving flight might request a visual approach. In that case IFR is canceled and the pilot will deviate from the STAR or IAP.
During takeoff, a pilot might request direct to an en-route point and thus exit the SID. If the current controller cannot grant it (let's say no radar contact), they will ask the next controller (en-route controller) if they can grant it. If yes (the en-route radar has the flight) then the flight is handed over to en-route and they are granted the direct, exiting the SID. That might not always be the case. If for example en-route control has radar contact, but because of heavy military activity the direct cannot be given safely, then the flight stays on the SID until the end.
Links to pdfs unfortunately don't work; you need to open them manually from NATS AIS page, linked in main answer.
(1) Or departure, sometimes it's the same controller.
(2) This phraseology is used in Greece (and I hope I remember it correctly). Other jurisdictions might use something slightly different.
Historically, aircraft would navigate cross country via landmarks, by flying from landmark to landmark. These were followed by radio beacons. The distance between these was limited by the visibility range of the landmark or the radio range of the beacon. Leaving the range of the previous beacon before entering the range of the next would result in navigation by dead reckoning, by compass and clock, increasing risk of errors due to winds aloft and instrument inaccuracy. Even as instruments became more accurate, such as inertial navigation systems, inaccuracy was still present. Flying off course could take you into terrain, obstructions, or restricted airspace.
Now that most airliners can navigate by GPS, they can fly to many more standard points than just radio beacons. But there are still good reasons to fly from waypoint to waypoint rather than direct.
It helps ATC keep things in order. If you have a bunch of planes enroute flying direct, it's harder to make sure they will all stay separated. It also gives pilots points that they can use as references when reporting positions. This allows pilots to calculate fuel and time enroute expected at each waypoint to track progress, and becomes especially important in oceanic airspace where there is no radar and ATC relies on these position reports to track aircraft.
As for SIDs and STARs, the arrival and departure segments are in the busiest airspace the flight will encounter. Flights are arriving from many directions and they all want to go to the same place (or vice versa, for departures). Standard procedures allow ATC to get the aircraft sequenced and separated as they descend or climb by providing standard points, altitudes, and speeds for pilots to fly. It also helps to keep departures separated from arrivals. It's much easier to keep the aircraft separated and get them where they need to go if they are in an orderly pattern than if they are each following a different route.
Sovereignty airspace - certain nations are not privy to fly aircraft through the airspace of another nation without agreements, etc.
Internal airspace - Flights are restricted from flying through certain airspace within a sovereign nation for a variety of reasons from national security to traffic separation.
Pre-established air routes - Juliette airways and Q routes have been created to provide aerial corridors allowing efficient movement of aircraft from point to point along them. It also allows for predictability and uniformity giving better traffic separation.
Efficient movement of traffic in high density terminal airspace - Airports like Hartsfield Jackson or LAX have a LOT of airplanes arriving and departing from their airspace. Routes to provide an expedient flow of traffic and sequencing for departures and arrivals were created in the form of DPs and STARs to do this. They are often laid out based on empirical data of how traffic is flowing into and out of these airports aided by experience on this from ATC controllers who work these aerodromes.
Now it is possible to file a point to point IFR flight plan. General aviation aircraft frequently do this when flying between two low traffic airports. But if you’re flying from KHOU direct to KATL in a jet, don’t expect any love from a controller, who will probably think you’re and idiot and growl out an amended clearance along a Q route following s published DP for you to copy and fly before he’ll release you.