A lot of people in this thread are forgetting that there are services that guarantees your luggage arrives intact and on time. They are private jet charter, private jet ownership, and fractional ownership. Airlines are dealing with a historical brand image that they have extended from the early days of their industry. Back when airlines started, only rich people regularly used them, they were quite literally once-in-a-lifetime experiences for the middle class, and they practically were private jets. Relative to that high bar, they have gone down market. Something always gives when that happens. Your checked-in luggage is one factor that gave way during this commoditization of air passenger travel.
Until the basic technology of moving around people improves beyond adaptations of 19th century passenger train processes and assumptions, the situation is not likely to improve. There are lots of ways the entire passenger flying process could be automated and service dramatically improved, but many of them involve rip-and-replacement of infrastructure that no one wants to abandon (and thence pay for new infrastructure, when they can barely afford just the maintenance on the current infrastructure). This is similar to the obstacles to change the shipping industry went through to get to the containerized state we recognize today (and even for them, there still remain massive loss problems).
I'm with some of the commentators who point out that being able to fly on budget tickets and counting the trip in hours or at most days around the entire frickin' world, for the cost of at most a year's gross wages of a working class person in any of the G10 developed nations, is an amazing feat of combined technologies and processes by itself. The carefully-budgeted yet still prodigious energy expenditure alone used to accomplish this feat is amazing. To complain about checked luggage when people are paying way below shipping values per unit weight and volume is to not keep the situation in perspective. Companies like FedEx and UPS can afford to continuously improve and upgrade their package handling facilities precisely because customers are paying these companies far more than they are willing to pay airlines for handling packages with worse attributes (FedEx won't allow you to ship most suitcases without additional securing, for example). If customers ponied up for the actual cost of the capital improvements to the shipping facilities at airports to keep them on par with the likes of FedEx, and sent through luggage that had the characteristics the freight companies demand, then the check-in luggage experience would improve to the expected levels.
In the meantime, there are tactics and technologies that help mitigate the experience today. Parent post hit a lot of these, and I want to expand upon them. A lot of these tactics are simply throwing more money at the situation. For example, most families with very young (and sometimes not-so-young) children just ride with the children in their lap. Even for entire trans-continental or oceanic flights. The reality is outside of the (mythical?) archetypal HN demographic of affluent youngsters, most airline customers do not have frequent flyer status or money to spend on a full ticket for a six-month old, for example. So the following tips are aimed at HN readers who I suspect might have the means to implement them, but this list is obviously out of reach for the general airline customer population.
* There are now socks and underwear that pack super flat, are antimicrobial, and fast drip-dry. They are really expensive (like as much as $30 retail for a boxer shorts), but you can actually get away with packing just a pair for a month. Wear one, hand wash it, hang it to dry while wearing the other; I've had them dry in as fast as three hours. The antimicrobial treatment gives way after a set number of washings. ExOfficio is one such brand supplying these.
* Pack with an eye to flattening anything you can if possible. It is worth it to get travel gear if it will save weight/volume/time. For example, I see lots of people run around with water bottles; I use a collapsible water bottle (just don't fold these up, only flatten them, and they will last a long time).
* If you get elite frequent flyer status, one of the perquisites is special tagging to prioritize your check-in luggage. Check in early and get this tagging, and you increase the odds your check-in luggage will arrive with you.
* If you travel enough, pay for the annual airline lounge subscription. I have found their ticketing agents are generally more experienced with working the reservations system than your average gate agent. Plus, the lounge makes it practical to arrive even half a day early if a client meeting goes better and faster than you thought, for example. This happens to me a lot, and I just get a lunch/dinner and work from the lounge. The lounges are very much worth it for wrangling children on long layovers. Bring anti-microbial wipes and fluids, wipe down the toys before and after letting the tykes pick them up and play with them, and you can pleasantly pass hours in the children's play area in the lounges. Tip very generously, and I suspect you can even get food delivered to you (though I've been generally satisfied with the lounge options). Lounges at major hubs usually have shower facilities, which are optimal for international itineraries.
* Attitude and timing is everything. If you are using airlines, even with elite frequent flyer status, just don't pack your schedule so tightly and expect problems, then try to get some work or play done in the given context to make up for the extra time spent around the traveling. If your situation is valuable and important enough to warrant super-tight scheduling and Type A douche canoe behavior if anything goes awry with the timing, then someone will pay up for that ride in the Gulfstream. Otherwise, many times I've gotten preferential treatment simply because I was laid back and chill after a flight cancellation, for example.
* Get the smartphone app for the airline(s) you are flying for your trip, and keep your airline's number on speed dial. Many times a gate change or schedule change shows up on the app sooner than it is announced at the gate. Being first to notice confers a large advantage when there are changes. This is one of the few tips I have that anyone, not just elite travelers, can use. When there is a gate change and if you are not a preferred boarding group, gather your belongings and head over to the new gate before it is announced to get a gate lounge seat that lets you get in the front of the boarding line. When there is a scheduling change (cancellation, long delay, etc.), speed-dial the reservations line and inquire about an alternate itinerary. Stall the reservations agent with booking the change until the gate agent announces the delay/cancellation (but not for longer than a few minutes, give the reservations agents a break), because on rare occasions a mechanical problem is resolved soon after it is reported, and the gate agent will know that before the central system that updates the app knows it.
This might make for a teachable moment on HN. If what you offer is actually lower-grade, perhaps you should set expectations appropriately, even if comparing to historical standards you take a hit on brand image by going down market (or spin off a new brand to go down market).
The airlines could probably introduce an upscale service for valet check-in luggage handling that comes with a high ($5K USD?) insurance coverage and a money-back arrives-when-you-do guarantee, and guarantees personalized (as opposed to bulk) attention to your check-in luggage that is hand-carried into a separate combination-lockable container with RFID tags plastered all over the outside instead of sent through the conveyor system by itself, or at airports with particularly antiquated or unreliable conveyor systems, actually hand-carried directly onto the aircraft itself. As much as it costs to ship luggage next day, they could probably cover their costs and turn a profit on the service by charging only half or 2/3 of that freight charge. Give the employees a healthy cut of the action and noticeable bonuses for a 100% success rate each month to cut down theft, and the airlines could address a vocal, unhappy segment of the customer base while adding an additional revenue stream at the same time.
In closing, my hypothesis is the airlines will not be able to offer a better experience at budget price points until the majority of the staffing on the ground is replaced by automation and some drastic changes are made in the basic materials and people movement infrastructure.