There are a great many problems with LLM's and AI in general. The underlying problem is, LLMs break the social contract in about as many ways as a human can dream up. There is no useful beneficial purpose that doesn't also open the door to intractable destructive forces manifold over that benefit; its a modern day pandora's box, and its hubris to think otherwise.
This underlying societal problem has knock-on effects just like chilling effects on free speech, but these effects we talk about are much more pernicious.
To understand, you have to know something about volunteer work, and what charity is. Charity is intended to help others who are doing poorly, at little to no personal benefit to you, to improve their circumstances. Its gods work, there is very little expectation involved on the giver. Its not charity if you are forced to give.
When you give something to someone as charity, and then that someone, or an observing third-party related to that charity extorts you and attempts to coerce you for more. What do you think the natural inclination is going to be?
There is now additional coercive personal cost attached to that which was not given freely; what naturally happens. This is theft.
Volunteer psychology says, you stop giving when it starts costing you more than you were willing to give. Those costs may not be monetary, in many cases they are personal, and many people who give often vet those who they are giving to so as to help good people rather than enable evil people. LLM's make no distinction.
Less people give, but that is not all. Less people who are intelligent give anything. They withdraw their support. Where the average person in society was once pulled up to higher forms of thought by those with intelligence and a gift for conveying thought, that vanishes as communication vanishes. From that point forward, the average is pulled only to baser and baser forms of thought and the average people seem like the smartest people in the room though that feeling is flawed (because they aren't that intelligent).
The exchange for charity is often I'll help you here, and you are obligated to use it to get yourself into a better position so you can help both yourself and other people along the way. Paying it forward if you can. Charity is wasted on those that would use what is given to support destructive and evil acts.
There are types of people who will just take until no more can be given which is a type of evil, which is why you need to vet people beforehand but an LLM is simply a programmatic implementation of taking. The entire idea of a robot worker replacing low-mid level jobs during limits of growth is one that seems enticing from the short-sighted producer side except it ends up stalling economic activity. Economies run through an equal exchange between two groups. The moment there is no exchange because money isn't being provided for labor, products can't be bought. Shortages occur, unrest, usually slavery then death when production systems for food break down.
We are already seeing this disruption in jobs in the Tech sector that is 5 times the national unemployment during peak hiring (where offpeak has hiring freezes), in a single year. When you can't keep talent, that talent has options and goes elsewhere, either contributing within the economy or creating their own (black markets). ECP in non-market socialist systems (like what we are about to have in about 5 years), guarantees a lot of death or slavery.
People volunteer to help others, if someone then uses those conversations to create a humunculus that steals that knowledge and gives it to anyone that asks, even potentially terrorist or other evil actors, no one who worked for the expertise will support that.
Inevitably it also means someone will use that same thing to replace those expert jobs entry level portions with a machine that can't reason, and no new candidates can get the experience to become an expert. Its a slow descent into an age of ruin that ends in collapse.
You end up with cyclical chaotic spirals of economic stagnation, followed by shortage, followed by slavery or death from starvation; no AGI needed. All that's needed is imposing arbitrary additional cost on those least able to bear it. Interference in the job market, making it impossible to find work, gender relations (making it impossible to couple up and have babies), political relations (that drive greater government and control at the same time limiting any future). Its dystopian but its becoming a reality because of evil people in positions that allow front-of-line blocking to course correct continue their evil work; and they are characteristically wllfully blind to the destructive acts they perform and the consequences.
There are a lot of low pay jobs that will just vanish, people don't have enough money to raise children as it is; old crowds out the new until finally, a great dying occurs. That is the future if we don't stop it and resolve the changes that were made (as the bridges were burnt) by the current generation in political power. They should have peaked in power in 95, exited in 2005-2010. They still hold the majority and its 2024 (30 years after taking power). They'll die of old age soon, and because they were the only ones that knew all the changes that were made, new bridges will have to be built from scratch with less knowledge, resources, goodwill/trust, and people. They claimed they wanted a better world for their children, while creating a hellscape for them instead made of lies and deceit.
Very few people are taught specifically what the social contract entails. Its well past time people learn, lest they end up being the supporting cause of their own destruction.
Thomas Paine's Rights of Man has a lot of good points on this.