There weren't exactly fewer jobs in textiles after the industrialization the original luddites were against. The jobs were just much, much crappier. (And more of them were filled by children).
They weren't "scared it's going to take their jobs" -- they were _seeing_ the jobs they had _actually disappear_ (not hypothetical future fear), replaced by worse-paying jobs with much worse conditions. They were exactly right that their actual lives were going to get a lot harder.
From a recent Cory Doctorow post: https://doctorow.medium.com/gig-work-is-the-opposite-of-stea... :
> Despite what you may have heard, the Luddites weren’t technophobes. They were skilled workers, expert high tech machine operators who supplied the world with fine textiles. Thanks to a high degree of labor organization through craft guilds, the workers received a fair share of the profit from their labors. They worked hard, but they earned enough through their labors to enjoy lives of dignity and comfort.
> Nineteenth century textile workers enjoyed a high degree of personal autonomy. Their machines were in their homes and they worked surrounded by family and friends, away from the oversight of the rich merchants who brought their goods to market. This was the original “cottage industry.”
> The factory owners who built their “dark, Satanic mills” weren’t interested in making life easier for textile workers by automating their labor. They wanted to make workers’ lives harder.
> Textile machines were valued because they were easier to operate than the hand-looms that preceded them, and that meant that workers who wanted a fair wage for a fair day’s work could be fired and replaced with new workers, without the logistical hassle of the multi-year apprenticeship demanded by the hand-loom and its brethren.
> As Brian Merchant documents in Blood in the Machine, his stunning, forthcoming history of the Luddites, the factory owners of the industrial revolution wanted machines so simple that children could work them, because that would let them pick over England’s orphanages, tricking young kids to come work in their factories for ten and twelve hour days.
> These children were indentured for a period of ten years, starved and mercilessly beaten when they missed quota. The machines routinely maimed or killed them. One of these children, Robert Blincoe, survived to write a bestselling memoir detailing the horrifying life of the factory owners’ child slaves, inspiring Dickens to write Oliver Twist.