LLM Usage in Debian: Three Proposals

I use an LLM to write code, but 90% of that usage is code review which raises suggestions or fixes which I go and implement myself. Occasionally I also have the LLM complete a single function that I know it will do correctly because the requirements are all present and unambiguous. Because the content of the function is all based on the unique context of my codebase, it has about the same chance of reproducing someone else's code that I have, nearly zero.

Under those circumstances, I'd be comfortable taking the responsibility that the code is free of legal issues. The LLM assisted review also would make me more confident of the quality of the code and capable of explaining it and justifying it.

This is vastly different from a scenario where a developer has an agent coordinate and write most of the content. There are many different ways to use LLMs and it seems like these policies need to take them into consideration. Developer responsibility for the legality/quality of the code should be no different than non-assisted code, but they could also give examples of safe and beneficial LLM use.

Think about how far we've come over the past three years. Programming with AI in ten years will be utterly unfathomable. It would be like showing Fable 5 to someone a decade ago. It will be unbelievable.

In twenty years, programming by hand might be somewhat akin to writing assembly today: a skill that a limited number of programmers understand, and fewer still actively practise. Writing code by hand will become the new "I want to be close to the metal."

Perhaps by 2050, the concept of a Linux "distro" will be obsolete, replaced by a GPL-licensed local AI that just directly assembles everything from scratch to meet your exact specifications. Instead of relying on a predefined package manager, it traverses a distributed network of trust to find suitable candidate parts, downloading and verifying every line of source code before compiling it all locally. Trust would be rooted in the local AI's ability to understand and evaluate it, meaning that what replaces the distro are the groups competing for the best models.

Perhaps by 2070, nobody even writes or distributes code any more. The future kernel and web browser aren't code, they're descriptions of verifiable intent. A software project is really an extremely detailed description of what the software is meant to do. Your local GPL-licensed AI would decide how to write, assemble, patch, optimise, and integrate it for your hardware and your threat model. That would also blur the distinction between kernels, operating systems, and applications. The AI doesn't construct a computing environment as we know it today with discrete applications. It builds a coherent system that satisfies all requirements, rebuilding parts whenever your priorities change.

Downloading compiled code? Implicitly trusting someone else's source code? These will become the things people only do on hardware that is sufficiently air-gapped from any outside influence.

OSS projects have in many cases been the first to be hurt by LLMs.

As you note, LLMs are heavily trained on OSS code. This makes them very good at "license-washing" copyleft code in to a decidedly-not-cleanroom reimplementation which is different enough at first glance that it would be hard to stop without a legal battle most OSS projects could not even dream of.

Plenty of ink has also been spilled on how LLMs have led to a significant increase in administrative workload for OSS maintainers by allowing users to produce a high volume of both bug reports and pull requests that are likely to be of varying quality or adherence to project standards but unlike a new human contributor will never develop in to a better community member. It's infinite newbies forever, a computer-generated Eternal September.

On top of all that, as noted in the Debian proposal the companies training these LLMs have proven themselves to have no shame or concern for the resources of others in the process of gathering data, inefficiently crawling every single URL they can find and nowadays even using their own models to generate new potential URLs that have never existed, in ways specifically designed to make filtering, rate limiting, or blocking the traffic altogether as hard as possible while performing what are often some of the most resource-expensive requests. These tactics have imposed substantial real costs on basically everyone who hosts their own infrastructure as well as community hosting projects, without even getting in to the hardware-related cost increases every single one of us have experienced.

LLMs have gone to a lot of OSS projects, stolen their code, beat up their maintainers, and emptied their wallets. Now the LLM people are coming by wanting big projects to work with them, "for their security" like a protection racket.