Live data from Hacker News

The Private Capture of Public Genius

wysr.xyz

61–70 of 111 posts

Re: The Private Capture of Public Genius

#61
post #48

Earlier quoted context omitted.

I do think it's easy, in this technology discussion bubble in which we dwell, to overestimate the centrality of LLMs to the arc of developments in our time. They'll be important, but I don't think they'll be _that_ important, because the rest of society and the economy don't move at the speed of SV. Instead, they'll be overtaken by other, more traditional categories of events, ruptures and dislocations. Moreover, fol…

Functionally speaking, current models are intelligent by any reasonable definition of the term. You can test them on their understanding of complex domains such as software systems, and within that domain as an example, you can judge them on their ability to diagnose bugs, fix bugs, explain code to humans, design systems given vague specifications, and implement new systems. On all these tasks, current models are mea…

It's quite trivial to show that an LLM doesn't have underlying intent, and that it can only emit direct textual convolutions of its training and not combine tokens in truly novel ways. This is the very thesis of the world-model folks, e.g. LeCun et al, that LLMs are a general intelligence dead-end because they lack any inner concept of the world around them, and do not reason from that.

Furthermore, LLMs clearly do not "reason", despite the marketing around this term; their "chains of thought" are the nothing more than the result of having been trained on explicitly verbalised multi-step processes. There are many cases where the putative result arrived at in the chain of thought does not match the result emitted.

Whether they are "better" at software development than I am greatly depends on whether one is asking them to retrace worn technology paths that are well-represented in their training--in effect, to copy prior art--or to do something in quite obscure technology, or something quite novel altogether. (However, I will happily concede that most everyday business programming involves neither.)

Still, if LLMs were actually intelligent, let alone superhuman in the sense you suggest, then we would expect major scientific breakthroughs to be raining from the sky. If, say, an Einstein, could transform physics with only the knowledge gleaned from a human's feeble capacity to retain the literature of the time, I'd expect LLMs, who retain orders of magnitude more information with far greater fidelity and precision, to have offered at least a small slither of evidence of their superhuman capabilities.

I would also expect the objective progress and capabilities of this galaxy brain to be accelerating, not substantially slowing down as it has. GPT-2 to GPT-3 was truly a quantum leap, GPT-3 to GPT-4 was a substantial jump, GPT-4 to GPT-5 was meh, 5+ is basically unimportant, and so it goes for the other models. There are, of course, holes plugged and benchmarks where these evolutions have been, in various niche ways, consequential, but in the plainspoken meaning of model capability, the low-hanging fruit of pretraining was clearly exhausted quite some time ago. The carnival has been running on "agentic" / MCP / RAG / tool-use fumes since. This is moderately impressive and adds quite a bit of runway, but intelligence it is not.

Re: The Private Capture of Public Genius

#62

The troubles over copyright infringement in AI training data remind me a bit of Eli Whitney and the cotton gin. There he suffered massive patent infringement, that basically stopped being enforced due to the sheer economic importance of the cotton gin. In a similar manner, I think there is a reasonably strong argument that it was wrong to use copyrighted material for AI training without paying royalties nor even aski…

Author here. Appreciate your thoughts and I mostly agree actually.

I'll explain more over the next few essays, but I am designing my proposed regulatory structures to try to accomplish 2 purposes in tension simultaneously like the Fed: 1. Maintain global competitiveness for frontier labs 2. Create a societal hedge against the AI bull case (AKA the economic black hole case)

A % of revenue scales in a way that I think balances the two well while avoiding all the other problems I mentioned in the essay. I’ll get into ratchets, timing, and thresholds in later essays, but I agree the China/competitiveness problem is central.

Re: The Private Capture of Public Genius

#63

This is a well written essay. I had hoped it might address the role of distillation and open source in diffusing ownership of this technology back to the public that made it possible. And the AI labs’ rank hypocrisy in this area.

Author here. Thanks for reading and the kind words. I will talk about distillation and OS in coming essays (the is a multi-part series).

Re: The Private Capture of Public Genius

#64
post #54

The troubles over copyright infringement in AI training data remind me a bit of Eli Whitney and the cotton gin. There he suffered massive patent infringement, that basically stopped being enforced due to the sheer economic importance of the cotton gin. In a similar manner, I think there is a reasonably strong argument that it was wrong to use copyrighted material for AI training without paying royalties nor even aski…

I'd be fine with the nuclear compromise: if AI training is allowed to infringe copyright, then there is no legal protection for the models themselves and their weights. Distillation should be explicitly legal. There will of course be a huge cat and mouse game about it, but let's have competition drive prices down on the stolen IP.

This is my position, as well, and I have switched exclusively to Chinese models in support of this view (and cost and because they're pretty awesome).

Re: The Private Capture of Public Genius

#66
post #48

Earlier quoted context omitted.

Functionally speaking, current models are intelligent by any reasonable definition of the term. You can test them on their understanding of complex domains such as software systems, and within that domain as an example, you can judge them on their ability to diagnose bugs, fix bugs, explain code to humans, design systems given vague specifications, and implement new systems. On all these tasks, current models are mea…

It's quite trivial to show that an LLM doesn't have underlying intent, and that it can only emit direct textual convolutions of its training and not combine tokens in truly novel ways. This is the very thesis of the world-model folks, e.g. LeCun et al, that LLMs are a general intelligence dead-end because they lack any inner concept of the world around them, and do not reason from that. Furthermore, LLMs clearly do n…

You're working with mystical definitions of words like "intelligence" and "reasoning", where there's some ineffable quality that you can't define that makes those properties difficult to achieve.

I'm not a mystic, so what I care about is functional behavior. Functionally speaking, models perform the functional equivalent of reasoning. Functionally speaking, they exhibit intelligence. Is it a perfect equivalent of human intelligence? Obviously not, but so what? The ball is in your court if you want to try to nail down the mystical quality you think isn't being achieved.

Re: The Private Capture of Public Genius

#67
post #58

Earlier quoted context omitted.

I don’t agree. I think they clearly lack general intelligence, which is why all the AI companies can think of is sourcing more and more niche domain data to plug more and more holes. When you get enough of that stuff, you can get a convincing illusion of general intelligence, but there is always another car wash test coming that shows it isn’t real.

What you’re doing is the classic No True Scotsman fallacy. “Isn’t real” is a tell for that. It’s the same as saying it’s not a “true” intelligence. Define what you mean by “true”, then. But for me, what I’m interested in is functional capability, not some mysterious ineffable quality that only humans can have. And in terms of functional capability, current models are certainly better at software development than you…

Perhaps I was unclear: the part that isn't real is the generalization. The models appear to generalize because they're fitted to so many discrete tasks that it almost doesn't appear to matter. But then it leaks, and the failure modes reveal no coherent model or process that generated the failure. The labs only have one answer for this, which is more duct tape.

> current models are certainly better at software development than you are. You’re just in denial.

I have no ego in this. It could be true. Wikipedia is also "smarter" than I am; it "knows" so many more concepts than I could ever. But regardless, I think the state of slopcoded messes like Claude Code shows that the models are missing something.

Re: The Private Capture of Public Genius

#68

A similar appropriative-use vs. public-trust evaluation is playing out this year as the California State Water Resources Control Board reevaluates Los Angeles’ right to divert water from the Mono Basin in the Eastern Sierra Nevada. The foundational case for Mono Lake as a public trust resource is National Audubon Society v. Superior Court (1983) [1]. The California Supreme Court evaluated appropriative water rights a…

Author here. This is a great point! I explored this case extensively and ultimately didn't include it in this essay because I didn't want to get bogged down in east coast riparian rights vs the west coast judgments. It is certainly a tumultuous time for contested public rights.

Re: The Private Capture of Public Genius

#69
post #52

The troubles over copyright infringement in AI training data remind me a bit of Eli Whitney and the cotton gin. There he suffered massive patent infringement, that basically stopped being enforced due to the sheer economic importance of the cotton gin. In a similar manner, I think there is a reasonably strong argument that it was wrong to use copyrighted material for AI training without paying royalties nor even aski…

> ... there is a reasonably strong argument that it was wrong to use copyrighted material for AI training without paying royalties nor even asking for permission. But equally, every country wants to have the most powerful models and enforcing such royalties would make it effectively impossible to train them as the amount of material required would cost an insane amount in royalty fees. i think you're spot on this is…

This is an astute observation. I think it reflects a larger and longer-running strain in the relationship between technocracy and the humanities, though, of which this latest iteration is just even more choleric and rote. The plumbers of capitalism always seem to have had deep contempt for the arts and the humanities, not in the least because they didn't do too well at them in school or didn't understand how philosophy relates to making money, or something.

This has led to some rather fantastical conclusions on both sides, however. On one side, there's an almost sadistic "revenge of the nerds" glee at the notion that these airy-fairy, frou-frou, and "feminised" liberal arts majors will finally crumble before the stochastic parrot machine god, and on the other side, a no less comical notion that after "AI" ushers in utopia, then high-brow artistic and literary pursuits will be all that remains for us to do.

Re: The Private Capture of Public Genius

#70

A similar appropriative-use vs. public-trust evaluation is playing out this year as the California State Water Resources Control Board reevaluates Los Angeles’ right to divert water from the Mono Basin in the Eastern Sierra Nevada. The foundational case for Mono Lake as a public trust resource is National Audubon Society v. Superior Court (1983) [1]. The California Supreme Court evaluated appropriative water rights a…

Also, thank you for including the passage about Elinor Ostrom's work. The conditions outlined as necessary for an enduring public commons go a long way toward explaining why the existing decision governing Mono Lake's public interest resources hasn't delivered the intended outcome.

Glad you noticed. My personal preference is a more aggressive pursuit of a true public data commons, but for better or worse I am trying to write the essay series by balancing what I can be implemented sooner/with less pain + heartache and I think that discussion is a much longer debate with much harder policy to define.
Post reply on HN