Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Private Capture of Public Genius

wysr.xyz

Post reply on HN
Print view View original

71–80 of 111 posts

Re: The Private Capture of Public Genius

#71

Post by abalashov »

anon373839 wrote: post #67
antonvs wrote: post #58

Earlier quoted context omitted.

What you’re doing is the classic No True Scotsman fallacy. “Isn’t real” is a tell for that. It’s the same as saying it’s not a “true” intelligence. Define what you mean by “true”, then. But for me, what I’m interested in is functional capability, not some mysterious ineffable quality that only humans can have. And in terms of functional capability, current models are certainly better at software development than you…

Perhaps I was unclear: the part that isn't real is the generalization. The models appear to generalize because they're fitted to so many discrete tasks that it almost doesn't appear to matter. But then it leaks, and the failure modes reveal no coherent model or process that generated the failure. The labs only have one answer for this, which is more duct tape. > current models are certainly better at software develop…

Although I agree with you, in fairness, there are some lively controversies in the world of cognitive science and philosophy of mind about whether this meaningfully differs from human thinking at sufficient scale.

The general idea is that the building blocks of "coherent models" and "processes" within the churning of the human intellect are also, in so many words, prior art and existing concepts, and so, while the human mind is not a text model, a sufficiently large and sensorily multimodal neural net would not be too different. Neural nets are, after all, inspired by what we understand of human cognition -- they'd say.

Re: The Private Capture of Public Genius

#72

Post by martialg »

marcus_holmes wrote: post #2

This was interesting right up until "The fund pays every eligible American the same amount each year. " I'm in Australia. I've contributed my share of dirt to the delta. Why do I not get a share of this? I get that the frontier companies are (for the moment) US companies. But that's just corporate ownership, it's not what we're talking about. We're talking about compensating the people who wrote the training data for…

[flagged]

Re: The Private Capture of Public Genius

#73

Post by martialg »

martialg wrote: post #6
marcus_holmes wrote: post #2

This was interesting right up until "The fund pays every eligible American the same amount each year. " I'm in Australia. I've contributed my share of dirt to the delta. Why do I not get a share of this? I get that the frontier companies are (for the moment) US companies. But that's just corporate ownership, it's not what we're talking about. We're talking about compensating the people who wrote the training data for…

Author here. Thanks for reading! I have additional essays coming out that will address this exact issue and other issues I know that people will raise. I’m building the essays series around arguing for practical policy I believe can get implemented and am sequencing it as thoughtfully as I can. I just can’t fit every argument into every essay.

[flagged]

Re: The Private Capture of Public Genius

#74

Post by abalashov »

antonvs wrote: post #66
abalashov wrote: post #61

Earlier quoted context omitted.

It's quite trivial to show that an LLM doesn't have underlying intent, and that it can only emit direct textual convolutions of its training and not combine tokens in truly novel ways. This is the very thesis of the world-model folks, e.g. LeCun et al, that LLMs are a general intelligence dead-end because they lack any inner concept of the world around them, and do not reason from that. Furthermore, LLMs clearly do n…

You're working with mystical definitions of words like "intelligence" and "reasoning", where there's some ineffable quality that you can't define that makes those properties difficult to achieve. I'm not a mystic, so what I care about is functional behavior. Functionally speaking, models perform the functional equivalent of reasoning. Functionally speaking, they exhibit intelligence. Is it a perfect equivalent of hum…

One hardly needs to soar to the exalted heights of "mysticism" or any sort of "ineffable qualities" to come into contact with the limitations of LLMs.

1) Models do not perform the functional equivalent of reasoning at all. When we reason, we don't simply babble out textual derivations of prior examples of "reasoning" to which we have been exposed, arrive at a conclusion, then occasionally state an altogether different conclusion while pointing at the largely irrelevant reasoning to substantiate it.

2) Models have real-world, not-at-all-mystical functional constraints that are directly relevant to the production of everyday economic work in which one attempts to involve them. Their inability to extrapolate or maintain clear mental models leads to staggering, head-scratching mistakes that even a very feeble and developmentally awry human intelligence would not make.

A basic, if well-worn example that was widely discussed in the last year or two:

https://medium.com/@JerryCuomo/why-ai-gets-the-strawberry-qu...

However, this is emblematic of a much larger idea: the LLM doesn't have any idea what a letter is or what you're asking it to do. This isn't a question of "ineffable qualia"; when it doesn't know what something is at any essential level, it can't competently solve problems related to it. One bumps up against this in everyday programming and all the time.

Also, what is "mystical" about my demand for the kind of scientific progress--no, forget that, any scientific progress--that a functional superhuman intelligence would yield? I am not a "mystic", either; I want functional results, show me the functionality.

Re: The Private Capture of Public Genius

#75

Post by martialg »

phrotoma wrote: post #65

Anybody know where that gordon moore quote comes from? A little searching didn't produce sources for me.

Author here! It's from a workshop in 2001 for the National Research Council's Board on Science, Technology, and Economic Policy. He gave a talk.

You can ctrl+f for it at this link

https://www.ncbi.nlm.nih.gov/books/NBK208682/

Re: The Private Capture of Public Genius

#76

Post by goodpoint »

typ wrote: post #5

We cannot always want to capture only the (temporary) winners whenever we see a lucrative business and expect to share a free ride. I'd also assume that most of the revenue these AI labs are making is turned into depreciating fixed capital (hardware) and OPEX at this point. Why don't we capture Meta and Google as they allegedly take advantage of more publicly available information for profit? Let alone the truly valu…

> we see a lucrative business and expect to share a free ride

the lucrative business are the freeloaders

Re: The Private Capture of Public Genius

#77

Post by martialg »

stale2002 wrote: post #41

I'm surprised that the Author has missed a very important corollary to the diffusion of "genius theft" that they are bringing up. And that is the diffusion of the beneficiaries. Maybe they think that OpenAI and Anthropic are actually worth, like a trillion dollars each and can therefore have value extracted from them. I'm not so convinced. What if there aren't frontier labs spending billions on training a model. What…

Author here - thanks for reading and thoughtfully replying

I’d personally love a world where open weights compete with proprietary ones, but I don’t believe it solves the core concentration issue. In that scenario most value still flows to capital holders, it’s just hardware holders not model weight holders.

I emphatically do not want to put the genie back in the bottle nor do I believe it’s possible. Technology has never been restrained for long (export controls on cryptography textbooks in the 90’s comes to mind here)

I also have already personally benefited a great deal from LLMs. I actually frame the entire essay series from this perspective in my prelude essay here: https://www.wysr.xyz/p/a-consigliere-on-every-desk-and-in

However, I believe we may disagree on the definition of a public good. If you’re referring to the free tiers of private models, then I’d argue that unless there is some legal framework passed that forces the frontier labs to offer that to everybody, it’s a customer acquisition cost laundered as a public good. It could disappear at any time and probably will as cutting edge model margins are reduced via competition.

In general, I believe the best AI policy balances allowing for maximum competitive market dynamics while hedging existential economic disruption risk for the general population.

I’ll go into this deeper over the next few essays. Appreciate the feedback

Re: The Private Capture of Public Genius

#78

Post by karahime »

inigyou wrote: post #51
stale2002 wrote: post #41

I'm surprised that the Author has missed a very important corollary to the diffusion of "genius theft" that they are bringing up. And that is the diffusion of the beneficiaries. Maybe they think that OpenAI and Anthropic are actually worth, like a trillion dollars each and can therefore have value extracted from them. I'm not so convinced. What if there aren't frontier labs spending billions on training a model. What…

There aren't any open source LLMs by the way

Sure there are. Ai2's OLMo, EleutherAI's Pythia, and LLM360's Amber are actually open source, top to bottom, training data to checkpoints to code, to name three of them. You also don't have to look far on Huggingface to find smaller models developed with open source processes all the time.

Re: The Private Capture of Public Genius

#79

Post by mhb »

wwind123 wrote: post #12
marcus_holmes wrote: post #2

This was interesting right up until "The fund pays every eligible American the same amount each year. " I'm in Australia. I've contributed my share of dirt to the delta. Why do I not get a share of this? I get that the frontier companies are (for the moment) US companies. But that's just corporate ownership, it's not what we're talking about. We're talking about compensating the people who wrote the training data for…

I agree UN sounds like a good organization to help distribute the wealth created by AI to the world. But this idea won't be considered by the current U.S. administration. In that case what other countries can do is probably to tax the AI companies at rates higher than regular companies.

The UN is a terrible organization for that. It is a corrupt, morally bankrupt institution. Giving it any more power or legitimacy is not a net positive.

Re: The Private Capture of Public Genius

#80

Post by appreciatorBus »

archagon wrote: post #39
marcus_holmes wrote: post #14

Earlier quoted context omitted.

It's a great idea, I hope you get traction with it :) I am personally coming to the conclusion that having these vast repositories of knowledge that can actually talk to us is actually great. We have some issues to solve, but the end-state of having a global repository of all knowledge that can talk to us and answer questions is actually an amazing outcome. We just need to solve those problems first; mostly getting p…

Imagine that AI lives up to its promise, gets captured by a handful of corporations, and ushers in a world of untouchable über-oligarchs that de facto rule the world. Would it have been worth it then? Can this problem even be solved?

Imagine that AI lives up to its promise, gets captured by unaccountable bureaucrats and socialist dictators, ushering in a world of untouchable über-oligarchs that de facto rule the world. Would it have been worth it then? Can this problem even be solved?
Post reply on HN