Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

141–150 of 362 posts

Re: Bag of words, have mercy on us

#141

Post by imcritic »

raincole wrote: post #122

> “Bag of words” is a also a useful heuristic for predicting where an AI will do well and where it will fail. “Give me a list of the ten worst transportation disasters in North America” is an easy task for a bag of words, because disasters are well-documented. On the other hand, “Who reassigned the species Brachiosaurus brancai to its own genus, and when?” is a hard task for a bag of words, because the bag just doesn…

Isn't it pretty clear just from the first paragraph that the author has graphomania? Such people don't really care about the thesis, they care about the topic and how many literary devices they can fit into the article.

Re: Bag of words, have mercy on us

#142

Post by eichin »

I'm just disappointed that noone here is talking about the "backhoe covered in skin and making grunting noises" part of the article. At very least it's a new frontier in workstation case design...

Re: Bag of words, have mercy on us

#143

Post by lotyrin »

altmanaltman wrote: post #129
jimbokun wrote: post #91

Best quote from the article: > That’s also why I see no point in using AI to, say, write an essay, just like I see no point in bringing a forklift to the gym. Sure, it can lift the weights, but I’m not trying to suspend a barbell above the floor for the hell of it. I lift it because I want to become the kind of person who can lift it. Similarly, I write because I want to become the kind of person who can think.

I don't really like the assumption that anyone who uses AI to, say, write an essay, is not the "kind of person who can think." And using AI to replace things you find recreational is not the point. If you got paid $100 each time you lifted a weight, would you see a point in bringing a forklift to the gym if it's allowed? Or will that make you a person who is so dumb that they cannot think, as the author is implying?

As capable as they get, I still don't see a lot of uses for these things, myself, still. Sometimes if I'm fundamentally uninspired I'll have a model roll the dice, decide what I do or don't like about where it went to create a sense of momentum, but that's the limit. There's never any of its output in my output, even in spirit unless it managed to go somewhere inspiring, it's just a way to let me warm up my generation and discrimination muscles. "Someone is wrong on the internet"-as-a-service, basically.

Generally, if I come across an opportunity to produce ideas or output, I want to capitalize on it for growing my skills and produce an individual and authentic artistic expression where I want to have very fine control over the output in a way that prompt-tweak-verify simply cannot provide.

I don't value the parts it fills in which weren't intentional on the part of the prompter, just send me your prompt instead. I'd rather have a crude sketch and a description than a high fidelity image that obscures them.

But I'm also the kind of person that never enjoyed manufactured pop music or blockbusters unless there's a high concept or technical novelty in addition to the high budget, generally prefer experimental indie stuff, so maybe there's something I just can't see.

Re: Bag of words, have mercy on us

#144

Post by ares623 »

globular-toast wrote: post #140
ares623 wrote: post #126

I think a better metaphor is the Library of Babel. A practically infinite library where both gibberish and truth exist side by side. The trick is navigating the library correctly. Except in this case you can’t reliably navigate it. And if you happen to stumble upon some “future truth” (i.e. new knowledge), you still need to differentiate it from the gibberish. So a “crappy” version of the Library of Babel. Very impre…

It's like a highly compressed version of the Library. You're basically trying to discern real details from compression artifacts.

And the halls and shelves keep shuffling around randomly.

Re: Bag of words, have mercy on us

#145

Post by raincole »

imcritic wrote: post #141
raincole wrote: post #122

> “Bag of words” is a also a useful heuristic for predicting where an AI will do well and where it will fail. “Give me a list of the ten worst transportation disasters in North America” is an easy task for a bag of words, because disasters are well-documented. On the other hand, “Who reassigned the species Brachiosaurus brancai to its own genus, and when?” is a hard task for a bag of words, because the bag just doesn…

Isn't it pretty clear just from the first paragraph that the author has graphomania? Such people don't really care about the thesis, they care about the topic and how many literary devices they can fit into the article.

I don't know enough about graphomania, but I do find this article, while I'm sure is written by a human, has qualities akin to LLM writing: lengthy, forced comparisons and analogies. Of course it's far less organized than typical ChatGPT output though.

The more human works I've read the more I feel meat intelligences are not that different from tensor intelligences.

Re: Bag of words, have mercy on us

#147

Post by meheleventyone »

akoboldfrying wrote: post #112
RayVR wrote: post #70

Earlier quoted context omitted.

Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others? If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

Human brains aren’t magic in the literal sense but do have a lot of mechanisms we don’t understand.

They’re certainly special both within the individual but also as a species on this planet. There are many similar to human brains but none we know of with similar capabilities.

They’re also most obviously certainly different to LLMs both in how they work foundationally and in capability.

I definitely agree with the materialist view that we will ultimately be able to emulate the brain using computation but we’re nowhere near that yet nor should we undersell the complexity involved.

Re: Bag of words, have mercy on us

#148

Post by jpkw »

akoboldfrying wrote: post #112
RayVR wrote: post #70

Earlier quoted context omitted.

Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others? If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

Right, I'm just going to teach my dog to do my job then and get free money as my brain is no more magic, special or different to theirs!

Re: Bag of words, have mercy on us

#149

Post by altmanaltman »

lotyrin wrote: post #143
altmanaltman wrote: post #129

Earlier quoted context omitted.

I don't really like the assumption that anyone who uses AI to, say, write an essay, is not the "kind of person who can think." And using AI to replace things you find recreational is not the point. If you got paid $100 each time you lifted a weight, would you see a point in bringing a forklift to the gym if it's allowed? Or will that make you a person who is so dumb that they cannot think, as the author is implying?

As capable as they get, I still don't see a lot of uses for these things, myself, still. Sometimes if I'm fundamentally uninspired I'll have a model roll the dice, decide what I do or don't like about where it went to create a sense of momentum, but that's the limit. There's never any of its output in my output, even in spirit unless it managed to go somewhere inspiring, it's just a way to let me warm up my generatio…

Yeah, that makes sense. If people don't see uses for AI, they shouldn't use it. But going out of the way to imply that people who use AI cannot think is pretty stupid in itself imo. I am not sure how to put this, but maybe to continue with your example, I like a lot of indie stuff as well, but I don't think anyone who watches, say, Fast and Furious, cannot think or is stupid, unless they explicitly make it the case by speaking, etc.

So my issue is that you shouldn't dismiss AI use as trash just because AI has been used. You should dismiss it as trash because it is trash. But the post says is that you should dismiss it as trash because AI was involved in it somewhere so i feel that's a very shitty/wrong attitude to have.

Re: Bag of words, have mercy on us

#150

Post by littlestymaar »

akoboldfrying wrote: post #112
RayVR wrote: post #70

Earlier quoted context omitted.

Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others? If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

> LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?

“Internal combustion engines and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?”

The question isn't about what an hypothetical mechanism can do or not, it's about whether the concrete mechanism we built does or not. And this one doesn't.

Post reply on HN