Live data from Hacker News

Anyone got a contact at OpenAI. They have a spider problem

mailman.nanog.org

261–270 of 400 posts

Re: Anyone got a contact at OpenAI. They have a spider problem

#261
post #141

Earlier quoted context omitted.

This sounds interesting, could you give more detail on what you're referring to?

I'm referring to his two works, the "Tractatus Logico-Philosophicus" and "Philosophical Investigations". There's a lot explored here, but Wittgenstein basically makes the argument that the natural logic of language—how we deduce meaning from terms in a context and naturally disambiguate the semantics of ambiguous phrases—is different from the sort of formal propositional logic that forms the basis of western philosop…

Wait, are you saying this something you read in both the Tractatus and the PI? They are quite opposed as texts! That's kinda why he wrote the PI at all..

I don't think Wittgenstein would agree, first of all, that there is a "natural logic" to language. At least in the PI, that kind of entity--"the natural logic of language"--is precisely the kind of weird and imprecise use of language he is trying to expose. Even more, to say that such a logic "allows" for anything (like metaphors) feels like a very very strange thing for Wittgenstein to assert. He would ask "what do you mean by 'allows'"?

All we know, according to him (in the PI), is that we find ourselves speaking in situations. Sometimes I say something, and my partner picks up the right brick, other times they do nothing, or hit me. In the PI, all the rest is doing away with things, like our idea of private language, the irreality of things like pain, etc. To conclude that he would make such assertions about the "nature" of language, of poetry, whatever, seems like maybe too quick a reading of the text. It is at best, a weirdly mystical reading of him, that he probably would not be too happy about (but don't worry about that, he was an asshole).

The argument you are making sounds much more French. Derrida or Lyotard have said similar things (in their earlier, more linguistic years). They might be better friend to you here.

Re: Anyone got a contact at OpenAI. They have a spider problem

#263
post #86

Earlier quoted context omitted.

I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet forum that I don't know the answer to, I'm certainly not going to post "I don't know" since that wouldn't be useful. In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence…

I'm really cross that the word "hallucination" has taken off to describe this as it's clearly in incorrect word. The correct word to describe it is "confabulation", which is clinically more accurate and a much clearer descriptor of what's actually going on. https://en.wikipedia.org/wiki/Confabulation

That’s a great word for some types of hallucinations. But some things that are called hallucinations may not be memory errors.

Re: Anyone got a contact at OpenAI. They have a spider problem

#264

Earlier quoted context omitted.

Gonna say you’re way off there. Once you decompress common crawl and index it for FTS and put it on fast storage you’re in for some serious pain, and that’s before you even put it in your ML pipeline. Even refined web runs about 2TB once loaded into Postgres with TS vector columns, and that’s a substantially smaller dataset than common crawl. It’s not just a dumping a to of zip files on your NAS, it’s making the data…

How important is full text search for training an LLM, compared to a pile of zip files with a gigabyte of text each?

Maybe not full full text search, but you'll generally want to remove the duplicates and suchlike.

Re: Anyone got a contact at OpenAI. They have a spider problem

#265
post #259

He's not done his robots.txt properly, he's commented out the bit that actually disallows it # silly bing #User-agent: Amazonbot #Disallow: / # buzz off #User-agent: GPTBot #Disallow: / # Don't Allow everyone User-agent: * Disallow: /archive # slow down, dudes #Crawl-delay: 60

The contents changed between then and now.

Re: Anyone got a contact at OpenAI. They have a spider problem

#267

Earlier quoted context omitted.

I proposed[1] the portmanteau "hallucofabulation" as a compromise, but it hasn't caught on yet. I'm totally shocked and dismayed by this, of course. [1]: https://news.ycombinator.com/item?id=36977935

The re-use of the "c" as a soft c in "hallucinate" and then a hard c in confabulate is confusing, and probably affecting the uptake of your neologism.

Yes, it would probably have to be halucinfabulation for purely phonetic reasons.

Re: Anyone got a contact at OpenAI. They have a spider problem

#269

Earlier quoted context omitted.

How important is full text search for training an LLM, compared to a pile of zip files with a gigabyte of text each?

Maybe not full full text search, but you'll generally want to remove the duplicates and suchlike.

I guess you want some fast extra storage for as long as it takes to run https://github.com/chatnoir-eu/chatnoir-copycat but that's a very temporary thing.

Re: Anyone got a contact at OpenAI. They have a spider problem

#270
post #104

Earlier quoted context omitted.

> The end state of training on web text has always been an ouroboros And when other mediums have been saturated with AI? Books, music, radio, podcasts, movies -- what then? Do we need a (curated?) unadulterated stockpile of human content to avoid the enshittification of everything?

>Do we need a (curated?) unadulterated stockpile of human content to avoid the enshittification of everything? Either that, or a human level AI.

Maybe we could print out knowledge on dead trees and store them somewhere, perhaps make it publically available? (stolen joke, not mine).
Post reply on HN