Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reddit is OpenAI’s moat

cyberdemon.org

Post reply on HN
Print view View original

111–120 of 319 posts

Re: Reddit is OpenAI’s moat

#111

Post by coffeebeqn »

JimtheCoder wrote: post #9

I may be in the minority here, but if I want the opinion of Redditors on an issue, I will use a search engine to look for it specifically, thus knowing the provenance of the information I am receiving. I don't really want the corpus of Reddit data influencing the output of a generative AI model...because it's Reddit, after all... Even though I am pretty sure it is already included in the training dataset already... I…

Reddits community is passive aggressive, thinks it’s really smart, loves memes. I certainly hope OpenAI doesn’t view it at as some kind of a source of truth

Re: Reddit is OpenAI’s moat

#114

Post by rchaud »

zdragnar wrote: post #28
JimtheCoder wrote: post #9

I may be in the minority here, but if I want the opinion of Redditors on an issue, I will use a search engine to look for it specifically, thus knowing the provenance of the information I am receiving. I don't really want the corpus of Reddit data influencing the output of a generative AI model...because it's Reddit, after all... Even though I am pretty sure it is already included in the training dataset already... I…

I genuinely don't understand the appeal of AI for search. The provenance of information is just as important as the resulting information for pretty much anything I search for. I almost never accept a single search result as authoritative unless I'm pretty familiar with the source. Reddit in particular seems like a terrible set of training data. Pretty much any opinion is going to have a counter-opinion somewhere in…

> I genuinely don't understand the appeal of AI for search. The provenance of information is just as important as the resulting information for pretty much anything I search for.

Search requires work on the part of the user to distinguish between good links and bad. AI is an oracle just tells you what you're looking for.

Now you and I might think this is a terrible way to evaluate the veracity of information. But think about the new generation of mobile-native users who were raised on simplistic discourse in tweet-length messages, and would rather watch a 1-min video on a topic than scan search results for 30 seconds.

For this group, searching for information where more than 2 clicks is required is going to be a "too complicated", anda "bad user experience".

Re: Reddit is OpenAI’s moat

#116

Post by coffeebeqn »

civilitty wrote: post #73

I feel like I’m taking crazy pills: Reddit is on CommonCrawl, which means its all available without API access, permanently from AWS’s CDN. These discussions are completely moot because the path of least resistance was already available and in use by OpenAI

Thank you, I had no idea about that site

Re: Reddit is OpenAI’s moat

#117

Post by kirubakaran »

whimsicalism wrote: post #104
kirubakaran wrote: post #77

Earlier quoted context omitted.

The really impressive feat here is not even that, but MS inventing time travel to acquire Github before OpenAI was a twinkle in Sam Altman’s eyes

OpenAI certainly existed in 2018 but I don't think I noticed most people being aware of them outside of the industry.

You’re right, my bad, I misremembered the Github acquisition year

Re: Reddit is OpenAI’s moat

#118

Post by bbor »

nostrademons wrote: post #62

The leaked Google memo "We have no moat, and neither does OpenAI" is instructive here: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... Original author is an ML researcher, and the crux of his argument is that most weights in a LLM are significantly overdetermined. Once you have ingested several terabytes of natural language, you know how to generate natural language. The remaining misses are facts tha…

This is a great analysis that really cleared up the impact of that memo for that me. You clearly know but for any readers who aren’t clear: that memo is the opinion of one engineer at google, and far from the (apparent) opinion of the relevant execs

Re: Reddit is OpenAI’s moat

#119

Post by zirgs »

bzmrgonz wrote: post #36
throwuwu wrote: post #19

Earlier quoted context omitted.

You’re not alone in thinking that. The value in using Reddit as training data would be the question response format of threaded comments. The downside is that the vast majority of comments on Reddit are very low quality and repetitive. You’d have to do a lot of filtering to make it usable and what you’d be left with would be a much smaller pile of training data.

repetitive is good tho, it presents validation. I'm sure programming a good AI means adding a grading system for repetitive facts, in reddit's case, it may even accommodate likes. The problem would be when we have run-away sarcasm/irony/memes which the llamas can't handle.

Repetition is not good, because too much of it leads to overfitting.

Re: Reddit is OpenAI’s moat

#120

Post by avs733 »

duxup wrote: post #64

> There is no question that Reddit is extremely valuable as training data. How often do you append “reddit” to your searches? Is it? When I worked in networking, and later web dev work I found Reddit to be a TERRIBLE place for Q & A type situations. Answers on Reddit are often skewed by truthy answers from people with limited perspective in the industry who are surprisingly sort of militant about a given topic. For e…

> When I worked in networking, and later web dev work I found Reddit to be a TERRIBLE place for Q & A type situations.

Training in this sense is about how people use language not what is the right answer. Among other sources - the exact thing you describe is a contributor to some of the things within the (in my opinion misnamed) internal taxonomy of 'hallucinations'.

Right or wrong - reddit is human beings using (mostly english) language to speak to each other, and so if you want to train an ML model to do that - its a great source.

Post reply on HN