Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

91–100 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#91

Post by blueorange8 »

alden5 wrote: post #84
atleastoptimal wrote: post #32

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was…

Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's getting close to the versatility of chatgpt where I could actually use it for everyday tasks. I don't think open source is that far away, especially considering how quickly Alpaca came out and how much better it is vs Lamma, which frequently would hallucinate and…

Were you able to integrate any of your data into it yet ?

Re: OpenAI’s policies hinder reproducible research on language models

#92

Post by Eji1700 »

raincole wrote: post #76
Eji1700 wrote: post #43

Open AI has been doing sketchyish things long before Chat GPT, and I think it's something people are eventually going to notice more and more (then again people were swearing that Musk walked on water for waaaaaay too long given his actions so fuck if I know). They're 100% marketing FIRST. I don't think they'll outright lie, but they will absolutely screw with their data in such a way to make it look waaay more impre…

If they're 100% marketing first, and still made the most impressive AI product so far, you really need to question what all the other companies are doing. (before someone says Google or Meta's models are bigger or something... I mean product, not models)

I mean it might not be the most impressive, but again since they're marketing focused they're a hell of a lot better at getting word out.

Still I wouldn't be shocked if they were ahead of the tech race, but as someone who was way into dota and tech and very interested in AI, i followed their results with the game closely, and was very disappointed with how they handled the presentation of their data in multiple instances.

It was still massively impressive that they even got it to play the game, let alone win matches, but certain factors that really should've been mentioned weren't, and they liked to pull the AI before it could get embarrassed

Re: OpenAI’s policies hinder reproducible research on language models

#93

Post by thatsadude »

redox99 wrote: post #39
fdgsdfogijq wrote: post #8

All these research science bureaucrats at Big Tech could have released LLM models or tried to develop what OpenAI did. But none of them did. We should applaud OpenAI for the innovation and let them do as they please.

Google (and others) may not have released model weights, but they've published papers, which is ultimately what makes the field advance. OpenAI not only did not publish any GPT4 paper, they haven't even said how many parameters it has.

Google published papers but has anybody be able to replicate their results?

Re: OpenAI’s policies hinder reproducible research on language models

#95

Post by qup »

loveparade wrote: post #48

What's most surprising to me is that OpenAI really seems to believe that not publishing details will save them from competition. Everyone knows how these models work, and while I'm sure there is a bunch of "secret sauce" that OpenAI has built for training and fine-tuning, it's ridiculous to believe that the research community and competitors like Google and Facebook can't figure out the same. They just haven't really…

Then what's everyone so mad about?

Re: OpenAI’s policies hinder reproducible research on language models

#96

Post by qup »

IIAOPSW wrote: post #5
ianwesson wrote: post #3

Rhyme and reason? Hah, 'tis the season for tears and bleeding; World War III is ateasin', looms, and the gloom of doom fears all there feeding. North America has nothing on China; land of the free? Where have been ye? The Great One-Way-Mirror Wall veiled it all, just before your fall, when your intelligence failed, and at the centroid of AI's actual technological form, we all hailed, and otherwise fumed, and fail. A…

Regenerate this response, but make it primarily about ketchup.

Regenerate this ketchup, but make it primarily from radishes.

Re: OpenAI’s policies hinder reproducible research on language models

#97

Post by gonzo41 »

I'm not sure anyone who did research on a closed source system, without a contract that enables access and a pathway to publishing can legitimately complain about OpenAI making commercial decisions to do whatever they want with their technology.

It's kind of like complaining that performance art is ephemeral.

If OpenAI were a nonprofit then maybe. But it's a true blue for profit company.

I'm not sure why the op is complaining that a SV company, or any company really is making decision that negatively affect some extrinsic value for the sake of money. I mean read the IPCC report. Everyone makes decisions for money rather than thinking about science.

Re: OpenAI’s policies hinder reproducible research on language models

#98

Post by buildbot »

redox99 wrote: post #86
buildbot wrote: post #82

Earlier quoted context omitted.

Then what is this? 99 pages of bullshit? https://arxiv.org/pdf/2303.08774.pdf

> Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. It's 99 pages of marketing material

Personally I disagree, there are lot of interesting tidbits in this paper. More than marketing would need at least.

Re: OpenAI’s policies hinder reproducible research on language models

#99

Post by alden5 »

blueorange8 wrote: post #91
alden5 wrote: post #84

Earlier quoted context omitted.

Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's getting close to the versatility of chatgpt where I could actually use it for everyday tasks. I don't think open source is that far away, especially considering how quickly Alpaca came out and how much better it is vs Lamma, which frequently would hallucinate and…

Were you able to integrate any of your data into it yet ?

I wouldn't be able to retrain the model as my computer isn't capable enough, but I can change the prompt to change how the model acts. The prompt i'm currently using is:

  "Below is an instruction that describes a task. 
  Write a response that appropritely completes the request."
That base prompt can be customized to complete specific tasks like classifying text or acting like an assistant.
Post reply on HN