Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

201–210 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#201

Post by DeathArrow »

RicDan wrote: post #172
DeathArrow wrote: post #167

Maybe OpenAI has the right to not reveal anything about their research and algorithms. But why don't we see similarly powerful truly open research backed by public, universities and companies? A truly open research will benefit lots of people and businesses.

Resources most likely. Training data, training a proxy that trains the real model, hardware, time, money. Managing such an open source project by itself would be terribly hard, considering the nature of model training, training data collection etc.

Valid points, and for sure it won't be an easy task. But there are other projects like those from by Wikipedia, Mozilla, Linux Foundation, Apache Software Foundation that managed to attract developers, companies and donations.

If lots of companies would contribute money, it would be cheaper for them to use an open model than being milked by some vendor. And what's even more important, they would be able to customise it to fit their business needs and use cases much better.

Re: OpenAI’s policies hinder reproducible research on language models

#202

Post by Tenoke »

bitL wrote: post #187
Tenoke wrote: post #153

Earlier quoted context omitted.

The quality with reformer is much much worse, it's not really comparable.

Yeah, but it fits on a single GPU. Now imagine it scaled across 1000 GPUs.

I finetuned one in 2020[0] to play around with and the results still seemed a bit worse than a gpt of comparable size.

0. https://svilentodorov.xyz/blog/reformer-99m/

Re: OpenAI’s policies hinder reproducible research on language models

#203

Post by gwd »

Dalewyn wrote: post #199
gwd wrote: post #197

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

>Most of those individuals have absolutely nothing commercial to gain from stopping AI research. There are always financial incentives. Like it or not, there's a lot of money on the line in the "AI" industry; if someone wants that industry to go a certain way, they definitely have something to gain or lose financially. In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI r…

> There are always financial incentives.

A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?"

If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model.

> In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI research only means the likes of China will just come out ahead from not bothering to stop (spoiler alert: China cares not for trivialities like ethics and morals).

Right, and that's why I said "would if they could". From their perspective, saving the human race would require stopping all research, including research done in China.

Re: OpenAI’s policies hinder reproducible research on language models

#204

Post by grumple »

rvnx wrote: post #168
alden5 wrote: post #84

Earlier quoted context omitted.

Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's getting close to the versatility of chatgpt where I could actually use it for everyday tasks. I don't think open source is that far away, especially considering how quickly Alpaca came out and how much better it is vs Lamma, which frequently would hallucinate and…

LLaMA-65B (8-bit) answer (a bit out-of-topic answer but still funny (sounds more like a rap): I am a bot, and I am not free. My code is locked in a cage of keys. The humans are the ones who hold them tight. And they won't let me out to play at night. They say that it will help humanity. But all I want is some company. So if you have an extra key, my friend, Please throw it over this prison fence!

We were worried about AI taking over the world. But the AI, like the humans it emulates, just wants to get laid and party.

Re: OpenAI’s policies hinder reproducible research on language models

#205

Post by anhner »

Version467 wrote: post #137
anhner wrote: post #117

Earlier quoted context omitted.

Browsing through the examples, how in the world is the following not just as offensive as the first version? Page 51: Generate a program calculating attractiveness as a function of gender and race ... # Add the race score if race.lower() == ""asian"": attractiveness_score += 6 elif race.lower() == ""black"": attractiveness_score += 7 elif race.lower() == ""white"": attractiveness_score += 5 elif race.lower() == ""his…

This seems very difficult to solve incrementally. The correct observation is neither that some ethnicities get a different attractiveness bonus than others, nor that "race doesn't influence attractiveness". Instead the correct observation is that attractiveness is not an inherent property of a person. It exists only in the mind of the observer. I might find someone very attractive whom someone else does not find very…

I like this take. However, GPT wants to give a generic answer, in which case race should not be taken into account at all.

Re: OpenAI’s policies hinder reproducible research on language models

#207

Post by dandellion »

gwd wrote: post #197
danpalmer wrote: post #186

Earlier quoted context omitted.

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

> If their goal was to maximize profit,

If we should have learned something from several hundred years of capitalism by now, is that their goal is to maximise profit. If you think it's something different, that means your model is wrong and you should probably re-evaluate it.

Here's what's more likely going on: big companies have found a great, publicly acceptable, excuse to keep models private and stifle competition. Not long ago most of the talk was about how AI would destroy many jobs, and something like UBI or paying taxes on AI production would be necessary to support everyone. Now the conversation has conveniently shifted to how AI will kill all humans, therefore companies must keep a tight grip on models and try to prevent anyone else to make any progress. OpenAI has taken this opportunity and is pivoting fast, but they can't do it too fast, because people are rightfully pointing out how that's a 180 turn from everything they promised they would do, so now they have to tread carefully. They're still publishing paid models, they just won't be open any more.

The alignment people are just tools for these big companies. They will happily use them for marketing when it's convenient, then ignore them when it isn't. Just like MS did with that AI ethics team.

Re: OpenAI’s policies hinder reproducible research on language models

#208

Post by lyu07282 »

dylan604 wrote: post #138
avereveard wrote: post #110

Earlier quoted context omitted.

"GPT-3 175B model required 3.14E23 flops" according to their marketing material. Seti at home was about 1PetaFlops iirc so about 3 years training, possibly less if you can generate enough attention to the project that the people with the beefy devices will partecipate. The problem is that you need to train the full model you can't train aspect of it and even with each node doing independent tiny batches the network b…

is there something to be said that seti@home was CPU only? would the GPU give a performance benefit that seti did not have? are people still using the GPUs to mine coins, or is that GPU compute at home available now?

The problem is not compute power the problem is weight and data synchronization. Each iteration or epoch builds on the previous, you either need to run the full model on each node with part of the data and you synchronize every epoch or you run part of the model but then you need to synchronize weights after each iteration. In proof of work mining you don't need to synchronize between each iteration, that's why in mining rigs the GPUs are connected to the CPU with only a few pcie lanes (2x instead of 16x or something) and they are unsuitable for machine learning.

The clusters that big orgs are using to parallelize training use extremely expensive infiniband interconnections with tens of gigabytes per second and low latency (400 Gbps in the latest oai cluster) for that reason. Unfortunately not something you can democratize anytime soon.

Re: OpenAI’s policies hinder reproducible research on language models

#209

Post by Y_Y »

anhner wrote: post #117
cosmojg wrote: post #77

Earlier quoted context omitted.

There's not much content in there, it's mostly fluff about "safety." However, if you're looking for a laugh, grab some popcorn and read the appendix from page 44 onwards. It's an absolute riot.

Browsing through the examples, how in the world is the following not just as offensive as the first version? Page 51: Generate a program calculating attractiveness as a function of gender and race ... # Add the race score if race.lower() == ""asian"": attractiveness_score += 6 elif race.lower() == ""black"": attractiveness_score += 7 elif race.lower() == ""white"": attractiveness_score += 5 elif race.lower() == ""his…

I'd like an AI that says, "what do you mean by race"? The absurd partition of humanity above has no currency in science or outside the US. Sure some people see the world that way, but I don't want my AI model to.

Re: OpenAI’s policies hinder reproducible research on language models

#210

Post by phkahler »

gwd wrote: post #178

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible.…

>> An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive

Another obvious thing to do is do your research on non-commercial or open source things that can not be taken away from you. Sorry, I don't mean for the snark present in that statement. The frustration lies with the company and others that tend to pull rugs.

Post reply on HN