Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Democratization in the Era of GPT-3

thegradient.pub

Post reply on HN
Print view View original

31–40 of 90 posts

Re: AI Democratization in the Era of GPT-3

#31

Post by claudeganon »

boulos wrote: post #27
csande17 wrote: post #25

Earlier quoted context omitted.

Surely the solution here is "put the model on BitTorrent, you cowards". Like, okay, the model's big and unwieldy to run. But hardware's always getting better, and there are lots of research use-cases where it's okay if it takes ten minutes to page the model in and out of SSD while generating predictions. Plus, maybe we'd get some more discoveries in the field of efficiently running huge models. The arguments about "s…

The author though raises concerns about both the availability (openness) of the model as well as the current ability to run it due to cost (equity / equality of access). Making the model available would still not make it equal access. I’m not saying I agree or disagree with the openness argument, but the equality argument is separate.

If they released it, people would figure out a way to run it “equitably” within months, if not weeks.

The amount of cheap GPU access floating out there is nuts. You can spin up a GPU instance to do best-in-class ML stuff using Fast.Ai on services like Paperspace or Colab, right now, for free.

Re: AI Democratization in the Era of GPT-3

#32

Post by anoncareer0212 »

ma2rten wrote: post #12
ve55 wrote: post #4

OpenAI's goal is definitely not to give everyone unlimited/equal access to powerful tools like GPT-3. We've had countless jokes about the name being 'OpenAI', and perhaps it's true that it's not the best name (along with 'democratizing' AI), but I'm not sure the author is suggesting a solution here rather than just venting that things seem kind of unfair, and no one outside of OpenAI really has much control or inform…

The article is hinting at this but I also think many people who complain that OpenAI didn't release the model don't understand how big this model actually is. Even if they had access to the parameters they couldn't do much with it. Assuming you used single precision the model is 350 gigabytes (175 billion * 2 bytes). For fast inference the model needs to be in GPU memory. Most GPUs have 16GB of memory, so you would n…

A one-time investment of $60,000, $200,000 in the worst case, isn't a way of dismissing the 'many people who complain the model [wasn't released]', especially given the alternative is 'being Microsoft', which costs $1,570,000,000,000.

Re: AI Democratization in the Era of GPT-3

#33

Post by anoncareer0212 »

mensetmanusman wrote: post #24

I learned some things. GTP-3 is so complex the model requires large cloud computing resources to run. Ergo, it is also very expensive to run. Assumption: Bleeding edge AI will require tens of millions of dollars of computation before new network architectures fall out of state space. After this, the models can be pruned to be ran by mere mortals. If this is true, OpenAI will not be able to move to the next level with…

Given the worst-case assumptions of people arguing the cloud is required[1], it's a one-time investment of $60,000, $200,000 in the absolute worst case.

[1] https://news.ycombinator.com/item?id=24601264

Re: AI Democratization in the Era of GPT-3

#34

Post by cblconfederate »

I don't see where is the monetization potential of GPT3. Creating more spam? Making poetry in bulk? Being SOTA in language tasks doesnt make it useful per se, as NLP measures are rather abstract anyway. As for its technology, it's huge but unless they are hiding some secret sauce, it's a straigtforwardly transformer-based, which means someone somewhere with cheap electricity is already training it on a dump of the entire web. Where's the moat here? Perhaps in some future version that they 'll keep secret? Well it will be a sad day for humanity if they keep its development in secret, given how much its development has benefited from open science around the world.

Re: AI Democratization in the Era of GPT-3

#35

Post by chime »

Not saying I am a fan of everything going on in tech world but it is pretty evident that AI is going to end up happening the West World “Rehoboam” big AI brain way than the Jetsons AI maid way, not purely due to corporate interests but rather due to the inherent cost of just making it work. Unless we can have PB/EB worth of storage locally then it’s going to be the cloud. And a cloud API with 1000 EB storage will always outperform your local 10 PB AI model.

The most practical approach that I can see would be to minimize the cost of accessing the model, making it free like author suggests for research, students, non-profits etc. and charging more for commercial usage, basically extending the existing cloud model.

One thing that I think should be free for everyone would be testing the model for biases. Basically if every AI API to check bias on any topic was free, then it could be improved by anyone including marginalized groups. If GPT-3 thinks all Indians are either doctors, coders, or gas station owners then I would like to be able to test, verify, and offer a patch without any cost, maybe even a reward. Otherwise GPT-4-5-6 will end up throwing away all Indian sounding last names applying for a construction job.

Re: AI Democratization in the Era of GPT-3

#36

Post by csande17 »

hanoz wrote: post #22
gdb wrote: post #14

Earlier quoted context omitted.

(I work at OpenAI.) > especially as someone that didn't get a response for my requests for GPT-3 beta access We are still working our way through the beta list — we've received tens of thousands of applications and we're trying to grow responsibly. We will definitely get to you (and everyone else who applies), but it may take some time. We are generally prioritizing people with a specific application they'd like to b…

> We are generally prioritizing people with a specific application they'd like to build. Why?

OpenAI's goals are (1) make money and (2) generate positive press coverage about OpenAI. (They make statements about wanting other things but that's mainly to help them achieve (2).)

Prioritizing people with concrete project ideas helps them in both areas: they're more likely to convert into paid customers down the line, and they're more likely to generate "OpenAI technology is now being used for X" press releases.

Re: AI Democratization in the Era of GPT-3

#37

Post by searchableguy »

searchableguy wrote: post #11
andreyk wrote: post #7

Earlier quoted context omitted.

Fun fact, OpenAI did not collect the data itself - they mainly used data from Common Crawl (in addition to a couple other datasets), which is compiled by a non profit that shares the dataset for free. So perhaps the license of such datasets can encourage free sharing of research outcomes. https://commoncrawl.org/

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

Curious about the downvotes. If you remove compute and code, what remains is data that isn't owned by a single entity in the case of gpt3. You can sell both compute and code which is what you own but should you be able to sell data?

You can't translate a commerical book without paying the copyright holder. You own the copyright for the translation but everything else still remains that of the original author. Why wouldn't this apply in case of a machine learning model trained on my images, messages, intents, likeness, etc?

Re: AI Democratization in the Era of GPT-3

#38

Post by robmsmt »

boulos wrote: post #9

Disclosure: I work on Google Cloud and have worked with the OpenAI folks on large models. This article mixes both “should research be open” and “is this work cheaply reproduced / accessible”: For smaller, open models: > The average person could not recreate models of this size from scratch, but the models can run on a single machine with a single GPU. but about GPT-3: > GPT-3 represents a new circumstance. For the fi…

how big do the models get? are you able to say?

Re: AI Democratization in the Era of GPT-3

#39

Post by jackcosgrove »

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point.

It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of training sophisticated models seems beyond the common developer even according to idealists.

AI in particular seems to be a centralizing technology at this point, given that a model is often a black box to the user. The amount of data as well requires a massive telemetry apparatus, and designing likely models seems the province of people with PhDs.

So yes I'm a bit pessimistic that AI will have a democratizing effect on technology or society, at least in the near future.

Re: AI Democratization in the Era of GPT-3

#40

Post by ma2rten »

liuliu wrote: post #19
ma2rten wrote: post #12

Earlier quoted context omitted.

The article is hinting at this but I also think many people who complain that OpenAI didn't release the model don't understand how big this model actually is. Even if they had access to the parameters they couldn't do much with it. Assuming you used single precision the model is 350 gigabytes (175 billion * 2 bytes). For fast inference the model needs to be in GPU memory. Most GPUs have 16GB of memory, so you would n…

You need tiny bit of memory for activations if you don't want fine-tuning. I think for GPT-3, fine-tuning is out of window. But it is reasonable to expect inference takes less than a minute with single 3090 and fast enough SSD.

OpenAI offers a fine-tuning api.

How did come up the one minute estimate? According to a quick google search I did, the fastest SSDs these days have a bandwidth of 3100 MB/s. So it would take 112s just to read the weights.

Post reply on HN