Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Democratization in the Era of GPT-3

thegradient.pub

Post reply on HN
Print view View original

51–60 of 90 posts

Re: AI Democratization in the Era of GPT-3

#51

Post by tobr »

decasteve wrote: post #42
jackcosgrove wrote: post #39

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point. It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of traini…

The airplane and the radio brought people closer. It also improved the coordination and delivery of mass destruction. Technology can be a rollercoaster ride of benefit and detriment.

Technology is an amplifier of things we’re already doing. It’s easy to think of all the amazing things people are capable of, and imagine that new technology will help those amazing things flourish.

But people are also capable of unspeakable cruelty. Depending on your outlook on humanity, that insight may or may not give you sympathy for neo-Luddites.

Re: AI Democratization in the Era of GPT-3

#52

Post by ssss11 »

jackcosgrove wrote: post #39

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point. It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of traini…

Isnt this just human nature? Not everyone is looking to improve the world and when they get power.. well, power corrupts. I’m equally pessimistic, not about new technologies but about who controls them - we need to empower societies not corporations.

Re: AI Democratization in the Era of GPT-3

#53

Post by srtjstjsj »

jackcosgrove wrote: post #39

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point. It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of traini…

> The PC was supposed to democratize computing

It was?

Re: AI Democratization in the Era of GPT-3

#54

Post by isx726552 »

searchableguy wrote: post #37
searchableguy wrote: post #11

Earlier quoted context omitted.

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

Curious about the downvotes. If you remove compute and code, what remains is data that isn't owned by a single entity in the case of gpt3. You can sell both compute and code which is what you own but should you be able to sell data? You can't translate a commerical book without paying the copyright holder. You own the copyright for the translation but everything else still remains that of the original author. Why wou…

But you don’t understand, OpenAI has hype and “rockstars” and big names attached, so those rules don’t apply.

Your data? It’s their data now. (And Microsoft’s.)

Re: AI Democratization in the Era of GPT-3

#55

Post by DSingularity »

srtjstjsj wrote: post #53
jackcosgrove wrote: post #39

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point. It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of traini…

> The PC was supposed to democratize computing It was?

Yes. Before that it was mainframes and those who had access to $ who had access to computing.

Re: AI Democratization in the Era of GPT-3

#56

Post by nl »

srtjstjsj wrote: post #53
jackcosgrove wrote: post #39

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point. It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of traini…

> The PC was supposed to democratize computing It was?

Yes. The name gives it away.

And Microsoft's mission for many years was "a computer in every home".

Re: AI Democratization in the Era of GPT-3

#57

Post by nl »

This a bad take. In 2 years time there will be a quantized, TensorCore optimised equivalent that will run fine on desktop GPUs.

We've seen this over and over again and there is no reason to believe this is different.

Re: AI Democratization in the Era of GPT-3

#58

Post by ben_w »

jackcosgrove wrote: post #39

The PC was supposed to democratize computing. It did, and then Microsoft found a choke point. The internet was supposed to democratize communication. It did, and then Google found a choke point. It seems like this two steps forward, one step back pattern might be the rule rather than the exception. Even the article defines AI democratization in terms of using models rather than training models, as the costs of traini…

How else would you define “AI democratization”? The fact I can even program puts me in an elite of less than 0.5% of the world population [0], and that includes all specialties not just AI.

I do iPhone apps these days; even though I can follow the various tutorials for how to train an AI to recognise handwritten digits [1], I don’t actually grok the maths behind the back-propagation algorithm and why it’s better than, say, simulated annealing of the weights (which I do grok).

True democracy puts the power in the hands of the masses; making it available to all developers and only developers is like giving the vote to only the richest half of all millionaires. Packaging AI into a simple magic black box makes it available to everyone.

I don’t know if normal people have the right expectations of the tech for that to be generally wise, but that is a separate problem and black boxes are the only way I can see to achieve the goal.

[0] https://www.daxx.com/blog/development-trends/number-software...

[1] https://kitsunesoftware.wordpress.com/2018/03/16/speed-of-ma...

Re: AI Democratization in the Era of GPT-3

#59

Post by FeepingCreature »

searchableguy wrote: post #11
andreyk wrote: post #7

Earlier quoted context omitted.

Fun fact, OpenAI did not collect the data itself - they mainly used data from Common Crawl (in addition to a couple other datasets), which is compiled by a non profit that shares the dataset for free. So perhaps the license of such datasets can encourage free sharing of research outcomes. https://commoncrawl.org/

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

Should you own part of my brain if I read your comment?

Re: AI Democratization in the Era of GPT-3

#60

Post by ebalit »

ma2rten wrote: post #12
ve55 wrote: post #4

OpenAI's goal is definitely not to give everyone unlimited/equal access to powerful tools like GPT-3. We've had countless jokes about the name being 'OpenAI', and perhaps it's true that it's not the best name (along with 'democratizing' AI), but I'm not sure the author is suggesting a solution here rather than just venting that things seem kind of unfair, and no one outside of OpenAI really has much control or inform…

The article is hinting at this but I also think many people who complain that OpenAI didn't release the model don't understand how big this model actually is. Even if they had access to the parameters they couldn't do much with it. Assuming you used single precision the model is 350 gigabytes (175 billion * 2 bytes). For fast inference the model needs to be in GPU memory. Most GPUs have 16GB of memory, so you would n…

I don't think it's a fair assessment as many researchers are disappointed that the model wasn't released. And I'm pretty sure they do understand the model size concerns.

Running inference on this massive model would be a really interesting challenge for people working on model compression and pruning as well as those working on low memory training. New challenges are always a good thing for research.

Personally, I just wish it was easier to get an access to their API. I have an experiment in mind that I can't wait to try.

Post reply on HN