161–170 of 260 posts
What does the future of open-source large neural nets look like? My understanding is GPT-3 takes ~600GB of GPU memory to run inference. Does an open source model just allow you a choice of a handful of cloud providers instead of one?
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
I don't necessarily see the "team of automated googlers" as a fundamental or damning problem with GPT-like approaches. First I think people may have a lot fewer truly original ideas then they are willing to admit. Original thought is sought after and celebrated in arts as a rare commodity. But unlike in arts, where there are almost not constraints, when it comes to science or engineering almost every incremental step…
The reason for this is that if your adult life consists of just a tiny, tiny, tiny fraction of the total time of all adults, and so if an idea is relevant to more people, odds decrease exponentially that no one thought of it before.
There are always new languages though, so a great strategy is to take old ideas and bring them to new languages. I count new high level, non programming languages as new languages as well.
Earlier quoted context omitted.
It's not me I'm worried about - it's the 50% [1] of people who get their news from social media and "entertainment" news platforms. These people vote, and can get manipulated into performing quite extreme acts. At the moment a lot of people seem to have trouble engaging with reality, and that seems to be caused by relatively small disinformation campaigns and viral rumours. How much worse could it get when there's a…
> made-up number, but doesn't feel far off. Pew Research says 18% report getting news primarily from social media (fielded 10/19-6/20)[0]. November 2019 research said 41% among 18-29 year olds, which was the peak age group. Older folks largely watch news on TV[1]. [0] https://www.journalism.org/2020/07/30/americans-who-mainly-g... [1] https://www.pewresearch.org/pathways-2020/NEWS_MOST/age/us_a...
Earlier quoted context omitted.
You missed the forest for the trees. If you got a tool that can use StackOverflow to solve simple programming tasks, or to generally solve any simple task with Google, then you're sitting on a gold mine.
Yes and no. It may be useful to hire less low skilled employees and keep a few senior ones that take input from machine and decide what to keep and what to throw away. I'm not sure if a senior engineer would be more productive patching up code written by a bot or writing it from scratch. It's going to be a hard sell while you still need human supervisors. You can't trust a machine that can't reason with code implemen…
I have no doubt writing from scratch would win hands down. The main reason we patch wonky legacy code is because it's already running and depended on. If you remove that as a consideration, a senior engineer writing the equivalent code (rather than debugging code generated randomly from Google searches) would -IMO- would be more efficient and produce a higher quality program.
Earlier quoted context omitted.
I know! One day it's going to get so bad people are going to have to deploy critical thinking instead of accepting what they read at face value and suffer the indignity of having to think for themselves.
Critical thinking won't help you when the majority (or all) of your sources are tainted and contradictory. At some point, the actual truth just gets swamped.
How does the outfit intend to fund the project? OpenAI spends millions on computing resources to train the models.
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
It still baffles me that GPT turned out to be more than a glorified markov chain text generator. It seems we’ve actually made it create a model of the world to some degree. And we kind of just stumbled on the design by throwing massive data and neural networks together?
(I hope)
Earlier quoted context omitted.
But I was told GPT-3 was too powerful for mere mortal hands (unless you have an account!) and that it would be used for hate speech and to bring about skynet. How will this project avoid those terrible outcomes?
I don't know a single intelligent person who believed this argument, it simply doesn't hold up.