81–90 of 260 posts
How does the outfit intend to fund the project? OpenAI spends millions on computing resources to train the models.
Earlier quoted context omitted.
Don't read news. Go to original sources and scientific papers. If you really want to understand something, a news website should only be your starting point to look for keywords. That is true today as it will be "post-GPT-3".
> Go to original sources and scientific papers. Given how much bunk “science” (and I'm talking things completely transparent to someone remotely competent in the field) gets published, especially in psychology, it's difficult to do even that.
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
It's a shame that it has turned out to be necessary to externally re-make and re-train a model that has come out of company called `OPEN`AI. Wasn't one of the founding principles of it that all of the research would be available to the public? Isn't that the premise on which the initial funding was secured? Best of luck to Eleuther.
Wild-Ass Guess (Ass-Guess) incoming: OpenAI was built to influence the eventual value chain of AI in directions that would give the funding parties more confidence that their AI bets would pay off. This value chain basically being one revolving around AI as substituting predictions and human judgement in a business process, much like cloud can be (oversimply) modeled as moving Capex to Opex in IT procurement. They sa…
Earlier quoted context omitted.
Does it also solve the problem of everyone having different hardware?
It does. For most models, your home broadband would be far too slow though.
Earlier quoted context omitted.
OpenAI turning out to be a total bait and switch. Especially true when your co-founder is actively calling you out on it[1] Remember kids: if it's not a non-profit organization it is a _for_ profit one! It was silly to expect anything else: > In 2019, OpenAI transitioned from non-profit to for-profit. The company distributed equity to its employees and partnered with Microsoft Corporation, who announced an investment…
It will be interesting to see the attitude of Microsoft towards this project in the light of their "Microsoft loves open source" propaganda.
Of course they can - just because you contribute to open source, and do that because you also benefit from open source projects, doesn't mean you have to do absolutely everything under open source.
Especially considering OpenAI isn't even Microsoft's IP or codebase.
I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…
I know! One day it's going to get so bad people are going to have to deploy critical thinking instead of accepting what they read at face value and suffer the indignity of having to think for themselves.
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
Damn, this could replace so many programmers, we're doomed!
Earlier quoted context omitted.
Wild-Ass Guess (Ass-Guess) incoming: OpenAI was built to influence the eventual value chain of AI in directions that would give the funding parties more confidence that their AI bets would pay off. This value chain basically being one revolving around AI as substituting predictions and human judgement in a business process, much like cloud can be (oversimply) modeled as moving Capex to Opex in IT procurement. They sa…
Apparently you don't need to be a large company to train GPT-3. EleutherAI is using free GPU from CoreWeave, the largest North American GPU miner, who agreed to this deal to get the final model open sourced and have their name on it. They are also looking at offering it as an API.
Open models will function a lot like Open Source does today, where there are hobby projects, charitable projects, and companies making bad strategic decisions (Sun open sourcing Java), but the bulk of Open AI (open research and models, not the company) will be funded and released strategically by large companies trying to maintain market power.
I'm thinking of models that will take $100 million to $1 billion to create, or even more.
We spend billions on chip fabs because we can project out long term profitability of a huge upfront investment that gives you ongoing high-margin capacity. The current (admittedly early and noisy) data we have about AI models looks very similar IMO.
The other parallel is that the initial computing revolution allowed a large scale shift of business activities from requiring teams of people doing manual activities, coordinated by a supervisor towards having those functions live inside a spreadsheet, word processor, or email.
This replaces a team of people with (outdated) specializations with fewer people accomplishing the same admin/clerical work by letting the computer do what it's good at doing.
I think a similar shift will happen with AI (and other technologies) where work done by humans in cost centers is retooled to allow fewer people to do a better job at less cost. Think compliance, customer support, business intelligence, HR, etc.
If that ends up being the case, donating a few million dollars worth of GPU time doesn't change the larger trends, and likely ends up being useful cover as to why we shouldn't be worried about what the large companies are up to in AI because we have access to crowdsourced and donated models.
Earlier quoted context omitted.
It will be interesting to see the attitude of Microsoft towards this project in the light of their "Microsoft loves open source" propaganda.
I don't know where people got this idea that Microsoft can't participate positively in Open Source, and do that sincerely, without open sourcing absolutely everything. Of course they can - just because you contribute to open source, and do that because you also benefit from open source projects, doesn't mean you have to do absolutely everything under open source. Especially considering OpenAI isn't even Microsoft's I…