Earlier quoted context omitted.
> other models are in the ballpark How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better, especially around code generation / understanding.
People really don't seem to understand just how far ahead OAI is in its reasoning abilities ( https://crfm.stanford.edu/helm/v0.2.2/?group=reasoning ) Maybe PaLM is near there (it's not evaluated on that page) but nothing else even comes close at all
What it feels like to work in AI right now
181–190 of 768 posts
Re: What it feels like to work in AI right now
#182Earlier quoted context omitted.
I think it's not just run cost - there's also a risk of ad revenue destruction, that has to be worrying.
I don't see ads as a big problem with ChatGPT. You could put a side-bar on the right and on the fly recommend products relevant to the on-going conversation. The cost of computing these ads would be a lot more than today's keyword-based approach, that's certainly a problem. But think of hyper-relevant ads, based on the chat itself. There's a lot of information there, that beats tracking people's behavior online all d…
Re: What it feels like to work in AI right now
#183From my perspective it’s just _confusing_ to work in AI right now. We have some massive models that are doing some really neat stuff, and apparently hundreds of millions of people are using them—but I keep wondering: to do _what_, exactly? I’m not asking what the models can do, I’m asking what people want the models to do every day, all the time. I’ve been shown some neat pictures people made that they thought were c…
I've used this part of ChatGPT before. Incredibly useful for getting the syntax of some library that you're going to use once in your life, then never again.
Had a sysadmin mate do something similar to generate a simple Chrome plugin for internal use at his work.
That alone justifies the price for ChatGPT Plus, IMO.
Re: What it feels like to work in AI right now
#184I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.
ChatGPT is cool and novel, but FAANG's requirements for ML/AI go far beyond what ChatGPT provides as a product. ChatGPT is good at answering questions based on an older data set. FAANG typically requires up to date real time inference for huge rapidly changing data sets. Working on the practical side of ML/AI at FAANG, you will probably be working with some combination of feature stores, training platforms, inference…
For the time being, I expect LLMs to start creeping their tendrils into various workflows where the underlying engineering work is light but the rate of this will be limited by the slow adaptability of the humans that are not yet completely disposable. The "low hanging fruit" is obvious, but EVPs who are asking "why can't we just replace our whole web experience with a chatbot interface?" may end up causing weird overcorrections among their subordinates.
Re: What it feels like to work in AI right now
#185I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.
ChatGPT is cool and novel, but FAANG's requirements for ML/AI go far beyond what ChatGPT provides as a product. ChatGPT is good at answering questions based on an older data set. FAANG typically requires up to date real time inference for huge rapidly changing data sets. Working on the practical side of ML/AI at FAANG, you will probably be working with some combination of feature stores, training platforms, inference…
Re: What it feels like to work in AI right now
#186Earlier quoted context omitted.
Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?
I dont work at Google, I think other FAANGS underinvested in this area as they didnt think it was promising. But I will admit, I am suspicious that Google is incompetent. Probably they can come back given how much money they will be forced to throw at it. But Bard is clearly behind and I dont believe their "abundance of caution" arguments for why Bard cant even code.
Google is run by smart people whose mission is to maximize clicks on ads. If a user finds what they’re looking for quickly, that’s lost revenue.
Google’s profit motives are not aligned with useful AI. The better AI is, the less people need to click through to lots of web pages and ads, the less revenue for Google.
I don’t think they can catch up without a major pivot in business model. It’s very hard to be deeply invested in providing more value if it means reducing your revenue.
Re: What it feels like to work in AI right now
#187I feel like the whole ChatGPT bubble has really highlighted what feel like some fundamental shortcomings in the worldview that is represented in tech. That is, there seems to be a winner take all dynamic baked into the tech world. Maybe this arises from the simple fact that tech places a lot of power in the hands of individuals. But there's an emergent downside to this which is that it makes those who were already mu…
Pre-tech, the winner take all dynamic was prevented by physics: - Customers were only willing to walk so far to go to a bakery - A store could only stock so many varieties on it's shelves Now, the cost of distributing goods across the globe, and the incremental cost of creating new goods has shrunk to virtually zero for digital goods, combined with people's general hesitance to the friction of switching providers, cr…
Without this the gravity is much less powerful. Seems like where to focus if you want to empower people against the winner take all monopoly effect.
Areas where switching costs are low are areas where you have standard APIs and where your data resides locally, is easy to move, or are not data intensive at all.
Re: What it feels like to work in AI right now
#188As much as it pains me to say this, I don't think the real money is in making this a service, or "the product." I think the real money is in using AI internally as a puzzle piece of your backend - ie. the secret sauce behind xyz product.
I'm being very narrow here, but you can only do so much integrating what openai has built into your products - eventually "everything" providing data from the same model brings "everything" to the same level. In contrast if you train and create your own models to make xyz do something specific, nobody knows how it was done, or it surely makes it a lot harder to kang.
I have zero proof, but I suspect Google for instance has models that would literally obliterate what openai has shown capability wise. They're probably not necessarily language models though. Again, nothing to stand on here but I doubt their search and analytics for example are driven by hard coded algorithms these days.
Bard may have been released sort of as a "psh, we've been there done that" when in reality they didn't, because they never planned to make the models they were/are working on "publicly" available to use. It makes me wonder if this is how Google has lead for some long with some areas - now openai sort of screwed it up for everyone by making it a service that can be integrated / adopted by nearly anyone.
The only people I guess that are really going to know are the devs working for these big orgs, and I'm sure that lock and key knowledge.
Re: What it feels like to work in AI right now
#189Earlier quoted context omitted.
100% this -- a Google search requires multiple input queries, refinement, and scrolling a list of possible answers that are really just links to other web sites. The ChatGPT experience is far superior to this, for the average consumer and getting close for the power user. It's a better way to ask the Internet what it knows with a more natural interface that everyone already knows how to use -- real natural language.…
> The ChatGPT experience is far superior Now it is. Google used to be good too, until ads started looking like search results, and then the first page became entirely ads. In the future, when you ask ChatGPT to help you write your resume, it will try to upsell you a premium account in linked in. It will withhold its best resume advice only for LinkedIn premium users after all. You think Clips was bad? You’ve seen not…
Re: What it feels like to work in AI right now
#190I'm probably way too late for this thought to get any traction / discussion - but I have this weird feeling that openai screwed up and showed it's "cool new thing" too early, and publicly. As much as it pains me to say this, I don't think the real money is in making this a service, or "the product." I think the real money is in using AI internally as a puzzle piece of your backend - ie. the secret sauce behind xyz pr…
Then why is Bard so bad? Bard feels like GPT-2 or LLaMA 7B with no finetuning most of the time (I tried it two or three times over the course of a week and went back to ChatGPT)