"AI features" will make search suck even more. I want my tools to be predictable, I want to be able to understand why they behave the way they do and I want to be able to customize them for my needs. This will be yet another step away from that.
Bard and new AI features in Search
771–780 of 1000 posts
Re: Bard and new AI features in Search
#772I was able to see LaMDA in action about 1.5 years ago. It was just as awe-inducing as ChatGPT. Google is not behind in this race, they're simply slower to roll out new functionality. And maybe rightly so? Some of the things I saw ChatGPT say would have been multi-day international bad headlines for Google, if a Google AI had said them instead of an upstart startup.
There is a phrase called "all publicity is good publicity" in fact. It expresses the very real truth that a media outlet reporting a bad interaction by Google AI would probably make people even more curious to try out the AI. If they get the business model nailed down, they should not be afraid of bad press. (Despite our imagination of the ways Google could profit from LLMs, I don't think Google itself has the busine…
Google got a lot of free publicity around LaMDA when they fired Blake Lemoine, but didn’t make any moves to take advantage of it in the AGI space. Now it seems OpenAI is here to take their lunch.
Re: Bard and new AI features in Search
#773Earlier quoted context omitted.
The point about the economics of running these models is an important one that slides under the radar a lot of times. The training costs for large language models like GPT are enormous, and the inference costs are substantial too. Right now things like ChatGPT are very cool parlor tricks, but there's absolutely no way to justify them in terms of the economics of running the service today. Obviously this is all going…
Inference costs are substantial, but not insurmountable, especially in the endgame. A decent chunk of the tech community can already run smaller T5 Flan models or the 6B EleutherAI LLM GPT-J (and the likely similarly sized upcoming Open Assistant) on their own machines, at decent inference speed ( What happens when models are updated every day automatically and you can run all your search and answer tasks on your loc…
It is a loosing uphill battle to reduce the operational costs as deep learning models get larger and more complex. Nvidia CEO says hardware alone will not be able to address the growing compute demand. The solution is computational optimization which is what we do.
Re: Bard and new AI features in Search
#774Earlier quoted context omitted.
There is a phrase called "all publicity is good publicity" in fact. It expresses the very real truth that a media outlet reporting a bad interaction by Google AI would probably make people even more curious to try out the AI. If they get the business model nailed down, they should not be afraid of bad press. (Despite our imagination of the ways Google could profit from LLMs, I don't think Google itself has the busine…
Google got a lot of free publicity around LaMDA when they fired Blake Lemoine, but didn’t make any moves to take advantage of it in the AGI space. Now it seems OpenAI is here to take their lunch.
LLM != AGI
Re: Bard and new AI features in Search
#775Earlier quoted context omitted.
AIUI, Netflix runs a lot of their business in AWS, but content streaming isn’t something they host in AWS.
Well they have those "last mile" boxes which they give out to ISPs to install close to the end users to improve performance (and reduce overall bandwidth needs), is this what you mean? Would love to read more about this if you have some links.
It’s called Open Connect if you want to search for it:
https://openconnect.netflix.com/
Some people who’ve worked on it post around here and they’ve funded things like FreeBSD development which is interesting for seeing the kind of problems you have at that kind of traffic volume.
Re: Bard and new AI features in Search
#776I agree this is bland corporate speak. But it reminded me of a question that's been floating around: A number of pundits, here on HN and elsewhere, keep referring to these large language models are "google killers." This just doesn't make sense to me. It feels like Google can easily pivot its ad engine to work with the AI-driven chat systems. It can augment answers with links to additional sources of information, be…
Those new language models are "Google killers" because they reset all the assumptions that people have made about search for several decades. Imagine that people start using those chat bots massively as a replacement for Google search. Then the notion of keyword disappears. Google AdSense becomes mostly irrelevant. Of course, Google is a giant today with a history of machine learning innovation. So they have a good c…
Google realized this years ago, hence Google Now (the voice interface) then Google Assistant. The problem right now is their backend isn't competitive, but Bard could change that.
Re: Bard and new AI features in Search
#777Being the first to achieve something does not make you a winner. This lesson has been taught countless times in tech from Yahoo to MySpace yet people believe OpenAI has won.
Re: Bard and new AI features in Search
#778I agree this is bland corporate speak. But it reminded me of a question that's been floating around: A number of pundits, here on HN and elsewhere, keep referring to these large language models are "google killers." This just doesn't make sense to me. It feels like Google can easily pivot its ad engine to work with the AI-driven chat systems. It can augment answers with links to additional sources of information, be…
>what am I missing? Google currently controls ~90% of the search market. AI-driven chat/search is a serious threat to this dominance. It's likely that after the market settles it won't have the same marketshare. Given how much Google has been dependent on Search/Ads and its other failures to execute, this is a serious revenue threat. This industry (and site) does have a tendency to exaggerate and take a current trend…
I’d like to think this could be good for the rest of the company if every PM was told that ad sales weren’t going to make their stock options skyrocket, so they need to build a profitable product of their own. Unfortunately, I suspect that’d mean cramming ads everywhere.
Re: Bard and new AI features in Search
#779Earlier quoted context omitted.
Those new language models are "Google killers" because they reset all the assumptions that people have made about search for several decades. Imagine that people start using those chat bots massively as a replacement for Google search. Then the notion of keyword disappears. Google AdSense becomes mostly irrelevant. Of course, Google is a giant today with a history of machine learning innovation. So they have a good c…
The point about the economics of running these models is an important one that slides under the radar a lot of times. The training costs for large language models like GPT are enormous, and the inference costs are substantial too. Right now things like ChatGPT are very cool parlor tricks, but there's absolutely no way to justify them in terms of the economics of running the service today. Obviously this is all going…
GLM-130B[1] is something comparable to GPT-3. It's a 130billion parameter model vs GPT-3's 175 billion, and it can comfortably run on current-gen high end consumer level hardware. A system with 4 RTX 3090s (The proverbial 'some guy on Twitter'[2] got it setup, and broke down the costs, demonstrated some prompts, and what not. The output's pretty terrible, but it's unclear to me whether that's inherent or a result of priority. I expect OpenAI spent a lot of manpower on supervised training, whereas this system probably had minimal, especially in English (it's from a Chinese university).
If these technologies end up as anything more than a 'novelty of the year' type event, then I expect to see them able to be run locally on phones within a decade. There will be a convergence between hardware improving and the software getting more efficient.
Re: Bard and new AI features in Search
#780Earlier quoted context omitted.
1. How much did it cost to train ChatGPT/GPT3? The only estimate I’ve seen was not enormous in the grand scheme of things (eg more money than I have but less than Google have stuck down the back of the sofa). I think that number didn’t count training precursor models or paying for people to come up with the models/training data/infra. 2. Don’t Google have specialised hardware for training neural networks? If the cost…
1. We don't know the exact cost, but its well into the millions. When Microsoft invested in OpenAI, it did so in the form of ~$500M in azure credits , so they're expecting at least that much in compute spend. Another company estimated that GPT-3 alone cost the equivalent of $4.5M in cloud costs (ignoring OpenAI's other models). 2. Yes. Yes they will/are developing custom silicon that will likely be a significant adva…
to clarify -- Microsoft invested $1bn in 2019, half of which was Azure credits. The other half is cash. Since then, they invested a further $2bn.