Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bard and new AI features in Search

blog.google

Post reply on HN
Print view View original

831–840 of 1000 posts

Re: Bard and new AI features in Search

#831

Post by cromwellian »

somenameforme wrote: post #822
cromwellian wrote: post #811

Earlier quoted context omitted.

Google currently handles 100,000 queries per second. The costs to run GLM-130B or GPT-3 at this rate would be astonishingly high.

Would they? An array of a million of these machines would cost $4 billion at consumer retail prices. That's 1.4% of Google's annual revenue for one-off bulk cost. The operational cost, at consumer retail electric using current inflated levels, was a small fraction of a cent per query. This is ignoring economy of scale, better electric pricing, caching, etc.

Where are you getting a CPU + RAM + RTX 3090 for $1k? To even install a million of these machines, you'd have to build a new datacenter, the capital costs are going to be beyond just the wholesale price of GPU boards, and you'll have to hire a ton of datacenter technicians.

But leaving that aside, look at OpenAI's pricing. $.02/1K tokens. Let's say the average query would be 20 tokens, so you'd get 50 queries/$.02 = 2500 queries/1$, or for 100k, $40/sec * 86400 * 365 = $1.2b. My guess is OpenAI's costs right now are not scaled to handle 100k QPS, so they're way underpriced for that load. This might be a cost Google could stomach.

I just think blindly shoe-horning these 100B+ param models into this use case is probably the wrong strategy, DeepMind's Chinchilla has shown it's possible to significantly reduce parameter size/cost while staying competitive in accuracy. I think Google's going to eventually get there, but they're going to do it more efficiently that brute forcing a GPT-3 style model. These very large parameter models are tech demos IMHO at this point.

Re: Bard and new AI features in Search

#832

Post by nostrademons »

acdha wrote: post #723
lmeyerov wrote: post #708

Earlier quoted context omitted.

They don't have to. Ad & search dependent companies need to answer for themselves how to handle the coming disruptions. As an analogy, Kodak was a disruption opportunity, not a problem, for apple and flickr. Yes, maybe some content dries up -- no more stock photo sites -- but entirely unclear how important and they can wait to see how zombie companies adjust. Ex: ChatGPT encourages us to put more api docs online, not…

Google has already taken a lot of heat from increasingly keeping people on the search results page rather than sending them to the content providers. Chat interfaces are going to take that problem to the next level since they not only present someone else’s content but do it without linking to them. At some point that destroys the web as sites move behind paywalls. Google or Facebook giving you less revenue is still…

That's not really a problem for the adoption or economic viability of ChatGPT, though. At some point, it hoovers up all the knowledge of the Internet, encodes it into its model, and then - the model just stagnates as content providers stop providing content. That's not a big deal for it - it'll continue to be the primary place people go for answers even when the source material has thrown in the towel and decided they don't want to play, just like how people continue to go to Google even though webspam & SEO have long since made the web a place where you don't bother to contribute.

Eventually the ecosystem might collapse when people realize they get more accurate, up-to-date information from sources other than ChatGPT. But considering that ChatGPT's answers are already more "truthy" than "truth", accuracy does not seem to be a top priority for most information-seekers.

Re: Bard and new AI features in Search

#833

Post by adt2bt »

sshnuke wrote: post #609
rvz wrote: post #87

But but but 'Google is done', 'it is all over for Google Search', 'Google will be dead in 2 years'. /s Such recent headlines and early proclamations of Google's competitiveness in AI have been erroneously under-estimated here. As soon as OpenAI announces their AI advancement and model, someone else with more money (i.e Google) scales up the parameters, training data to surpass them. The only way to make both of Googl…

Well, I see a big problem with this release. Let's imagine a user journey: User makes a request and then there's text, no ads (for now) no links to get you out of search page. So where does Adwords get triggered if there are no linked events? Then advertisers will see a drop in traffic because Google is literally giving the answer instead of making people click to search. This will accelerate the cycle of reducing ad…

There’s a big difference between a query for a fact and a query for a problem. Search engines probably make the big bucks on the latter (users with problems likely spend $ to solve them). I can see the middle man affiliate blog doing poorly because of LLM query responses, but advertisers will probably pay handsomely to have their products recommended after a user prompts the language model with a related query, or conversation.

Re: Bard and new AI features in Search

#834

Post by px43 »

13years wrote: post #130
dizzydes wrote: post #46

> Soon, you’ll see AI-powered features in Search Despite this blog obviously being written for Wall St, this line is key. This is why anyone is going to struggle to compete with Google in AI. Google Search is used so widely and its taxonomy of meaning so vast already that once they flip this on they’ll be unstoppable. (Assuming any rate of improvement whatsoever as we search.) I’m not mentioning all other learning da…

After all the buzz of decentralization, AI is going to be the new era of centralization. One AI interface to rule them all.

Hey look, a decentralized chatbot running BLOOMZ-176B (an open source LLM about the size of GPT-3)

http://chat.petals.ml

I'm contributing to the project by running a node in my garage with a single RTX 3060ti in it, and you can too: https://github.com/bigscience-workshop/petals

It's early days, but the tech is super promising.

Re: Bard and new AI features in Search

#835

Post by aardvarkr »

svnt wrote: post #825
DannyBee wrote: post #816

Earlier quoted context omitted.

+1 For a group of people on a site frequented by startup people, did nobody read the terms of MS's investment into OpenAI? "Microsoft would reportedly put down $10 billion for a 75% share of the profits that OpenAI earns until the money on the investment is paid back. Then when Microsoft breaks even on the $10 billion investment, they would get a 49% stake in OpenAI. These are not the terms you would take if, tomorro…

Not having read the terms in any detail, I will say this: it can be very easy to not report profits for a very long time.

The point was not that the profits are long tailed but rather if openai thought there were massive future profits and the potential to become a google killer then they wouldn’t give away half of their company today for $10B.

Re: Bard and new AI features in Search

#836

Post by ddalex »

DannyBee wrote: post #813
alfor wrote: post #808

Earlier quoted context omitted.

1- the chips are not efficient currently (graphic card reused as neural net) 10X-100X gain 2- Moore’s Laws 3- Algorithm/architecture improvement

1. Im not sure who you believe will produce these chips, or who will use them. You are correct that specialized inference chips will get you a 10x gain. So what? If you want several million of them in a datacenter, that's a tall order. That's on top of A. the hundreds of millions it will take to get a design to production. B. The complete and total lack of allocation at anybody who could make you chips, except at ver…

I think you missed the TPU, which is a Google chip that gets you the 10x in inference, and there are millions of them ALREADY in the datacenters, designed, fabricated and installed. You can use one, for free, with Colab

Re: Bard and new AI features in Search

#837

Post by Keyframe »

mrkramer wrote: post #565
generalizations wrote: post #419

Earlier quoted context omitted.

Anyone else remember the beginnings of youtube? Google tried to compete with them, with their own Google Videos. It sucked - and I remember reading how the engineers running it couldn't even figure out why they lost. In the end, Google just had to buy their competition, because they couldn't figure out any other way to win. They've tried to compete elsewhere, too, and I don't think they've ever been able to make a go…

Google already has a trackrecord of beating Microsoft. Do you remember Windows Phone? Well Google made Android in response to Windows Phone + Bing threat not in response to iPhone.

..and Chrome

Re: Bard and new AI features in Search

#838

Post by taylorius »

q7xvh97o2pDhNrh wrote: post #782
ElFitz wrote: post #759

Earlier quoted context omitted.

> Imagine no one actually reads web sites. Instead a blog post to share an opinion, I'll probably want to make sure my opinion gets picked up by the language model. How do I do that? Computed knowledge may render many websites and blogs obsolete. Realising that has made me wonder why I should bother write anything publicly accessible online. Aside from pure altruism and love for my fellow human, or some unexplainable…

> What do I stand to gain from it The AI will absorb your words, and some small part of you will gain immortality. In some small but very real way, you'll live forever, some part of you ensconced safely for all eternity in a handful of vectors deep inside a pile of inscrutable matrices. ...at least, until some CEO lays off that whole team to juice the stock price.

"The AI will absorb your words, and some small part of you will gain immortality. In some small but very real way, you'll live forever, some part of you ensconced safely for all eternity in a handful of vectors deep inside a pile of inscrutable matrices."

That sounds like a sort-of-religion of the future, actually.

Re: Bard and new AI features in Search

#839

Post by hansvm »

cromwellian wrote: post #831
somenameforme wrote: post #822

Earlier quoted context omitted.

Would they? An array of a million of these machines would cost $4 billion at consumer retail prices. That's 1.4% of Google's annual revenue for one-off bulk cost. The operational cost, at consumer retail electric using current inflated levels, was a small fraction of a cent per query. This is ignoring economy of scale, better electric pricing, caching, etc.

Where are you getting a CPU + RAM + RTX 3090 for $1k? To even install a million of these machines, you'd have to build a new datacenter, the capital costs are going to be beyond just the wholesale price of GPU boards, and you'll have to hire a ton of datacenter technicians. But leaving that aside, look at OpenAI's pricing. $.02/1K tokens. Let's say the average query would be 20 tokens, so you'd get 50 queries/$.02 =…

Why would cost per query go up measurably for a highly parallelizeable workload?

Re: Bard and new AI features in Search

#840

Post by hansvm »

galaxyLogic wrote: post #824
somsak2 wrote: post #780

Earlier quoted context omitted.

to clarify -- Microsoft invested $1bn in 2019, half of which was Azure credits. The other half is cash. Since then, they invested a further $2bn.

And META reportedly has spent billions on the meta-verse. It's kind of interesting that language models are now making meta-tech look outdated. Then again maybe language models will create the metaverse

Maybe we need to progress tech a bit before we try to move up a level to meta tech.
Post reply on HN