Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bard and new AI features in Search

blog.google

Post reply on HN
Print view View original

551–560 of 1000 posts

Re: Bard and new AI features in Search

#551

Post by [deleted] »

killjoywashere wrote: post #267
wiremine wrote: post #232

I agree this is bland corporate speak. But it reminded me of a question that's been floating around: A number of pundits, here on HN and elsewhere, keep referring to these large language models are "google killers." This just doesn't make sense to me. It feels like Google can easily pivot its ad engine to work with the AI-driven chat systems. It can augment answers with links to additional sources of information, be…

I don't think most people realize how much infrastructure separates something like ChatGPT from Google-scale deployment. OpenAI isn't suddenly building transoceanic fiber and datacenters near most major population centers. They aren't signing production-ready contract vehicles with most major OEMs and governments. And in the gap it would take a new entrant to acquire 10% of those assets, Google has 100,000 engineers…

[deleted]

Re: Bard and new AI features in Search

#552

Post by mark_l_watson »

rushingcreek wrote: post #433
hn_throwaway_99 wrote: post #304

Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…

https://phind.com (YC S22) uses LLMs grounded in internet contexts to answer questions. There's an interesting tradeoff between grounding LLM answers in web contexts and answering questions from parametric memory like ChatGPT. The former is more accurate and verifiable, but the latter can be more creative and specific to you. We will have both of those answers soon. (Disclaimer: I am the co-founder).

I had not heard of phind.com either. I asked the same detailed question about myself to both perplexity.ai and phone.com, and I really liked both answers with references. Both are likely more useful to me, right now, than Google search. That said, the technique of a accepting a question, get content text with web searches, and then use LLMs like squad, etc. to answer questions - this is something the Google and Bing search will soon do also, with many of their improvements.

As a slightly disinterested third party, it will be fun the watch the competition.

Re: Bard and new AI features in Search

#553

Post by RangerScience »

pffft8888 wrote: post #530
jiggawatts wrote: post #512

Earlier quoted context omitted.

> So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. It's the opposite. Getting an LLM to learn the basics of grammar and paragraph-level language structure is the "hard" part. Once you have that, further fine-tuning, specialisation, or other incremental changes are…

It's not about "Well behaved." All we're teaching it is our biases.

Ehh. From what I've seen in a lot of places - MIRI, AI + prisoner's dilemma experiments, moral philosophy, life - there do seem to be the categories of "clear good behavior" and "clear bad behavior" even if there are also really big categories of "unclear good/bad behavior".

In other words, while sure, some "well behaved" is "passing on our biases", there does (IMO) seem to be a big chunk that's "universally well behaved".

Re: Bard and new AI features in Search

#554

Post by vitorgrs »

jw1224 wrote: post #548
vitorgrs wrote: post #525

Earlier quoted context omitted.

ChatGPT 4 seems to have newer data/practices already. At least the Bing ChatGPT integration can get data from websites and all of that, and cite the sources. Probably using REALM. Not sure if is ChatGPT-4 that would use REALM together or if is MSFT integrating both. https://medium.com/@owenyin/scoop-oh-the-things-youll-do-wit... Similar to how http://perplexity.ai/ works to get info on current data.

ChatGPT 4? I didn’t even think they’d announced v2 yet. Source?

I also though that was silly, as we are very early on 3.5 yet. https://www.windowscentral.com/software-apps/microsoft-bing-...

But the leaked Bing Chat feature indeed does show things that for now ChatGPT don't have.

Re: Bard and new AI features in Search

#555

Post by sidibe »

moufestaphio wrote: post #524
paganel wrote: post #508

Earlier quoted context omitted.

MS cannot deliver successful stuff on the web at scale. They can deliver unsuccessful stuff on the web at scale, they can deliver successful stuff that turns out to be inconsequential for their bottom line on the web in the long-term (AJAX came from MS), but it's just not in their DNA to take over the web. They had lots of chances to do it during the last 20 years or so, they had all the silver bullets at their dispo…

Uh sorry what? Azure is the second largest cloud provider, well ahead of google.. That's like the definition of 'web scale' Not sure what decade you're stuck in here with comments about ajax

Not agreeing with GP but the capacity of their public clouds is very different from total capacity. Amazon and Microsoft have bigger public clouds, but Google's own workload is probably bigger than anyone else's or even the public clouds by a large factor.

Re: Bard and new AI features in Search

#556

Post by saberience »

gretch wrote: post #509
generalizations wrote: post #419

Earlier quoted context omitted.

Anyone else remember the beginnings of youtube? Google tried to compete with them, with their own Google Videos. It sucked - and I remember reading how the engineers running it couldn't even figure out why they lost. In the end, Google just had to buy their competition, because they couldn't figure out any other way to win. They've tried to compete elsewhere, too, and I don't think they've ever been able to make a go…

Disclosure: work at google YouTube was founded in ~2005. Google bought it in 2006. It is now 2023. YouTube has spent 2 years as its own company and 17 as part of Google. Try to remember what YouTube functionality was in 2006. It was very different and has grown a lot. The narrative that Google doesn’t know how to innovate YouTube doesn’t add up.

Youtube now is 90% of what it was 10 years ago and what might it good in 2006 is the same reason it's good now, the UI is clean and it's easy to use. That's it. Also, following the acquisition, for many years most of the people working on it were the original Youtube folk, not the "Google people."

Google hasn't shown they can do new product in a very long time... see the GCP mess, Stadia, and the hundreds of other total failures (Plus, Wave, and many I've forgotten).

Re: Bard and new AI features in Search

#557

Post by mrkramer »

alain94040 wrote: post #388
wiremine wrote: post #232

I agree this is bland corporate speak. But it reminded me of a question that's been floating around: A number of pundits, here on HN and elsewhere, keep referring to these large language models are "google killers." This just doesn't make sense to me. It feels like Google can easily pivot its ad engine to work with the AI-driven chat systems. It can augment answers with links to additional sources of information, be…

Those new language models are "Google killers" because they reset all the assumptions that people have made about search for several decades. Imagine that people start using those chat bots massively as a replacement for Google search. Then the notion of keyword disappears. Google AdSense becomes mostly irrelevant. Of course, Google is a giant today with a history of machine learning innovation. So they have a good c…

>Those new language models are "Google killers" because they reset all the assumptions that people have made about search for several decades. Imagine that people start using those chat bots massively as a replacement for Google search. Then the notion of keyword disappears. Google AdSense becomes mostly irrelevant.

Look up the term "native advertising", that should help you in understanding how online ad ecosystem works.

Re: Bard and new AI features in Search

#558

Post by [deleted] »

rchaud wrote: post #398

Anybody remember how we used to learn new words and build our vocabulary? Perhaps it went something like: "What does this word mean"? "Look it up in the dictionary" "Why can't you just tell me??" I remember the hassle of looking things up in a paper dictionary. Over the long term, it massively helped improve my ability to not just recall a simple definition, but also learn new words, as they were adjacent to what I w…

[deleted]

Re: Bard and new AI features in Search

#559

Post by shawntan »

MuffinFlavored wrote: post #417
shawntan wrote: post #369

Earlier quoted context omitted.

Like the earlier responses said, if it is retrieving information from an external data source, that would be a much easier way to update things. REALM ( https://arxiv.org/abs/2002.08909 ) comes to mind as an example of active work in this area. Updating a monolithic LLM seems like a harder problem at the moment, and probably would fall under the umbrella of "continual learning", though that sub-field would have their…

> Like the earlier responses said, if it is retrieving information from an external data source I was under the impression it went through like a multi-month long ultra expensive training process involving many GPUs on terabytes of data "snapshotted" point in time. Is it possible to do this at a smaller scale once a day at the end of the day with "all new content scraped from the Internet nightly"?

How I imagine what you're thinking of is a kind of small update to an existing model by doing backpropagation training on 'new' data.

There are existing problems with that, one keyword to lookup is 'catastrophic forgetting', where you update on recent stuff so much that you overfit on that and forget all the prior stuff you learned before. There are methods to mitigate that, but I would call that an 'active area of research'.

Re: Bard and new AI features in Search

#560

Post by tarsinge »

killjoywashere wrote: post #267
wiremine wrote: post #232

I agree this is bland corporate speak. But it reminded me of a question that's been floating around: A number of pundits, here on HN and elsewhere, keep referring to these large language models are "google killers." This just doesn't make sense to me. It feels like Google can easily pivot its ad engine to work with the AI-driven chat systems. It can augment answers with links to additional sources of information, be…

I don't think most people realize how much infrastructure separates something like ChatGPT from Google-scale deployment. OpenAI isn't suddenly building transoceanic fiber and datacenters near most major population centers. They aren't signing production-ready contract vehicles with most major OEMs and governments. And in the gap it would take a new entrant to acquire 10% of those assets, Google has 100,000 engineers…

I’m sure people were saying the same things regarding IBM in the 80’s, Microsoft or Cisco in the 90’s… Google is a big corporation with inertia and averse to risk regarding its core business, they are not immune to being disrupted.
Post reply on HN