Live data from Hacker News

AI chatbots are not a replacement for search engines

iai.tv

231–240 of 415 posts

Re: AI chatbots are not a replacement for search engines

#231
post #179

Earlier quoted context omitted.

query: "apple watch" 1st result: https://www.apple.com/watch/ Google worked perfectly for me when I wanted to buy a product.

query: “best smart watch” Pages and pages of SEO spam, generally ranked by how big & established the site pushing said spam is, rather than any unique insights contained within.

That query is generic so you get generic results. What is best is is poorly defined.

Re: AI chatbots are not a replacement for search engines

#232
post #199

Earlier quoted context omitted.

Actually it was designed for that, and it was top line OKR for a long time. Amit Signhal rewrote the search engine and was a great leader caring a lot about relevancy until he was kicked out for harassment (it was deserved as far as I know, but at the same time Google lost an amazing engineer). In my experience the detoriation came after the change in leadership.

It is important to recontextuilize harassers. They are typically parasites on the organization, destroying others by stealing their work and eliminating others from the work force. Undoubtedly he came off as clever, but being a harasser means that his achievements were outweighed by the benefits.

I think it's a big mistake to assume that if someone is toxic, they must also be stupid/incompetent/generally useless, all evidence to the contrary. If nothing else, it will make you consistently underestimate some pretty dangerous people. It's important to know your enemy, especially their strengths.

Re: AI chatbots are not a replacement for search engines

#233

Earlier quoted context omitted.

query: "apple watch" 1st result: https://www.apple.com/watch/ Google worked perfectly for me when I wanted to buy a product.

For the most trivial queries. Try something like “best frypan” and it becomes a lot harder to find real content that isn’t sponsored junk. Adding “reddit” to the query is the best option so far.

Google doesn't know the fact of what frying pan is objectively the best and websites containing such an answer are going to be biased. That's an impossible query to answer with more personalization which may not even be possible.

Re: AI chatbots are not a replacement for search engines

#234
post #22

The way I see it: - Today's search engines will give you links to millions of documents - ChatGPT, if asked correctly, will instead generate one perfect document based on millions of the documents To me, that's a clear evolution of the search engine, especially with all the SEO & ad spam that's plaguing Google and others currently. I wasn't ready to pay a monthly fee for an ad-free Google. I am ready to pay for somet…

The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts, and it shows. This model cut its teeth writing articles about the discovery of unicorns in the Andes[0] for goodness sake! It's a language model, and a very impressive one at that, but language is used to express falsehoods and fiction just as regularly as it is used to express truth. This doesn't mean that it…

Absolutely this.

I just did a discussion with ChatGPT on intuitionistic logic and it gave answers that were subtly wrong in my understanding of the subject. I have an MA in math but I couldn't be sure whether my impression of its wrongness was correct until asked it for a reference and it gave a nonexistent book by Per Martin-Lof, "Intuitionistic Logic", along with a list of actual books by him. It's an impressive and scary bullshit generator at this point.

And if someone can find a reference to Intuitionistic Logic by Per Martin-Lof, Oxford University Press, 1982, well, I'm wrong. But I went through Wikipedia, Google, Worldcat and Google Books.

Re: AI chatbots are not a replacement for search engines

#235
post #22

The way I see it: - Today's search engines will give you links to millions of documents - ChatGPT, if asked correctly, will instead generate one perfect document based on millions of the documents To me, that's a clear evolution of the search engine, especially with all the SEO & ad spam that's plaguing Google and others currently. I wasn't ready to pay a monthly fee for an ad-free Google. I am ready to pay for somet…

The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts, and it shows. This model cut its teeth writing articles about the discovery of unicorns in the Andes[0] for goodness sake! It's a language model, and a very impressive one at that, but language is used to express falsehoods and fiction just as regularly as it is used to express truth. This doesn't mean that it…

It seems that at the current velocity of improvement it is only a matter of time - very near future - that those problems will be mostly worked out. ChatGPT already outperforms Google search in many ways. For example I was doing some research on Scandinavian history and found the results far superior on ChatGPT, while I had to sift through pages of junk on Google. I took the results with a grain of salt just like I would with Wikipedia and verified each point, and found it to be very accurate. With this I can cut my research time down from hours to minutes on a problem.

Re: AI chatbots are not a replacement for search engines

#236
post #179

Earlier quoted context omitted.

query: “best smart watch” Pages and pages of SEO spam, generally ranked by how big & established the site pushing said spam is, rather than any unique insights contained within.

That query is generic so you get generic results. What is best is is poorly defined.

Back in the day you’d buy a magazine which would contain reviews of all the current years products and it would be somewhat decent. Now it’s all auto generated mass spammed garbage and affiliate links.

I guess the market responded with brand loyalty. If you have an iPhone, you just buy the Apple Watch, you don’t look further. If you have a pixel phone, you just buy whatever google sells.

Re: AI chatbots are not a replacement for search engines

#237

Earlier quoted context omitted.

The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts, and it shows. This model cut its teeth writing articles about the discovery of unicorns in the Andes[0] for goodness sake! It's a language model, and a very impressive one at that, but language is used to express falsehoods and fiction just as regularly as it is used to express truth. This doesn't mean that it…

ChatGPT in its current form can't replace google but plausible to think in its next few incarnations it will get there. Why can't it have a mode where it returns more rigorous results, and includes some system of links as citation? It can include footnotes for various passages of it's answer to indicate ambiguity, varying levels of confidence, etc.

This is the mental leap folks need to be making, the question is when will this type of configurably factual large language model be research benchmarked and productionized.

I have read many similar comments to the grandparent, all saying the same thing, “I tried it, and it turned up wrong facts” all the while they forgot that they were deeply engaged with the content it was producing. Just think of how much our attention is monetized already, money will be poring into this space from here on. It’s only a matter of time.

Re: AI chatbots are not a replacement for search engines

#238
post #22

The way I see it: - Today's search engines will give you links to millions of documents - ChatGPT, if asked correctly, will instead generate one perfect document based on millions of the documents To me, that's a clear evolution of the search engine, especially with all the SEO & ad spam that's plaguing Google and others currently. I wasn't ready to pay a monthly fee for an ad-free Google. I am ready to pay for somet…

> To me, that's a clear evolution of the search engine, especially with all the SEO & ad spam that's plaguing Google and others currently.

Clearly Google thinks so, too:

https://www.nytimes.com/2022/12/21/technology/ai-chatgpt-goo...

The podcast Hard Fork discussed the idea of searching being enhanced with this tech in their first ChatGPT episode.

Microsoft invested $1B in OpenAI. (As Hard Fork said) will we all be using Bing in 5 years?

Re: AI chatbots are not a replacement for search engines

#239
post #22

The way I see it: - Today's search engines will give you links to millions of documents - ChatGPT, if asked correctly, will instead generate one perfect document based on millions of the documents To me, that's a clear evolution of the search engine, especially with all the SEO & ad spam that's plaguing Google and others currently. I wasn't ready to pay a monthly fee for an ad-free Google. I am ready to pay for somet…

I’m assuming models will be updated in near real-time to reflect searches with results that often change unexpectedly? Like what time my physician’s office is open on Christmas Day vs. tomorrow vs. Black Friday?

Re: AI chatbots are not a replacement for search engines

#240

Earlier quoted context omitted.

The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts, and it shows. This model cut its teeth writing articles about the discovery of unicorns in the Andes[0] for goodness sake! It's a language model, and a very impressive one at that, but language is used to express falsehoods and fiction just as regularly as it is used to express truth. This doesn't mean that it…

ChatGPT in its current form can't replace google but plausible to think in its next few incarnations it will get there. Why can't it have a mode where it returns more rigorous results, and includes some system of links as citation? It can include footnotes for various passages of it's answer to indicate ambiguity, varying levels of confidence, etc.

For the same reason why it might produce a bogus answer: it's a language model, not a general AI. Prompt it for footnotes and it will generate footnotes for books that don't exist or that don't actually contain the citation. They'll match the form of a footnote, because that's what the model was trained to do, but the citations won't resolve.

For example, I asked who the first president of Argentina was. It gave the wrong name (Juan Manuel de Rosas) with a half-true bio. I then asked it for books where I can learn more about this person.

It produced five books. None of the book-author pairings were correct, though three of the books were real and four of the authors actually wrote books. Of the four authors that existed, three actually did write about Argentina's history, while one is a professor of Spanish and gender studies.

This is my point: ChatGPT was trained to produce text, not to produce facts. What I'm seeing here isn't just something that can be optimized away, it's fundamental to the model's design. It was basically told to produce text that looks like it could have been written on the internet. Its text is really convincing: when I saw the list of books I thought for a minute it had actually done it! But it's still just inventing text that reads well, because that's what its training was optimizing for.

It could be paired with a different system that searches for facts, but then it's just a friendly abstraction layer on top of Google or some other system, not a revolution in search.

Post reply on HN