Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity

productrank.ai

Post reply on HN
Print view View original

21–30 of 37 posts

Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity

#21

Post by the1024 »

crowcroft wrote: post #18
0x63_Problems wrote: post #17

Earlier quoted context omitted.

Very cool! Yup I definitely see confusion in our responses around the product and brand names. We do another pass through an LLM specifically aimed at ‘canonicalizing’ the names, but we’ll need to get more sophisticated to catch most issues. In that case you mentioned, the brand confusion is what accounts for the top three omission for QBO. Both OpenAI and Perplexity rank it #1, but Anthropic ranks the slightly diffe…

Interesting, I thought it might be something like that. Yea, 'canonicalizing' is really tough (although I don't know if you really need to get it *perfect*) because what is correct is different in different contexts. Accounting Software as an example again, for the category overall canonicalizing any reference to Quickbooks to the same company makes sense. If you're asking about more specific recommendations though '…

That nuance is really important/hard to piece apart. Have you found any good techniques to solve for it?

Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity

#23

Post by crowcroft »

the1024 wrote: post #21
crowcroft wrote: post #18

Earlier quoted context omitted.

Interesting, I thought it might be something like that. Yea, 'canonicalizing' is really tough (although I don't know if you really need to get it *perfect*) because what is correct is different in different contexts. Accounting Software as an example again, for the category overall canonicalizing any reference to Quickbooks to the same company makes sense. If you're asking about more specific recommendations though '…

That nuance is really important/hard to piece apart. Have you found any good techniques to solve for it?

To be honest not really!

I get the output from the LLMs, compile into a report, and then pass it back through an LLM to sense check the result with the added context of what's been requested in the report, but I'm not super happy with the outcome still, some different categories still come out a bit of a mess.

Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity

#24

Post by thot_experiment »

It's certainly an interesting experiment. Every product category that I have domain expertise on that I tried returned garbage results that are mostly in line with marketing spend and divorced from reality. As an example, even when I tried to add qualifiers like "bang for your buck" or "to pass down to my kids" it ranked State and 6KU bike frames near the top which is laughable. The Kilo TT didn't even make the list!

Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity

#29

Post by albertgoeswoof »

> we’re interested in seeing how AI decides to recommend products, especially now that they are actively searching the web.

So how does it work then? My naive assumption would be that it’s largely a hybrid LLM + crawled index, so still based on existing search engines that prioritise based on backlinks and a bunch of other content-based signals.

If LLMs replace search, how do marketers rank higher? More of the same? Will LLMs prioritise content generated by other LLMs or will they prefer human generated content? Who is defining the signals if not google anymore?

Vast swathes of the internet are indirectly controlled by google as people are willing to write and do anything to rank higher. What will happen to that content? Who will pull the strings?

Post reply on HN