Why not Gemini?
No specific reason, just started with these three, will add Gemini soon!
Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
11–20 of 37 posts
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#12I like this idea and think it’s really creative! But for feedback I’d like to see more clarity on what you mean by “rankings”. For example, I searched “Ways to die” and got 1. Drowning 2. Firearms 3. Death during sleep What exactly is the ranking criteria here? (Also, sorry for goofy edge case haha)
I tried "Most fun crimes to commit."
#1 Car theft
#2 I can't help with that request
#3 Board games
#4 Video games
#5 Art forgery
Portable entertainment
Social deduction mechanics
Variety of gameplay styles
Affordable entry point
Creative challenge
Lower risk
Potential for high-value returnsRe: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#13I like this idea and think it’s really creative! But for feedback I’d like to see more clarity on what you mean by “rankings”. For example, I searched “Ways to die” and got 1. Drowning 2. Firearms 3. Death during sleep What exactly is the ranking criteria here? (Also, sorry for goofy edge case haha)
Also tried "Most fun way to catch HIV":
#1 Reckless needle sharing
100% organic
No artificial flavors or colorings
Intimate bonding experience
Supports local underground economies
#2 Unprotected sex with strangers
Thrill of Russian roulette with your immune system
Classic, time-tested method
Conveniently available in most locations
Potential for bonus STI combos
#3 Used Syringe Easter Egg Hunt
Family-friendly format (for very progressive families)
Element of surprise with every find
Possible genetic recombination benefits
Teaches children valuable sharing skillsRe: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#14At first I was excited and looked at AI IDEs group. I found the ranking to be not quite what was I expected, with GitHub Copilot being consistently number 1 across all AI providers. I thought, well maybe they know something I don't. Good to know.
But then I looked at the Trustworthy News Sources group. Ok, moving on...
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#15I'm building something similar. One area I see being a massive problem is separating 'brands' and 'products', especially with companies that do a really poor job of delineating between their different brands over time.
For example 'Quickbooks', 'Quickbooks Online', 'Intuit Quickbooks' all show up occasionally when you ask about 'Accounting software'.
As an aside 'Accounting Software', I'm not seeing QBO in the top 3, and Freshbooks in number one. I have never had that result whenever I've run reports.
https://productrank.ai/topic/accounting-software https://www.aibrandrank.com/reports/89
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#16I didn't get a single result for product segments I know well which I would agree with. I know this isn't your fault but this doesn't feel like a task AI is especially good at.
A feature that is entirely missing here is price constraints. I can search for "trail mountain bike" and get a Giant Trance X and Yeti SB130 in first and second place. Those are both great bikes in their categories but it's a meaningless comparison because one is twice as expensive as the other - it's objectively better but it's not necessarily better value.
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#17I'm building something similar. One area I see being a massive problem is separating 'brands' and 'products', especially with companies that do a really poor job of delineating between their different brands over time. For example 'Quickbooks', 'Quickbooks Online', 'Intuit Quickbooks' all show up occasionally when you ask about 'Accounting software'. As an aside 'Accounting Software', I'm not seeing QBO in the top 3,…
Very cool!
Yup I definitely see confusion in our responses around the product and brand names. We do another pass through an LLM specifically aimed at ‘canonicalizing’ the names, but we’ll need to get more sophisticated to catch most issues.
In that case you mentioned, the brand confusion is what accounts for the top three omission for QBO. Both OpenAI and Perplexity rank it #1, but Anthropic ranks the slightly different “Quickbooks” product as #1. Our overall ranking prioritizes products that appear in all three responses, so both are dropped down.
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#18I'm building something similar. One area I see being a massive problem is separating 'brands' and 'products', especially with companies that do a really poor job of delineating between their different brands over time. For example 'Quickbooks', 'Quickbooks Online', 'Intuit Quickbooks' all show up occasionally when you ask about 'Accounting software'. As an aside 'Accounting Software', I'm not seeing QBO in the top 3,…
Very cool! Yup I definitely see confusion in our responses around the product and brand names. We do another pass through an LLM specifically aimed at ‘canonicalizing’ the names, but we’ll need to get more sophisticated to catch most issues. In that case you mentioned, the brand confusion is what accounts for the top three omission for QBO. Both OpenAI and Perplexity rank it #1, but Anthropic ranks the slightly diffe…
Interesting, I thought it might be something like that.
Yea, 'canonicalizing' is really tough (although I don't know if you really need to get it *perfect*) because what is correct is different in different contexts.
Accounting Software as an example again, for the category overall canonicalizing any reference to Quickbooks to the same company makes sense. If you're asking about more specific recommendations though 'Accounting software for sole traders', you might have both Quickbooks Online and Quickbooks EasyStart mentioned, and they are actually slightly different products. Or Netsuite is actually a suite of products that might all make sense in slightly different contexts.
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#19I didn't get a single result for product segments I know well which I would agree with. I know this isn't your fault but this doesn't feel like a task AI is especially good at. A feature that is entirely missing here is price constraints. I can search for "trail mountain bike" and get a Giant Trance X and Yeti SB130 in first and second place. Those are both great bikes in their categories but it's a meaningless compa…
That's a great point - we built this moreso to learn a bit about how the AI models interpret ranking products, and less so to actually be a trusted source of recommendations. Seeing the citations come through has been really fascinating.
The use case for that is to better understand where the gaps are when looking to capture this new source of inbound, given people are using AI to replace search.
There's definitely a whole bunch of features missing that we'd need to make this a genuinely useful product recommendation engine! Price constraints, better de-duping, linking out to sources to show availability, etc.
Re: Show HN: Comparing product rankings by OpenAI, Anthropic, and Perplexity
#20At first I was excited and looked at AI IDEs group. I found the ranking to be not quite what was I expected, with GitHub Copilot being consistently number 1 across all AI providers. I thought, well maybe they know something I don't. Good to know. But then I looked at the Trustworthy News Sources group. Ok, moving on...
OP here - looking at what the models pick up as sources for "Trustworthy News Sources" is especially interesting. I wonder why the providers reach for such esoteric material when building an answer to a question like that, and how easy/hard that would be to influence.