Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perplexity Labs Playground

labs.perplexity.ai

Post reply on HN
Print view View original

41–50 of 197 posts

Re: Perplexity Labs Playground

#41

Post by cchance »

paxys wrote: post #36
LeoPanthera wrote: post #23

Earlier quoted context omitted.

People asking LLMs math puzzles and then thinking a wrong answer is some kind of gotcha is getting tiresome. We know that LLMs are bad at math. It's a fundamental limitation of a neural network that thinks in words, and not in numbers. ChatGPT offers the Wolfram plugin to work around this issue, but it's not a bug, or a fault, it's just how LLMs work.

While true, I think this is still valid criticism considering so many people are quick to jump on the "AGI" bandwagon when discussing the current generation of LLMs.

No ones thinking a 7b-70b LLM is going to be an AGI lol, a 700b-1T llm likely gets pretty damn close especially with some of the newer attention concepts.

Re: Perplexity Labs Playground

#42

Post by alsodumb »

supafastcoder wrote: post #32

I asked “what is this” and it responded with: Google Lens is an application that allows users to search and identify objects, translate text, and perform various tasks using just a camera or a photo. It is available on both Android and iOS devices. Some key features of Google Lens include: Using the camera to identify objects, such as artwork, plants, or everyday items. Translating text in real-time from over 100 lan…

Google Lens is the first result you get on Google if you search “what is this”.

It seems like Google Lens team SEOed their way to the top of Google search and since Perplexity response works by using RAG with search engine content it responds with the info from the top search result plus some own context/hallucination lol.

Re: Perplexity Labs Playground

#43

Post by paxys »

cchance wrote: post #41
paxys wrote: post #36

Earlier quoted context omitted.

While true, I think this is still valid criticism considering so many people are quick to jump on the "AGI" bandwagon when discussing the current generation of LLMs.

No ones thinking a 7b-70b LLM is going to be an AGI lol, a 700b-1T llm likely gets pretty damn close especially with some of the newer attention concepts.

And yet GPT-4 with 1-2 trillion parameters still fails at the most basic math, sometimes even for tasks like adding up a set of ten numbers (hence the Wolfram comment). That's as clear evidence as any that intelligence is more than just language proficiency.

Re: Perplexity Labs Playground

#44

Post by ignoramous »

ghoomketu wrote: post #6

I recently downloaded ollama on my Linux machine and even with 3060 12gb gpu and 24 GB Ram I'm unable to run mistral or dolphin and always get an out of memory error. So it's amazing that these companies are able to scale these so well handling thousands of requests per minute. I wish they would do a behind the scenes on how much money, time, optimisation is done to make this all work. Also big fan of anyscale. Their…

> optimisation is done to make this all work

Obviously still a nascent area but https://lmsys.org/blog do a good job of diving into engineering challenges behind running these LLMs.

(I'm sure there are others)

Re: Perplexity Labs Playground

#45

Post by roughly »

SamBam wrote: post #17

Interesting. I tried it [edit: on the default model when you open it] with a physics question which ChatGPT 4 got incorrect the first time, but was able to work out with some coaxing. This got it (more) incorrect and then stuck to its guns, telling me my physics textbook was incorrect. I tried a few more questions, each copied from earlier ChatGPT conversations. In general, I was extremely impressed by its speed, it'…

> This got it (more) incorrect and then stuck to its guns, telling me my physics textbook was incorrect.

Well, it passes the Turing Test, at least

Re: Perplexity Labs Playground

#46

Post by recursive »

Workaccount2 wrote: post #26
a_imho wrote: post #24

Earlier quoted context omitted.

Tiresome for you perhaps, but this was my very first deliberate interaction with LLMs. Hardly a puzzle btw, more like basic arithmetic.

No, it's tiresome for just about everyone paying attention to LLMs.

Prepare now. You're going to see it a lot more until there's a general understanding of how these things work. I think it's going to be a while. Even here on HN, I don't think most people understand. I know I don't.

Re: Perplexity Labs Playground

#50

Post by MaximilianEmel »

a_imho wrote: post #21

Failed on the very first try, still not convinced LLM served by Perplexity Labs To solve the expression 2+2+2+2+2*1, you should follow the order of operations (PEMDAS/BODMAS), which stands for Parentheses, Exponents, Multiplication and Division, and Addition and Subtraction. Applying the order of operations to the given expression: Perform multiplication: 2 * 1 = 2 Perform addition and subtraction from left to right:…

mixtral-8x7b-instruct gets this correct:

  According to the order of operations (PEMDAS/BODMAS), you should perform multiplication and division before addition and subtraction. So for the expression 2 + 2 + 2 + 2 + 2 * 1, you should do the multiplication first: 2 * 1 = 2 Then, perform the addition: 2 + 2 + 2 + 2 + 2 = 10 So, the solution to the expression 2 + 2 + 2 + 2 + 2 * 1 is 10.
Post reply on HN