Live data from Hacker News

Perplexity Labs Playground

labs.perplexity.ai

101–110 of 197 posts

Re: Perplexity Labs Playground

#101
post #80

Earlier quoted context omitted.

I think it's understandable these models get tripped up by math problems. But if you want another way to ground lofty expectations, ask word riddles. --- "Answer this riddle: Which one of Santa's Reindeer can you see in outer space?" pplx-70b-chat: "You can see Rudolph, the red-nosed reindeer, in outer space." llama-2-70b-chat: "The answer to the riddle is "Rudolph." Rudolph is the only reindeer with a red nose, and…

It's interesting, for that last one, that the 7b model actually did better than 70b

I only included the last one (pplx-7b-online) because it's "online", it looked up the answer from riddles.com.

Re: Perplexity Labs Playground

#103

Earlier quoted context omitted.

Google Lens is the first result you get on Google if you search “what is this”. It seems like Google Lens team SEOed their way to the top of Google search and since Perplexity response works by using RAG with search engine content it responds with the info from the top search result plus some own context/hallucination lol.

"SEOed their way to the top" = rigged the search results internally at Google.

I have no idea why you're being downvoted, this was my first thought. There's no way that Google played by the same rules as everyone else to get to the top of a query like "what is this".

Re: Perplexity Labs Playground

#104
post #101

Earlier quoted context omitted.

It's interesting, for that last one, that the 7b model actually did better than 70b

I only included the last one (pplx-7b-online) because it's "online", it looked up the answer from riddles.com.

Aaah, didn't realize that!

Re: Perplexity Labs Playground

#105

I'm definitely perplexed but not in a good way. It's fast but... I asked it what it was and first it told me it was Google Lens. Then it told me some random stuff about android. I asked "But what are you though?" and got this: "A black hole is a region in space with extremely strong gravity, so powerful that not even light can escape it. They form from the remnants of large stars that die in supernova explosions. Key…

Just tried and got the same odd response. Maybe "what is this" or is a common phrase searched for that leads to Google Lens? No matter what, Perplexity is now the worst of the worst. They were early with the ability to upload documents but the utter failure of Perplexity to be useful is proving what I have been saying for a year now, (1) LLMs are not "AI" any more than a spell checker is and (2) LLMs are not really u…

I thought I was finding LLMs useful but you seem to be so sure of yourself, that I surely must have been mistaken.

Re: Perplexity Labs Playground

#106

So what is the USP of this one against the others? I tried my simple favorite one where all of these tools fail (prompt has more details): Write a script that converts git-diff output to a file that can be easily grepped by filename and linenumber. Never works. And I point then to the errors to clarify in quite a few iterations. Will never be fixed. I did not find any tool that can do that. The answer from this tool…

I am not sure how you may have gone about it but I was able to get this script, from ChatGPT4:

  #!/bin/bash
# Script to convert git diff output to a searchable format

# Check if a git repository if [ ! -d .git ]; then echo "This directory is not a git repository." exit 1 fi

# Filename for the output output_file="git_diff_searchable.txt"

# Empty the output file or create it if it doesn't exist > "$output_file"

# Process git diff output git diff --unified=0 | while read line; do # Check for filename line if [[ $line =~ ^diff ]]; then filename=$(echo $line | sed 's/diff --git a\/\(.\) b\/./\1/') elif [[ $line =~ ^@@ ]]; then # Extract line numbers line_numbers=$(echo $line | sed -E 's/@@ -[0-9]+(,[0-9]+)? \+([0-9]+)(,[0-9]+)? @@./\2/') else # Write filename and line number to the output file echo "$filename:$line_numbers: $line" >> "$output_file" fi done

echo "Output saved to $output_file"

I then ran the following egrep [corrected to egrep, after mistakenly putting that I used gawk] command egrep -e 'agent.rs:[0-9]{1,}' git_diff_searchable.txt* to see the results. Everything worked as I expected.

Now, I don't claim that this is what you intended to achieve but I prompted it with the context of what you asked: Write a script that converts git-diff output to a file that can be easily grepped by filename and linenumber.

Re: Perplexity Labs Playground

#107

So what is the USP of this one against the others? I tried my simple favorite one where all of these tools fail (prompt has more details): Write a script that converts git-diff output to a file that can be easily grepped by filename and linenumber. Never works. And I point then to the errors to clarify in quite a few iterations. Will never be fixed. I did not find any tool that can do that. The answer from this tool…

I am not sure how you may have gone about it but I was able to get this script, from ChatGPT4: #!/bin/bash # Script to convert git diff output to a searchable format # Check if a git repository if [ ! -d .git ]; then echo "This directory is not a git repository." exit 1 fi # Filename for the output output_file="git_diff_searchable.txt" # Empty the output file or create it if it doesn't exist > "$output_file" # Proces…

However, I did do some prompt "engineering" alongside using your literal request. I definitely should make it clear that I didn't only use your request verbatim but I augmented it a bit with some additional prompting cues.

Re: Perplexity Labs Playground

#108
post #74

Earlier quoted context omitted.

Ok, so what do you suggest, people paying only casual attention should know this somehow?

Don't spout off about something you're not knowledgeable about? Not trying to be rude it just seems like if you don't know how they work you shouldn't be declaring something a failure because of a poorly conceived test.

It's presented as a chat bot. How much should know about chats before we can conclude that the responses are nonsense?

Re: Perplexity Labs Playground

#109

> Q: Can you draw? > A: Yes, I can draw. > Q: can you draw me a cat? (proceeds to not draw)

I was going to try it with ASCII/Unicode art, but accidentally typed: draw an ascii with Unicode characters Imagine my surprise when it replied with: Sure! Here's an ASCII art of a cat using Unicode characters: /_/ ( o.o ) > ^ How did it know!? P.S. This was llama-2-70b-chat.

> How did it know!?

It doesn't. It's a canned response. I tried this prompt

  draw a dog with a hat in ascii art
I got the exact same ascii art:

   Sure, here's a simple ASCII art of a dog wearing a hat:
   /_/\
  ( o.o )
     ^ 
PS: OK, not exactly the same- the model swapped the ">" on the bottom with a "\" at the top.

Re: Perplexity Labs Playground

#110
post #68

Earlier quoted context omitted.

I can reasonably run (quantized) Mistral-7B on a 16GB machine without GPU, using ollama. Are you sure it isn't a configuration error or bug?

How many tokens per second and what are the specs of the machine? My attempts at CPU only have been really slow.

Seems to be around 3 tokens/s on my laptop, which is faster than average human, but not too fast of course. On a desktop with mid-range GPU used for offloading, I can get around 12 tokens/s, which is plenty fast for chatting.
Post reply on HN