Live data from Hacker News

Claude 3 model family

anthropic.com

641–650 of 723 posts

Re: Claude 3 model family

#641
post #585

Earlier quoted context omitted.

This is a weird demand to have in my opinion. You have plenty of applications on your computer and they only do what they were designed for. You can't ask a note taking app (even if it's open soured) to do video editing, unless you modify the code.

My note taking app has never refused my input of a swear word.

...yet...

Re: Claude 3 model family

#642
post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

This is what my local LLama 13B Q5 had to say on this: > There are two ways to approach this problem. First, we can count each person by gender and then add them up. Second, we can use the given information about the siblings and their genders to find a quicker solution. > Let's go with the second method: > 1. Sally has one sister (herself). > 2. Each brother has two sisters. So, there are 3 sisters for each of her b…

Sally's parents are in for a big surprise.

Re: Claude 3 model family

#643
post #484

Earlier quoted context omitted.

This is why I doubt all the AI hype. These things are supposed to have PhD level smarts, but the above example can't reason about the problem well at all. There's a difference between PhD level information and advanced reasoning , and I'm not sure how many people can tell the difference (I'm no expert). In an adjacent area - autonomous driving - I know that lane following is f**ing easy, but lane identification and o…

> These things are supposed to have PhD level smarts Whoever told you that?

Anthropic's marketing claiming high scores on supposed intelligence measurements.

Re: Claude 3 model family

#644
post #602
post #600

Earlier quoted context omitted.

It's not about you. It's about Joe Drugdealer who wants to use it to learn how to make meth, or do other nefarious things.

Because such information isn't already readily available online, or from other drug dealers...

To be fair, the search engine monopoly has done a pretty good job of making that information quite difficult to actually find.

Not impossible, but much more difficult than you might assume.

Re: Claude 3 model family

#645

Earlier quoted context omitted.

Hi Simon, Big fan of your work with the LLM tool. I have a cool use for it that I wanted to share with you (on mac). First, I created a quick action in Automator that recieves text. Then I put together this script with the help of ChaptGPT: escaped_args="" for arg in "$@"; do escaped_arg=$(printf '%s\n' "$arg" | sed "s/'/'\\\\''/g") escaped_args="$escaped_args '$escaped_arg'" done result=$(/Users/XXXX/Library/Python/…

have you tried http://openinterpreter.com ? it takes that a step further

That is really cool, but for it to be useful you'd need:

1) Some safeguards re privacy and data ownership. Do you just send the file to the web? Do you run everything locally? 2) Can open interpreter be used with voice? So, what if I don't want to type but I want to dictate?

Re: Claude 3 model family

#646

Earlier quoted context omitted.

Mixtral is pretty good at almost a thing I’ve thrown at it. It’s still mostly worse than GPT4, but it’s so much better than any other model I can run locally. I have a niche question about modelling using some called SHACL that most models except GPT4 got right. Bard, Gemini, Llama all got it wrong. Gemini Ultra gets it right. And Mixtral also gets it right! One weakness of Mixtral for me is its support for Norwegian…

Yeah Mixtral is between GPT3.5 and GPT4 in perf. Better than 3.5, but trailing behind 4. > One weakness of Mixtral for me is its support for Norwegian. I recently added grammar correction and summarization feature to my app (which uses different system prompts based on the language). And one of the Norwegian speaking users on discord told me the same thing. He also told me that the smaller Nous-Hermes-2-SOLAR-10.7B s…

Thanks for the heads up :) I will try it out!

Re: Claude 3 model family

#647
post #364

Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…

Try this prompt instead: "Sally has 3 brothers. Each brother has 2 sisters. Give each person a name and count the number of girls in the family. How many sisters does Sally have?"

The "smart" models can figure it out if you give them enough rope, the dumb models are still hilariously wrong.

Re: Claude 3 model family

#648

This part continues to bug me in ways that I can't seem to find the right expression for: > Previous Claude models often made unnecessary refusals that suggested a lack of contextual understanding. We’ve made meaningful progress in this area: Opus, Sonnet, and Haiku are significantly less likely to refuse to answer prompts that border on the system’s guardrails than previous generations of models. As shown below, the…

You don't think that if the hammer company had a way (that cost them almost nothing) to make sure that the hammer its never used to attack human beings they wouldn't add such feature? I think many would, if anything by pressure of their local goverment or even the competition ("our hammers can't hurt your baby on accident like those other companies!") , but its impossible to add such feature to hammer; so maybe the l…

> that cost them almost nothing

Adding guardrails comes at significant expense, and not just financial, either.

Re: Claude 3 model family

#649
post #602

Earlier quoted context omitted.

Because such information isn't already readily available online, or from other drug dealers...

To be fair, the search engine monopoly has done a pretty good job of making that information quite difficult to actually find. Not impossible, but much more difficult than you might assume.

https://wikileaks.org/gifiles/attach/130/130179_Secrets_of_M...

seems to be a cookbook, but I'm no chemist. took me a couple of minutes via Google.

Re: Claude 3 model family

#650

Earlier quoted context omitted.

I'm convinced GPT is running separate helper functions on input and output tokens to fix the 'tokenization' issues. As in, find items of math, send it to this hand made parser and function, then insert result into output tokens. There's no other way to fix the token issue. For reference, Let's build the GPT Tokenizer https://www.youtube.com/watch?v=zduSFxRajkE

I'd almost say anyone not doing that is being foolish. The goal of the service is to answer complex queries correctly, not to have a pure LLM that can do it all. I think some engineers feel that if they are leaning on an old school classically programed tool to assist the LLM, it's somehow cheating or impure.

It's not cheating or impure. It's a path that is not pointing towards AGI. Heterogenous architectures are seen with contempt nowadays. Everyone told us that it is better to have a single huge model, than to specialize anything at all.
Post reply on HN