Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3 model family

anthropic.com

Post reply on HN
Print view View original

631–640 of 723 posts

Re: Claude 3 model family

#631

Post by google234123 »

ml-anon wrote: post #629
CorpOverreach wrote: post #583

This part continues to bug me in ways that I can't seem to find the right expression for: > Previous Claude models often made unnecessary refusals that suggested a lack of contextual understanding. We’ve made meaningful progress in this area: Opus, Sonnet, and Haiku are significantly less likely to refuse to answer prompts that border on the system’s guardrails than previous generations of models. As shown below, the…

The sense of entitlement is epic. You're offended are you? Are you offended that Photoshop won't let you edit images of money too? Its not your model. You didn't spend literally billions of dollars developing it. So you can either use it according to the terms of the people who developed it (like literally any commercially available software ever) or not use it at all.

Would you be offended if Microsoft word didn’t let you write anything criticizing one political party?

Re: Claude 3 model family

#632

Post by yangcheng »

wesleyyue wrote: post #107
addandsubtract wrote: post #77

Earlier quoted context omitted.

So double is like copilot, but free? What's the catch?

No catch. We're pretty early tbh so mostly looking to get some early power users and make the product great before doing a big launch. It's been popular with yc founders in the latest batches thus far but we haven't really shared publicly. We'll charge when we launch. If you try it now, I hope you'll share anything you liked and didn't like with us!

First time saw it, would love to try, do I need to uninstall co-pilot plugin to use double?

Re: Claude 3 model family

#633

Post by sigmoid10 »

yaj54 wrote: post #519
sigmoid10 wrote: post #505

Earlier quoted context omitted.

Did you use GPT3.5 for chat? I just tried it on vanilla ChatGPT using GPT4 with no extra stuff and it immediately gets the correct answer: "Sally has 3 brothers, and each of them has 2 sisters. The description implies that Sally's brothers are her only siblings. Therefore, the two sisters each brother has must be Sally and one other sister. This means Sally has just one sister."

Weird. I tested with GPT4 Chat. I just tried again and got a differently worded incorrect answer. Interestingly my default responses are in the form " ." while it looks like your response was in the form " ". The reasoning needs to come first for it to impact the answer. I'm not sure why yours is. Have you added any custom instructions in your settings? Mine are all default.

Everything is set to default. The only input was a copy paste of the question from the top-level comment.

Re: Claude 3 model family

#634

Post by sigmoid10 »

not2b wrote: post #538
sigmoid10 wrote: post #505

Earlier quoted context omitted.

Did you use GPT3.5 for chat? I just tried it on vanilla ChatGPT using GPT4 with no extra stuff and it immediately gets the correct answer: "Sally has 3 brothers, and each of them has 2 sisters. The description implies that Sally's brothers are her only siblings. Therefore, the two sisters each brother has must be Sally and one other sister. This means Sally has just one sister."

But the second sentence is incorrect here! Sally has three siblings, one is her sister, so her brothers are not her only siblings. So ChatGPT correctly gets that Sally has one sister, but makes a mistake on the way.

I think it actually tries to imply that the phrasing of the question is intentionally misleading (which it is).

Re: Claude 3 model family

#635

Post by sigmoid10 »

uoaei wrote: post #546
sigmoid10 wrote: post #475

Earlier quoted context omitted.

We're rapidly approaching the compute capacity of the human brain in individual server racks. This "moon" is neither unreachable nor is there any doubt that we will cross the threshold soon.

I find it incredibly hard to believe we stumbled upon an efficient architecture that requires nothing but more compute not 10 years after the AI winter thawed. That's incredibly optimistic to the point of blind hope. What is your background and what makes you think we've somehow already figured everything out?

I have been working on architectures in this field for almost a decade now and I've seen firshand how things have changed. It might seem hard to believe if you have been to university ~10 years ago and only know the state of deep learning from the early revolutions back then, but we are in a totally different era now. With the transformer, we now have a true general-purpose, efficiently scalable, end-to-end differentiable algorithm. Meaning you can apply it to any task as long as you convert it to the right embedding space, you can train gigantic models that compress huge amounts of information using enormous datasets and you can still use good-ol' gradient descent to optimize it (which is kind of sad since we still haven't found a better way of training models, but hey it works).

Re: Claude 3 model family

#636

Post by chpatrick »

evantbyrne wrote: post #587
chpatrick wrote: post #579

Earlier quoted context omitted.

Your brain is undergoing some process and outputting the next word which has some reasonable statistical distribution. You're not consciously thinking about "hmm what word do I put so it's not just random gibberish" but as a whole you're doing the same thing. From my point of view as someone reading the comment I can't tell if it's written by an LLM or not, so I can't use that to conclude if you're intelligent or not…

"Your brain is undergoing some process and outputting the next word which has some reasonable statistical distribution. You're not consciously thinking about "hmm what word do I put so it's not just random gibberish" but as a whole you're doing the same thing. From my point of view as someone reading the comment I can't tell if it's written by an LLM or not, so I can't use that to conclude if you're intelligent or no…

> There is no scientific evidence that LLMs are a close approximation to the human brain in any literal sense.

I never said that, just that as a black box system that generates words it doesn't matter if it's similar or not.

Re: Claude 3 model family

#637

Post by fragmede »

eliya_confiant wrote: post #224
simonw wrote: post #189

I just released a plugin for my LLM command-line tool that adds support for the new Claude 3 models: pipx install llm llm install llm-claude-3 llm keys set claude # paste Anthropic API key here llm -m claude-3-opus '3 fun facts about pelicans' llm -m claude-3-opus '3 surprising facts about walruses' Code here: https://github.com/simonw/llm-claude-3 More on LLM: https://llm.datasette.io/

Hi Simon, Big fan of your work with the LLM tool. I have a cool use for it that I wanted to share with you (on mac). First, I created a quick action in Automator that recieves text. Then I put together this script with the help of ChaptGPT: escaped_args="" for arg in "$@"; do escaped_arg=$(printf '%s\n' "$arg" | sed "s/'/'\\\\''/g") escaped_args="$escaped_args '$escaped_arg'" done result=$(/Users/XXXX/Library/Python/…

have you tried http://openinterpreter.com? it takes that a step further

Re: Claude 3 model family

#638

Post by fragmede »

montecarl wrote: post #348
nopinsight wrote: post #283

Earlier quoted context omitted.

“Claude 3 gets ~60% accuracy on GPQA. It's hard for me to understate how hard these questions are—literal PhDs (in different domains from the questions) [spending over 30 minutes] with access to the internet get 34%. PhDs in the same domain (also with internet access!) get 65% - 75% accuracy.” — David Rein, first author of the GPQA Benchmark. I added text in […] based on the benchmark paper’s abstract. https://twitte…

I really wanted to read the questions, but they make it hard because they don't want the plaintext to be visible on the internet. Below is a link toa python script I wrote, that downloads the password protected zip and creates a decently formatted html document with all the questions and answers. Should only require python3. Pipe the output to a file of your choice. https://pastebin.com/REV5ezhv

thank you for not just posting the questions and answers. now we just have to hope that a nascent agi model can't run that script and feed it back to itself for training purposes.

Re: Claude 3 model family

#639

Post by fragmede »

torginus wrote: post #459
nopinsight wrote: post #283

Earlier quoted context omitted.

“Claude 3 gets ~60% accuracy on GPQA. It's hard for me to understate how hard these questions are—literal PhDs (in different domains from the questions) [spending over 30 minutes] with access to the internet get 34%. PhDs in the same domain (also with internet access!) get 65% - 75% accuracy.” — David Rein, first author of the GPQA Benchmark. I added text in […] based on the benchmark paper’s abstract. https://twitte…

Not sure, but I tried using GPT4 in advent of code, and it was absolutely no good.

absolutely? it got a couple of the early ones, didn't it?

Re: Claude 3 model family

#640

Post by anon373839 »

vjerancrnjak wrote: post #619
anon373839 wrote: post #557

Earlier quoted context omitted.

I don’t think all of those tools have become obsolete. NER, for example, can be performed way more efficiently with spaCy than prompting a GPT-style model, and without hallucination.

There was this assumption that for high level tasks you’ll need all of the low level preprocessing and that’s not the case. For example, machine translation attempts were morphing the parse trees , document summarization was pruning the grammar trees etc. I don’t know what your high level task is, but if it’s just collecting names then I can see how a specialized system works well. Although, the underlying model for…

Oh, right. If the high-level task is to generate a translation or summary, I think that’s been swallowed up by the Bitter Lesson (though isn’t it an open question if decoder-only models are the best fit? I’d like to see a T5 with the scale and pretraining that newer models have had).

On the other hand, people seem to be using GPT-4 for simple text classification and entity extraction tasks that even a small BERT could do well at a fraction of the cost.

Post reply on HN