Europeans, don't bother signing up - it will not work and it will only tell you once it has your e-mail registered.
Works in the UK, for anyone wondering.
Claude 3 model family
391–400 of 723 posts
Re: Claude 3 model family
#392Pricing is shown on log scale lol.
Re: Claude 3 model family
#393Earlier quoted context omitted.
I attempted similar mechanics multiple times in the past, but always ditched them, as there was always a non-negligable amount of cross-contamination happening between the individual instances you are batching. That caused so much of a headache that it wasn't really worth it.
Yeah that's definitely a risk with language models but it doesn't seem to be too bad for my use cases. Can I ask what tasks you used it for? I don't really intend for this method to be final. I'll switch everything over to finetunes at some point. But this works way better than I would have expected so I kept using it.
One thing I tried using it for was for a summarization/reformulation tasks, where it did RAG of ~3-4 smallish (~single sentence) documents per instance where each should be in the end form a coherent sentence. There, batching either caused one of the facts to slip into an adjacent instance or two instances to be merged into one.
Another thing I used it for was data extraction, where I extracted units of measurements and other key attributes out of descriptions from classifieds listings (my SO and me were looking for a cheap used couch). Non-batched it performed very well, while in the batched mode, it either mixed dimensions of multiple listings or after the summary for the initial listing it just gave nulls for all following listings.
Re: Claude 3 model family
#394Opus (and all prior Claude models) still do not correctly answer the Sally question, unfortunately: Q: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? A: Okay, let's organize the information we have: Sally is a girl. Sally has 3 brothers. Each of Sally's brothers has 2 sisters. Now, let's think about this step by step: We know that Sally is a girl, so she is a sister to he…
This is why I doubt all the AI hype. These things are supposed to have PhD level smarts, but the above example can't reason about the problem well at all. There's a difference between PhD level information and advanced reasoning , and I'm not sure how many people can tell the difference (I'm no expert). In an adjacent area - autonomous driving - I know that lane following is f**ing easy, but lane identification and o…
An AI smart enough to eclipse the average person on most basic tasks would even warrant far more hype than there is now.
Re: Claude 3 model family
#395Earlier quoted context omitted.
Temperature 1 - It answered 1 sister: https://i.imgur.com/7gI1Vc9.png Temperature 0 - it answered 0 sisters: https://i.imgur.com/iPD8Wfp.png
By virtue of increasing randomness, we got the correct answer once ... a monkey at a typewriter will also spit out the correct answer occasionally. Temperature 0 is the correct evaluation.
So your theory would have it that if you repeated the question at temp 1 it would give the wrong answer more often than the correct answer?
Re: Claude 3 model family
#396Earlier quoted context omitted.
Temperature 1 - It answered 1 sister: https://i.imgur.com/7gI1Vc9.png Temperature 0 - it answered 0 sisters: https://i.imgur.com/iPD8Wfp.png
By virtue of increasing randomness, we got the correct answer once ... a monkey at a typewriter will also spit out the correct answer occasionally. Temperature 0 is the correct evaluation.
Here's a quick analysis of the model vs it's peers:
Re: Claude 3 model family
#397Surpassing GPT4 is huge for any model, very impressive to pull off. But then again...GPT4 is a year old and OpenAI has not yet revealed their next-gen model.
Right but the people who were instrumental in the creation of GPT are now...working at Anthropic.
Re: Claude 3 model family
#398No one bashing Claude for having different names for all of its products...
Re: Claude 3 model family
#399Earlier quoted context omitted.
I was talking more about science. On the people-direction side, I expect the span of control will substantially broaden, which will probably lead to fewer manager/leader jobs (that pay more). You'll always need someone to do the last 5% that it doesn't make sense to data engineer inputs/outputs into/from AI.
Yeah. Right now, its been helping me be more productive in my science by writing code quicker...mainly on the data management side of things. I do however wonder, at what point do I just describe the hypothesis, point to the data files, and have it design an analysis pipeline, produce the results, interpret the results, then suggest potential follow-up hypotheses, do a literature search on that, then have it write up…
It'll probably be like automating most other tasks: the effort is dominated by finding the right data, transforming it into a standardized input stream, then transforming the output back into actions.
Programming became high-level programming (of compilers) became library-glueing/templating/declarative programming... becomes data engineering.
Re: Claude 3 model family
#400Earlier quoted context omitted.
This is why I doubt all the AI hype. These things are supposed to have PhD level smarts, but the above example can't reason about the problem well at all. There's a difference between PhD level information and advanced reasoning , and I'm not sure how many people can tell the difference (I'm no expert). In an adjacent area - autonomous driving - I know that lane following is f**ing easy, but lane identification and o…
An AI smart enough to eclipse the average person on most basic tasks would even warrant far more hype than there is now.
Sure, but it would also be an IA much smarter than the ones we have now, because you cannot replace a human being with the current technology. You can augment one, making her perform the job of two or more humans before for some tasks, but you cannot replace them all, because the current tech cannot reasonably be used without supervision.