[deleted]
Claude 3 model family
231–240 of 723 posts
Re: Claude 3 model family
#232Earlier quoted context omitted.
I'm convinced GPT is running separate helper functions on input and output tokens to fix the 'tokenization' issues. As in, find items of math, send it to this hand made parser and function, then insert result into output tokens. There's no other way to fix the token issue. For reference, Let's build the GPT Tokenizer https://www.youtube.com/watch?v=zduSFxRajkE
I'd almost say anyone not doing that is being foolish. The goal of the service is to answer complex queries correctly, not to have a pure LLM that can do it all. I think some engineers feel that if they are leaning on an old school classically programed tool to assist the LLM, it's somehow cheating or impure.
> I'd almost say anyone not doing that is being foolish
The problem is, such tricks are sold as if there's superior built-in multi-modal reasoning and intelligence instead of taped up heuristics, exacerbating the already amped up hype cycle in the vacuum left behind by web3.
Re: Claude 3 model family
#233Earlier quoted context omitted.
> I see plenty of use cases for such a big context, but re-paying, at every API call, to re-submit the exact same knowledge base seems very inefficient. If you don't care about latency or can wait to set up a batch of inputs in one go there's an alternative method. I call it batch prompting and pretty much everything we do at work with gpt-4 uses this now. If people are interested I'll do a proper writeup on how to i…
I attempted similar mechanics multiple times in the past, but always ditched them, as there was always a non-negligable amount of cross-contamination happening between the individual instances you are batching. That caused so much of a headache that it wasn't really worth it.
Agreed, some problem here.
Re: Claude 3 model family
#234Does any of those LLM-as-a-service companies provide a mechanism to "save" a given input? Paying only for the state storage and the extra input when continuing the completion from the snapshot? Indeed, at 1M token and $15/M tokens, we are talking of $10+ API calls (per call) when maxing out the LLM capacity. I see plenty of use cases for such a big context, but re-paying, at every API call, to re-submit the exact sam…
FWIW the use case you're describing is very often achievable with RAG. Embedding models are deterministic, so while you're still limited by the often-nondeterministic nature of the LLM, in practice you can usually get the same answer for the same input. And it's substantially cheaper to do.
With 1M tokens, if snapshotting the LLM state is cheap, it would beat out-of-the-box nearly all RAG setups, except the ones dealing with large datasets. 1M tokens is a lot of docs.
Re: Claude 3 model family
#235Does this have 10x more censorship than the previous models? I remember v1 being quite usable.
Re: Claude 3 model family
#236They claim that the new models "are significantly less likely to refuse to answer prompts that border on the system’s guardrails than previous generations of models", looks like about a third of "incorrect refusals" compared to Claude 2.1. Given that Claude 2 was completely useless because of this, this still feels like a big limitation.
The guard rails on the models make the llm-market a complete train wreck. Wish we could just collectively grow up and accept that if a computer says something bad that doesn't have any negative real world impact - unless we let it - just like literally any other tool.
Re: Claude 3 model family
#237Earlier quoted context omitted.
100% OpenAI is Microsoft in a trenchcoat.
They are funded mostly by Microsoft, and dependent on them for compute (which is what this funding is mostly buying), but I'd hardly characterize that as meaning they are "Microsoft in a trenchcoat". It's not normal to identify startups as being their "VC in a trenchcoat", even if they are dependent on the money for growth.
Satya Nadella during the OpenAI leadership fiasco: “We have all of the rights to continue the innovation, not just to serve the product, but we can, you know, go and just do what we were doing in partnership ourselves. And so we have the people, we have the compute, we have the data, we have everything.”
Doesn’t sound like a startup-investor relationship to me!
Re: Claude 3 model family
#238Earlier quoted context omitted.
Awesome! I like the inline completions. But could you let the users choose their keyboard shortcuts before setting the default ones?
Thanks for the feedback. I was actually reworking the default shortcuts and the onboarding process when I got pre-empted by claude. I was planning to change the main actions to alt-j, alt-k to minimize conflicts. Are you asking because it conflicts with an existing shortcut on your setup? Or another reason?
Yes, it conflicts with some of my other shortcuts, but more generally, I think it'd be better to have consistent shortcuts, like CMD-CTRL-i for inline completion, CMD-CTRL-c for chat, etc.
Re: Claude 3 model family
#239Earlier quoted context omitted.
Hi, CISO of Anthropic here. Thank you for the feedback! If you can share any details about the image, please share in a private message. No LLM has had an emergent calculator yet.
Regardless of emergence, in the context of "putting safety at the frontier" I would expect Claude 3 to be augmented with very basic tools like calculators to minimize such trivial hallucinations. I say this as someone rooting for Anthropic.
LLMs are building blocks and I’m excited about folks building with a concert of models working together with subagents.
Re: Claude 3 model family
#240What is the probability that newer models are just overfitting various benchmarks? A lot of these newer models seem to underperform GPT-4 in most of my daily queries, but I'm obviously swimming in the world of anecdata.
High. The only benchmark I look at is LMSys Chatbot Arena. Lets see how it perform on that https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
We are tracking LMSys, too. There are strange safety incentives on this benchmark: you can “win” points by never blocking adult content for example.