51–60 of 723 posts
The Opus model that seems to perform better than GPT4 is unfortunately much more expensive than the OpenAI model. Pricing (input/output per million tokens): GPT4-turbo: $10/$30 Claude 3 Opus: $15/$75
The Opus model that seems to perform better than GPT4 is unfortunately much more expensive than the OpenAI model. Pricing (input/output per million tokens): GPT4-turbo: $10/$30 Claude 3 Opus: $15/$75
Yeah the output pricing I think is really interesting, 150% more expensive input tokens 250% more expensive output tokens, I wonder what's behind that? That suggests the inference time is more expensive then the memory needed to load it in the first place I guess?
Probably that and what you mentioned.
The Opus model that seems to perform better than GPT4 is unfortunately much more expensive than the OpenAI model. Pricing (input/output per million tokens): GPT4-turbo: $10/$30 Claude 3 Opus: $15/$75
My point is that there’s plenty of room for high priced but only slightly better models.
I hate that they require a phone number but this might be the only way to prevent abuse so I'll have to bite the bullet. > We’ve made meaningful progress in this area: Opus, Sonnet, and Haiku are significantly less likely to refuse to answer prompts that border on the system’s guardrails than previous generations of models. Finally someone who takes this into account, Gemini and chatGPT is such an obstacle sometimes…
To the other point, yes it's crazy that "When inside kitty, how do I get my python inside latex injected into Julia? (It somehow works using alacritty?)" Despite the question being pretty underspecified or confusing, it still shouldn't read as inappropriate.
Unfortunately, many image generation systems will refuse prompts with latex in them (I assumed it was a useful term for styling).
My best guess is that it thinks latex is more often used as a clothing item or something, and it's generally associated with inappropriate content. Just unfortunate for scientists :/.
I don't put a lot of stock on evals. many of the models claiming gpt-4 like benchmark scores feel a lot worse for any of my use-cases. Anyone got any sample output? Claude isn't available in EU yet, else i'd try it myself. :(
The Opus model that seems to perform better than GPT4 is unfortunately much more expensive than the OpenAI model. Pricing (input/output per million tokens): GPT4-turbo: $10/$30 Claude 3 Opus: $15/$75
That's quite expensive indeed. At full context of 200K, that would be at least $3 per use. I would hate it if I receive a refusal as answer at that rate.
Earlier quoted context omitted.
How would that work technically, from a cost of goods sold perspective? (honestly asking, curious)
I think the answer's in the original question: the provider has to pay for extra storage to cache the model state at the prompt you're asking to snapshot. But it's not necessarily a net increase in costs for the provider, because in exchange for doing so they (as well as you) are getting to avoid many expensive inference rounds.
Does any of those LLM-as-a-service companies provide a mechanism to "save" a given input? Paying only for the state storage and the extra input when continuing the completion from the snapshot? Indeed, at 1M token and $15/M tokens, we are talking of $10+ API calls (per call) when maxing out the LLM capacity. I see plenty of use cases for such a big context, but re-paying, at every API call, to re-submit the exact sam…
How would that work technically, from a cost of goods sold perspective? (honestly asking, curious)
At this point I wonder how much of the GPT-4 advantage has been OpenAI's pre-training data advantage vs. fundamental advancements in theory or engineering. Has OpenAI mastered deep nuances others are missing? Or is their data set large enough that most test-cases are already a sub-set of their pre-training data?
More than pretraining data, I think the advantage was ChatGPT and how quickly it grew. Remember it was 3.5, and within a month or two, it generated so many actual q&a pairs with rating, feedback, and production level data of how a model will be used by actual users. Those queries and subsequent RLHF + generating better answers for the questions meant the model would have been improved a lot at the SFT stage. Think th…
But maybe that was still enough time for them to instruction tune it based on ChatGPT feedback, or at least to focus more of their fine tuning iteration in the areas they learned were strong or weak for 3.5 based on ChatGPT usage?