71–80 of 723 posts
It feels absolutely amazing to build an AI startup right now: - We struggled with limited context windows [solved] - We had issues with consistent JSON output [solved] - We had rate limiting and performance issues with 3rd party models [solved] - Hosting OSS models was a pain [solved] It's like your product becomes automatically cheaper, more reliable, and more scalable with every major LLM advancement. I'm going to…
Earlier quoted context omitted.
That's quite expensive indeed. At full context of 200K, that would be at least $3 per use. I would hate it if I receive a refusal as answer at that rate.
cost is relative. how much would it cost for a human to read and give you an answer for 200k tokens? Probably much more than $3.
Surpassing GPT4 is huge for any model, very impressive to pull off. But then again...GPT4 is a year old and OpenAI has not yet revealed their next-gen model.
Sure, OpenAI's next model would be expected to regain the lead, just due to their head start, but this level of catch-up from Anthropic is extremely impressive. Bear in mind that GPT-3 was published ("Language Models are Few-Shot Learners") in 2020, and Anthropic were only founded after that in 2021. So, with OpenAI having three generations under their belt, Anthropic came from nothing (at least in terms of models -…
Earlier quoted context omitted.
One good sign is they're only a slight improvement on knowledge recall evals but a big improvement on code and reasoning evals. Hope this stands up to scrutiny and we get something better than GPT-4 for code generation. Although the best model is a lot more expensive.
On the other hand, programmers are very expensive. At some level of accuracy and consistency (human order-of-magnitude?), the pricing of the service should start approaching the pricing of the human alternative. And first glance at numbers, LLMs are still way underpriced relative to humans.
Earlier quoted context omitted.
One good sign is they're only a slight improvement on knowledge recall evals but a big improvement on code and reasoning evals. Hope this stands up to scrutiny and we get something better than GPT-4 for code generation. Although the best model is a lot more expensive.
On the other hand, programmers are very expensive. At some level of accuracy and consistency (human order-of-magnitude?), the pricing of the service should start approaching the pricing of the human alternative. And first glance at numbers, LLMs are still way underpriced relative to humans.
It would be an ironic thing that it was open source that killed the programmer; as how would they train it otherwise?
As a scientist, should I continue to support open access journals, just so I can be trained away?
Slightly tongue in check, but not really.
Does any of those LLM-as-a-service companies provide a mechanism to "save" a given input? Paying only for the state storage and the extra input when continuing the completion from the snapshot? Indeed, at 1M token and $15/M tokens, we are talking of $10+ API calls (per call) when maxing out the LLM capacity. I see plenty of use cases for such a big context, but re-paying, at every API call, to re-submit the exact sam…
Just added Claude 3 to Chat at https://double.bot if anyone wants to try it for coding. Free for now and will push Claude 3 for autocomplete later this afternoon. From my early tests this seems like the first API alternative to GPT4. Huge!
No update on availability in European Union (still unavailable) :/
Crazy to be so ahead of the curve but sacrifice all first mover advantage in an entire continent like this.
It feels absolutely amazing to build an AI startup right now: - We struggled with limited context windows [solved] - We had issues with consistent JSON output [solved] - We had rate limiting and performance issues with 3rd party models [solved] - Hosting OSS models was a pain [solved] It's like your product becomes automatically cheaper, more reliable, and more scalable with every major LLM advancement. I'm going to…
I'd argue it's actually risky to build an AI startup now. Most any feature you bring to the table will be old news when the AI manufacturers add that to their platform.