71–78 of 78 posts
Meta builds its own models. How similar is this to a story with the headline “OpenAI limits Anthropic’s use of its ChatGPT AI models.”?
Not similar at all, as explained in the article below the headline.
Earlier quoted context omitted.
Given Meta’s current AI situation though, I wouldn’t be surprised if they were trying to do distillation and the capacity story is a cover
You can't actually "distill" reasoning from a model that doesn't expose it's genuine thinking tokens, and none of these do. When Anthropic accuse Alibaba of distilling their models, you have run that by a reality check of what is actually possible. 1) You can use another model as "LLM as judge" to rate alternative outputs that your own model has generated. Useful data perhaps, but certainly not distillation. 2) If wh…
Earlier quoted context omitted.
I don't know numbers, but their APIs have a bad uptime in my experience for some models. Too often failure because of "traffic too high".
Yeah I had a trial for AI Pro or whatever it's called and could never use Gemini CLI (when it still existed) because it was constantly "overloaded". Using the API directly (wihtout a subscription) sometimes works but the models are so buggy and the endpoints constantly spew errors that it's not usable. See this forum thread for example: https://discuss.ai.google.dev/t/frequent-503-errors-service-... it started with 5…
I use Gemini daily, on a paid basis. Mostly 3.5 Flash Medium. It’s very good for software development, has a great context window, and pricing is about 1/3rd of Claude Opus. I haven’t experienced availability problems except once or twice, certainly no more often than OpenAI or Anthropic.
Seems like the complaints about it are mostly from people moaning about Google not giving them AI for free. Good luck with that.
Must be to classify/moderate images for social media. They're pretty good at that. I can't imagine what else you'd want to use Gemini models for, certainly not coding.
Earlier quoted context omitted.
I run the CI infra for our company, and our bare metal costs (sans my salary baked in), are one order of magnitude less than if using any other CI saas provider like github or others. Like literally 10x times more expensive to do so, to run CI jobs... I dont want to imagine the margin AWS has like generally, cause it can easily be a 90% too
Their aggregate margin is typically around 30%-40% based on Amazon financials, but that reflects a lot of cost they have as a provider that you as a user wouldn't, e.g. cost of sales and the complexities of managing a multi-tenanted setup, as well as a number of huge customers that clearly are not paying list prices -anyone who is paying six figures or more at AWS and paying the publicly listed prices are overpaying…
Earlier quoted context omitted.
Yeah I had a trial for AI Pro or whatever it's called and could never use Gemini CLI (when it still existed) because it was constantly "overloaded". Using the API directly (wihtout a subscription) sometimes works but the models are so buggy and the endpoints constantly spew errors that it's not usable. See this forum thread for example: https://discuss.ai.google.dev/t/frequent-503-errors-service-... it started with 5…
You linked to a comment about someone on the free tier. Someone else mentioned preview. I use Gemini daily, on a paid basis. Mostly 3.5 Flash Medium. It’s very good for software development, has a great context window, and pricing is about 1/3rd of Claude Opus. I haven’t experienced availability problems except once or twice, certainly no more often than OpenAI or Anthropic. Seems like the complaints about it are mos…
Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?
And I'd expect Google will want to prioritize capacity for those - they don't want their google pixel phone to error or google search to barf.
Using LLMs for development is not efficient. All of the problems these companies are having trying to provide enough compute and energy are proof. Understanding the actual problems we are trying to solve with code and efficiently coming up with solutions (essentially, pre-LLM development) will always be better than wastefully brute forcing solutions with LLMs.
Counterpoint, 80% of the code I write is not hard and I don't care about it past it being close to a reference implementation