51–60 of 78 posts
Using LLMs for development is not efficient. All of the problems these companies are having trying to provide enough compute and energy are proof. Understanding the actual problems we are trying to solve with code and efficiently coming up with solutions (essentially, pre-LLM development) will always be better than wastefully brute forcing solutions with LLMs.
Earlier quoted context omitted.
Because you don't have to wait for weeks just for delivery. And you pay for elastic usage.
You can order bare metal servers delivery time in minutes from any number of hosting providers and the cost difference is so huge you can afford to keep excess capacity and still come out ahead.
Like literally 10x times more expensive to do so, to run CI jobs...
I dont want to imagine the margin AWS has like generally, cause it can easily be a 90% too
It's interesting that Meta is heavily using Google's models (as opposed to Anthropic or OpenAI) given that they are not SOTA for coding. I wonder if this for some strategic/competitive reason, or maybe for cost saving?
Google tends to be very good at vision and smaller/ edge
And their safety tuning is neither effective nor precise on edge models.
Facebook does seem to be falling behind. Does anyone here use Llama over more recent options for any technical reasons?
if you use this as a rough gauge: https://openrouter.ai/models?order=top-weekly Llama Meta 70b is 50th or so down the list of popular models. It has 24.1b tokens used in 7 days vs the top models that have trillions or hundreds of billions of tokens. So practically dead!
Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?
Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?
Earlier quoted context omitted.
You can order bare metal servers delivery time in minutes from any number of hosting providers and the cost difference is so huge you can afford to keep excess capacity and still come out ahead.
I run the CI infra for our company, and our bare metal costs (sans my salary baked in), are one order of magnitude less than if using any other CI saas provider like github or others. Like literally 10x times more expensive to do so, to run CI jobs... I dont want to imagine the margin AWS has like generally, cause it can easily be a 90% too
I assume you're using your owned server and not a provider like Hetzner? So you did have a substantial delivery time. Although in my city is a recycled that resells used servers, and I could show up there with a truck and get a server within hours if I'm not too picky. Or use some random desktop or laptop off the pile, short-term.