Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Developers are choosing older AI models

augmentcode.com

Post reply on HN
Print view View original

141–150 of 179 posts

Re: Developers are choosing older AI models

#141

Post by sebastiennight »

mcny wrote: post #84
Dilettante_ wrote: post #77

Earlier quoted context omitted.

But the progress goes both ways: In five years, you would still want to use whatever is running on the cloud supercenters. Just like today you could run gpt-2 locally as a coding agent, but we want the 100x-as-powerful shiny thing.

That would be great if that was the case but my understanding is that the progress is plateauing. I don't know how much of this is anthorpic / Google / openAI holding itself back to save money and how much is the state of the art improvement slowing down though. I can imagine there could be a 64 GB GPU in five years as absurd as it feels to type that today.

> a 64 GB GPU in five years

Is there a digit missing? I don't understand why this existing in 5 years is absurd

Re: Developers are choosing older AI models

#142

Post by Workaccount2 »

Buttons840 wrote: post #128

This is how the bubble pops. I've been thinking the AI bubble wouldn't pop, because even the AI advances we've already seen can change the majority of industries if it is carefully integrated with existing technology. But if there's a mass movement to use older and/or smaller models, then yeah, all the money going into newer bigger models will pop. Or, maybe the training datasets getting polluted with AI slop will me…

It's important to remember that coding is ~5% of total LLM usage, at least with OpenAI.

50% of usage is guidance and seeking information.

Re: Developers are choosing older AI models

#144

Post by SylonZero »

Multiple models is a must, mostly due to the sometimes unpredictable variations in responses to specific situations/contexts/languages and frameworks. I find that Sonnet 4, Gemini Pro 2.5 are solid in comparison to newer models (especially Sonnet 4.5 which I find frequently to underperform). When one model is stuck in a loop, switching to a model like GPT-5 often breaks it but which model will work is subject to circumstance. P.S. I spend at least 3-4 hours a day in code-gen activities of various levels using Cursor as my primary IDE.

Re: Developers are choosing older AI models

#145

Post by unparagoned »

rho4 wrote: post #52
gptfiveslow wrote: post #5

GPT5 is HELLISHLY slow. That's all there is to it. It loves doing a whole bunch of reasoning steps and prolaim how mucf of a very good job it did clearing up its own todo steps and all that mumbo jumbo, but at the end of the day, I only asked it a small piece of information about nginx try_files that even GPT3 could answer instantly. Maybe before you make reasoning models that go on funny little sidequests wher they…

This. Speed determines whether I (like to) use a piece of software. Imagine waiting for a minute until Google spits out the first 10 results. My prediction: All AI models of the future will give an immediate result, with more and more innovation in mechanisms and UX to drill down further on request. Edit: After reading my reply I realize that this is also true for interactions with other people. I like interacting wi…

Grok fast is fast but doing a lot of stupid stuff fast actually ends up being slower

Re: Developers are choosing older AI models

#146

Post by s1mplicissimus »

Antibabelic wrote: post #75
ruszki wrote: post #63

Earlier quoted context omitted.

But… they do all the time. Almost everybody uses some mix of Office, Slack, Notion, random email providers, random “security” solutions etc. The exception is the opposite. The only thing prevents info leaking is ToS, and there are options for that even with LLMs. Nothing changed regarding that.

In my personal experience, it's very common for big companies to host email, messengers, conferencing software on their own servers.

> In my personal experience, it's very common for big companies to host email, messengers, conferencing software on their own servers.

Mind sharing a clarification on your understanding of "common" and "big"?

Re: Developers are choosing older AI models

#147

Post by s1mplicissimus »

infecto wrote: post #123
s1mplicissimus wrote: post #23

Earlier quoted context omitted.

Most people I know can't afford to leak business insider information to 3rd party SaaS providers, so it's unfortunately not really an option.

This is a poor take imo. Depends on the industry but the worlds businesses run on the shoulders of companies like Microsoft and heavily use OneDrive/Sharepoint. Most entities, even those with sensitive information are legally comfortable with that arrangement. Using a LLM does not change much so long as the MSA is similar.

> Depends on the industry but the worlds businesses run on the shoulders of companies like Microsoft and heavily use OneDrive/Sharepoint

I am sure MS employees need to tell themselves that to sleep well. The statement itself doesn't seem to hold much epistemological value above that though.

Re: Developers are choosing older AI models

#148

Post by infecto »

s1mplicissimus wrote: post #147
infecto wrote: post #123

Earlier quoted context omitted.

This is a poor take imo. Depends on the industry but the worlds businesses run on the shoulders of companies like Microsoft and heavily use OneDrive/Sharepoint. Most entities, even those with sensitive information are legally comfortable with that arrangement. Using a LLM does not change much so long as the MSA is similar.

> Depends on the industry but the worlds businesses run on the shoulders of companies like Microsoft and heavily use OneDrive/Sharepoint I am sure MS employees need to tell themselves that to sleep well. The statement itself doesn't seem to hold much epistemological value above that though.

It goes in direct conflict with your idea. I am sure you know some people within your circle that say they cannot leak data but the fact remains. Over 85% of Fortune 500 companies use some combo of OneDrive or Sharepoint. The companies have already gotten familiar with the risks and legally are comfortable with the MSAs. So I am not sure what legs you are standing on.

Absolutely there are specific companies or industries where they think the risk is too great but for many, outsourcing the process is either the same or less risk then doing it all inhouse.

Re: Developers are choosing older AI models

#149

Post by busymom0 »

darkwater wrote: post #25
busymom0 wrote: post #21

Earlier quoted context omitted.

I am currently using a local model qwen3:8b running on a 2020 (2018 intel chip) Mac mini for classifying news headlines and it's working decently well for my task. Each headline takes about 2-3 seconds but is pretty accurate. Uses about 5.3 gigs of ram.

Can you expand a bit on your software setup? I thought running local models was restricted to having expensive GPUs or latest Apple Silicon with unified memory. I have a Intel 11th gen home server which I would like to use to run some local model for tinkering if possible.

It really is a very simple setup. I basically had an old Intel based Mac mini from 2020. The intel chip inside it is from 2018). It's a 3 GHz 6-core Core i5. I had upgraded the ram on it to 32 GB when I bought it. However, the ollama only uses about 5.5 gigs of it. So it can be run on 16gb Mac too.

The Qwen model I am using is fairly small but does the job I need it to for classifying headlines pretty decently. All I ask it to do is whether a specific headline is political or not. It only responds to me with in True or False.

I access this model from an app (running locally) using the `http://localhost:11434/api/generate` REST api with `think` set to false.

Note that this qwen model is a `thinking` model. So disabling it is important. Otherwise it takes very long to respond.

Note that I tested this on my newer M4 Mac mini too and there, the performance is a LOT faster.

Also, on my new M4 Mac, I originally tried using the Apple's built in Foundation Models for this task and while it was decent, many times, it was hitting Apple's guardrails and refusing to respond because it claimed the headline was too sensitive. So I switched to the Qwen model which didn't have this problem.

Note that while this does the job I need it to, as another comment said, it won't be much help for things like coding.

Re: Developers are choosing older AI models

#150

Post by raincole »

blitzar wrote: post #18

GPT-5 usage is 20% higher on days that start with "S" Nevertheless, 7 datapoints does not a trend make (and the data presented certainly doesnt explain why). The daily variation is more than I would have expected, but could also be down to what day of the week the pizza party is or the weekly scrum meetings is at a few of their customers workplaces.

All these are relatively new models anyway. The author tried really hard to produce an article out of nothing.
Post reply on HN