Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

sutro.sh

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#31

Post by achierius »

diggan wrote: post #24
ChromaticPanic wrote: post #23

Earlier quoted context omitted.

Yes some have existing infra

I'm having a somewhat hard time believing a corporation where getting a API key for a LLM service is very difficult, somehow has the (GPU) infrastructure already running for doing the same thing themselves, unless they happen to be a ML corporation, but I don't think we're talking about those in this context.

No, this is very real. One reason why this can happen: a company has elaborate processes for protecting their internal data from leaking, but otherwise lets engineers do what they want with resources allocated to them.

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#32

Post by pegasus »

diggan wrote: post #18
xfalcox wrote: post #13

Earlier quoted context omitted.

You'd be surprised how often people in enterprise can be left waiting months to get an API key approved for an LLM provider.

Are you saying that it's faster for them to get the hardware to run the weights themselves? Otherwise I'm not sure what the relevancy is.

Unless they are already in the possession of such hardware (like an M3 mac, for example).

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#33

Post by [deleted] »

diggan wrote: post #26
oooyay wrote: post #25

Earlier quoted context omitted.

Nah this is definitely a real scenario. Getting access to public models requires a lot of security review, but proving through Bedrock is much more simple. I may be spoiled in having worked for companies that have ML departments and developer XP departments though.

Not sure Bedrock counts as self-hosting though, isn't it a managed service Amazon provides? > I may be spoiled in having worked for companies that have ML Sounds likely, yeah, how many companies have ML departments today? DS departments seem common, but ML i'm not too sure about

[deleted]

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#34

Post by jbellis »

lostmsu wrote: post #30
jbellis wrote: post #21

Earlier quoted context omitted.

no, but if you're not latency sensitive you should probably be using DeepSeek v3 (cheaper than flash, significantly smarter)

What makes you believe DeepSeek is smarter than Flash 2.5? It is lower on all leaderboards.

you're right, I should clarify that I'm talking about no thinking mode, otherwise flash goes from "a bit more expensive than dsv3" to "10x more expensive"

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#35

Post by jbellis »

dinosaurdynasty wrote: post #27
jbellis wrote: post #20

This is a useful analysis, but only as a first cut and sometimes not even that -- Grok 3 mini and DeepSeek V3 are by far the least expensive coding models that are worth trying for scenarios where you do and don't care about the vendor training on your requests, respectively. One of those is "open source" (by which he seems to mean "open weights") but far too large to run locally. [I guess that must be a useful marke…

DeepSeek has a lot of competing providers that at least state they don't train on API data, OpenRouter lists a bunch of them: https://openrouter.ai/deepseek/deepseek-chat-v3-0324/provide... (This is a big advantage of open weight models; even if they're too big to host yourself, if it's worth anything there's a lot of competition for inference)

and all of them are so much more expensive than OG deepseek as to completely remove themselves from consideration

you should probably use grok 3 mini if you want "cheapest model that is reasonably good at code"

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#36

Post by fourthark »

diggan wrote: post #26
oooyay wrote: post #25

Earlier quoted context omitted.

Nah this is definitely a real scenario. Getting access to public models requires a lot of security review, but proving through Bedrock is much more simple. I may be spoiled in having worked for companies that have ML departments and developer XP departments though.

Not sure Bedrock counts as self-hosting though, isn't it a managed service Amazon provides? > I may be spoiled in having worked for companies that have ML Sounds likely, yeah, how many companies have ML departments today? DS departments seem common, but ML i'm not too sure about

A lot of companies think they do

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#37

Post by dTal »

cortesoft wrote: post #15
ramesh31 wrote: post #2

Flash is just so obscenely cheap at this point it's hard to justify the headache of self hosting though. Really only applies to sensitive data IMO.

There is a wide range of opinions on what should be considered sensitive data. Many people would classify a vast majority of their data as sensitive.

Not only that but it's a liability having two pipelines, one secure and one insecure. Apart from technical overhead, since the "insecure" pipeline is surely better/faster/cheaper/convenient (or else why have it at all), it creates a perverse incentive when classifying data as "sensitive" or not.

We already went through this with https everywhere. Previously, encryption was considered "only for sensitive data".

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#38

Post by aitchnyu »

jbellis wrote: post #35
dinosaurdynasty wrote: post #27

Earlier quoted context omitted.

DeepSeek has a lot of competing providers that at least state they don't train on API data, OpenRouter lists a bunch of them: https://openrouter.ai/deepseek/deepseek-chat-v3-0324/provide... (This is a big advantage of open weight models; even if they're too big to host yourself, if it's worth anything there's a lot of competition for inference)

and all of them are so much more expensive than OG deepseek as to completely remove themselves from consideration you should probably use grok 3 mini if you want "cheapest model that is reasonably good at code"

The above link disproves your "more expensive than OG deepdeek"

Re: Workhorse LLMs: Why Open Source Models Dominate Closed Source for Batch Tasks

#40

Post by jbellis »

aitchnyu wrote: post #38
jbellis wrote: post #35

Earlier quoted context omitted.

and all of them are so much more expensive than OG deepseek as to completely remove themselves from consideration you should probably use grok 3 mini if you want "cheapest model that is reasonably good at code"

The above link disproves your "more expensive than OG deepdeek"

Only if you can't read. DSv3 is 4x cheaper than the cheapest option in that link during US business hours.
Post reply on HN