Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hetzner is working on LLM Inference

sliplane.io

Post reply on HN
Print view View original

81–90 of 92 posts

Re: Hetzner is working on LLM Inference

#81

Post by atherton94027 »

idiotsecant wrote: post #77
atherton94027 wrote: post #72

Earlier quoted context omitted.

We're not even talking cutting edge. If you browse hetzner's auction page you can see they're renting machines with almost decade old CPUs like the Xeon W-2145 or i7 6700. That's just a lot of old hardware..

So? If it's cheaper and gets the job done who cares?

I mean if you run serious stuff on a non-ECC CPU from 2015, you'll probably have a bad time.

They're great for hobbyists but people on here keep bringing them up as the flagbearer of euro hosting

Re: Hetzner is working on LLM Inference

#82

Post by frizlab »

expedition32 wrote: post #78
beernet wrote: post #74

Earlier quoted context omitted.

To all mentions of IONOS / StackIT / OVH: All of these are lightyears behind US/China LLM offerings. None of them offer any model close to open-source SOTA. Runtime and reliability is a disaster and sure enough you have to pay a "sovereignty" mark up.

Freedom isn't free. Unless you want to live in a world were your children have to read the bible in class sacrifice has to be made.

I live in the EU and want to live in a world where my children actually CAN read the Bible in class. However that’s not really possible anymore…

(Also no, modern science is (still) not incompatible with (the Christian) religion. I want my children to have both, as do I, or at least be taught both, so they can decide for themselves what they want when they are grown enough to do so.)

Who is sacrificing what, really?

Re: Hetzner is working on LLM Inference

#83

Post by Yiin »

atherton94027 wrote: post #81
idiotsecant wrote: post #77

Earlier quoted context omitted.

So? If it's cheaper and gets the job done who cares?

I mean if you run serious stuff on a non-ECC CPU from 2015, you'll probably have a bad time. They're great for hobbyists but people on here keep bringing them up as the flagbearer of euro hosting

serious stuff like what? majority of people don't work on scale you probably are thinking about.

Re: Hetzner is working on LLM Inference

#84

Post by swiftcoder »

atherton94027 wrote: post #81
idiotsecant wrote: post #77

Earlier quoted context omitted.

So? If it's cheaper and gets the job done who cares?

I mean if you run serious stuff on a non-ECC CPU from 2015, you'll probably have a bad time. They're great for hobbyists but people on here keep bringing them up as the flagbearer of euro hosting

I have several production services running right now on AWS t2 instances, which were released in 2016. For a service that isn't performance-critical, it's perfectly fine (and a significant cost savings) to run on decade-old hardware.

ECC vs non-ECC is also pretty irrelevant on a box with < 64GB of memory

Re: Hetzner is working on LLM Inference

#87

Post by danlitt »

jonas_scholz wrote: post #70
danlitt wrote: post #61

Earlier quoted context omitted.

Hasn't that already happened?

yeah, but why would LLM inference now triple the price of the other products? Or am I misunderstanding what you're saying?

I am implying that they will subsidise the LLM inference product by increasing the prices of other products.

Re: Hetzner is working on LLM Inference

#88

Post by jonas_scholz »

danlitt wrote: post #87
jonas_scholz wrote: post #70

Earlier quoted context omitted.

yeah, but why would LLM inference now triple the price of the other products? Or am I misunderstanding what you're saying?

I am implying that they will subsidise the LLM inference product by increasing the prices of other products.

They wont, they are very public about never subsidising any products. I doubt they change that after 29 years now.

Re: Hetzner is working on LLM Inference

#89

Post by atherton94027 »

swiftcoder wrote: post #84
atherton94027 wrote: post #81

Earlier quoted context omitted.

I mean if you run serious stuff on a non-ECC CPU from 2015, you'll probably have a bad time. They're great for hobbyists but people on here keep bringing them up as the flagbearer of euro hosting

I have several production services running right now on AWS t2 instances, which were released in 2016. For a service that isn't performance-critical, it's perfectly fine (and a significant cost savings) to run on decade-old hardware. ECC vs non-ECC is also pretty irrelevant on a box with < 64GB of memory

A t2 instance is virtualized, AWS keeps its servers for 5 years (see their 10k https://www.sec.gov/ix?doc=/Archives/edgar/data/1018724/0001... ), so your t2 does get moved around regularly to newer servers.

And no, ECC is relevant with less than 64 gigs of ram, there's a reason it's been a standard on servers since the 90s

Re: Hetzner is working on LLM Inference

#90

Post by NetOpWibby »

satvikpendem wrote: post #56
NetOpWibby wrote: post #54

This is interesting because I thought Hetzner was anti-crypto? LLMs aren't the same but they're often lumped in with crypto as "things no one wants."

Who lumps LLMs with crypto? The former has actual usage and purpose, the latter doesn't. Anyone who does must think any new technology is all the same.

A lot of people on Mastodon, for one.
Post reply on HN