Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training LLMs from ground zero as a startup

yitay.net

Post reply on HN
Print view View original

91–100 of 125 posts

Re: Training LLMs from ground zero as a startup

#91

Post by PeterStuer »

joe_the_user wrote: post #6

So essentially a startup in this context has a small number of people and a large amount of money for training clusters. The article describes many operation leasing servers - that you assume to go many startups (or existing firms). So it seems like you have the various LLM creators all doing roughly the same sort of thing (training with text and image data) with similar hardware and similar data. Each of these natur…

"this seems like a massive, energy intensive exercise in redundancy"

This is commonly refered to as a market working as intended. Yes, the waste from this type of redundency can be massive, especially if you realize that ultimately just a tiny percentage of these efforts will result in even moderate success. But it is the price to pay at the edge of progress. A planned monopoly might be more efficient (despite popular banter that just compares a megacorp or a gov, which is basically the same, to a single succesfull startup ignoring the 999 that tried and failed), but those seldom beat a market on innovation.

Re: Training LLMs from ground zero as a startup

#92

Post by TrackerFF »

Big question is, how do small startups manage to get funding for LLM products if they don’t have the “correct” background / pedigree?

The world of LLM startups is beginning to look like the world of hedge funds and private equity firms - where the prerequisites for seed/funding are:

A) Prestigious employment history / correct pedigree.

B) Solid network of investors ready to jump before any product has even begun.

Re: Training LLMs from ground zero as a startup

#93

Post by samus »

joe_the_user wrote: post #6

So essentially a startup in this context has a small number of people and a large amount of money for training clusters. The article describes many operation leasing servers - that you assume to go many startups (or existing firms). So it seems like you have the various LLM creators all doing roughly the same sort of thing (training with text and image data) with similar hardware and similar data. Each of these natur…

There are not that many of these startups actually. Most use cases of LLM can be backed with a fine-tune of an off-the-shelf foundation model. If you're training foundation models from scratch, you're entering a difficult-to-monetize market where the big boys could eat your lunch by just releasing a new foundation model that might be able to do more than 95% of what yours does.

Re: Training LLMs from ground zero as a startup

#95

Post by friendzis »

eru wrote: post #85
pests wrote: post #62

Earlier quoted context omitted.

Alignment just means making it do what you want. LLMs just continue the sequence, the chat questions and response style we have now is an example of alignment (to what humans want).

Alignment can mean making sure your LLM doesn't continue the sequence in embarrassing ways, eg by spouting politically incorrect sequences of words (even though those might have been common in the training data).

In what way does this do more good than harm?

Re: Training LLMs from ground zero as a startup

#96

Post by StarCyan »

choppaface wrote: post #47
abeppu wrote: post #4

It's worth taking a second to note that the author just assumes that readers understand "the wilderness" to mean "not Google". This post gives a lot of credit to Google's infra and hardware teams, and I'd love to read a perspective from one of those insiders who then went on to do related work elsewhere.

Really telling quote: > I was completely taken aback by the failure rate of GPUs as opposed to my experiences on TPUs at Google Should be "I was completely unaware of the failure modes of GPUs, because all my career I've been inside Google and used Google TPUs and was well-acquainted with those failure modes." I've used GPUs mostly, and when I tried TPUs the jobs failed all the time for really hard-to-debug reasons.…

When was this? I use JAX+TPUs to train LLMs and haven't experienced many issues. IMO it was way easier to set up distributed training, sharding, etc compared to Pytorch+GPUs.

Re: Training LLMs from ground zero as a startup

#97

Post by quadrature »

choppaface wrote: post #47
abeppu wrote: post #4

It's worth taking a second to note that the author just assumes that readers understand "the wilderness" to mean "not Google". This post gives a lot of credit to Google's infra and hardware teams, and I'd love to read a perspective from one of those insiders who then went on to do related work elsewhere.

Really telling quote: > I was completely taken aback by the failure rate of GPUs as opposed to my experiences on TPUs at Google Should be "I was completely unaware of the failure modes of GPUs, because all my career I've been inside Google and used Google TPUs and was well-acquainted with those failure modes." I've used GPUs mostly, and when I tried TPUs the jobs failed all the time for really hard-to-debug reasons.…

I think the OP is referring to hardware failures rather than software not playing well together.

Re: Training LLMs from ground zero as a startup

#98

Post by dotancohen »

planet_y wrote: post #66

I’m wondering if the title should read “from the ground up” instead of “ground zero”? https://en.wikipedia.org/wiki/Hypocenter

Yes, the title sounds like somebody confused two idioms. That's not the type of author from whom I want to learn.

Re: Training LLMs from ground zero as a startup

#99

Post by rvz »

Then what happens when the LLM or AI performs worse than expected? Spend more money fine tuning?

By the time you get it all working, not only you've spend lots of your VC capital on training alone, your competitors (Google, Meta, etc) already released a more powerful model much better and quicker than you before you could your run the second training epoch.

Another example of a startup incinerating the VC pump and dump scheme for vaporware AI snake-oil.

Re: Training LLMs from ground zero as a startup

#100

Post by makoto12 »

planet_y wrote: post #66

I’m wondering if the title should read “from the ground up” instead of “ground zero”? https://en.wikipedia.org/wiki/Hypocenter

could be intentional. Implying LLMs are a proverbial nuclear bomb to the tech landscape. but honestly it threw me as well
Post reply on HN