Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Status: Multiple engines are down

status.openai.com

Post reply on HN
Print view View original

161–167 of 167 posts

Re: OpenAI Status: Multiple engines are down

#161

Post by askiiart »

KirillPanov wrote: post #140
askiiart wrote: post #131

Earlier quoted context omitted.

Note: I assume you mean the Tesla K80. It's actually 2 GPUs on one card. But yes, it's a very good value.

I just checked ebay and they are shockingly cheap. I can't even get DDR3 memory for the price they're selling 24GB of G DDR 5 ... with a GPU thrown in for free. Why is this? Did some large cloud vendor just upgrade? Are there any deals like this on AMD hardware? Not having to deal with proprietary binary drivers is worth a lot of money and reduced performance to me. A lot.

No, AFAIK there aren't any deals like this.

These are pretty old, and all the companies are upgrading. But no one is upgrading from AMD hardware - basically no companies care if they use proprietary drivers. They want a good price-to-performance ratio, so they use NVIDIA stuff.

Plus, everyone wants CUDA.

Re: OpenAI Status: Multiple engines are down

#162

Post by int_19h »

t3rmi wrote: post #30
la64710 wrote: post #15

Earlier quoted context omitted.

This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.

Open AI's CEO Sam Altman's take is that they will only ever allow API access to their models to avoid misuse. I don't get HN's take with wanting everything open sourced. Some things are expensive to create and dangerous in the wrong hands. Not everything can and should be open sourced.

My take is that it's not good for democracy when the CEO of a private company is the one who decides what constitutes "misuse" and whose hands are "wrong" when it comes to access to a major technological breakthrough.

Re: OpenAI Status: Multiple engines are down

#163

Post by int_19h »

msm_ wrote: post #160
avindroth wrote: post #145

Earlier quoted context omitted.

I really don’t like the argument that you should make things free just because it makes the world better. What happened to ownership and respecting the effort it takes to create something? I see a similar line of reasoning can be used to justify theft from the rich.

> What happened to ownership and respecting the effort it takes to create something? Ironic taking into consideration that the current generation of AI are more or less copyright laundering for the big corporations. Github Copilot being an extreme example of using GPL projects to generate "proprietary" closed source code. What happened to ownership and respecting the effort it takes to create something?

It sounds almost like a rehashing of Locke's labor theory of property wrt ownership of land that's very popular with classical liberals and libertarians. As that goes, land is initially nobody's, but when some person applies labor to improve or develop it somehow, that labor being "mixed in" makes the whole thing the property of the laborer.

Here, instead of common land, what we have is the common content. And they're saying that, by "developing" that content into a model that can do more useful things, the authors of the model are entitled to full private property rights on it.

I really hope that's not where we're going to end up, legally speaking.

Re: OpenAI Status: Multiple engines are down

#164

Post by motoboi »

lossolo wrote: post #117
motoboi wrote: post #73

Earlier quoted context omitted.

Something like 1,5Tb memory to run this model in inference mode.

You meant 700 GB? 32 bit precision in 175B model is around ~700GB + overhead or around 350 GB if they use half-precision.

I was wrong. It seems they use fall precision. So 350gb.

What is amazing, human language (languages?) and knowledge encoded in so little space.

Re: OpenAI Status: Multiple engines are down

#165

Post by earthboundkid »

TOMDM wrote: post #80
earthboundkid wrote: post #79

Earlier quoted context omitted.

I think from Apple’s POV, this is great news. Moore’s Law has been dead for years, and there has been no good reason to upgrade your devices until now. AI means it’s 1990 again, and you need to buy a new device every 18 months because the performance leap is so meaningful to the UX.

I'd agree if Apple were in the business of making datacentre infrastructure. Nobody is running these large scale models on their personal devices. Sure, some of the image generation tech is seeing personal use, so you'd have a point there, but these immense language models are something else entirely.

Someone has to pay for that compute power. There are returns to scale and savings from sharing a machine when it would be idling, but those aren’t a factor of 10x. The AI will get in your pocket eventually and they’ll be ready to charge a premium for it.

Re: OpenAI Status: Multiple engines are down

#166

Post by earthboundkid »

TOMDM wrote: post #127
somenameforme wrote: post #113

Earlier quoted context omitted.

GLM-130B[1] (a 130 billion parameter model vs GPT-3's 175 billion parameter model) is able to run optimally on consumer level high-end hardware, 4xRTX 3090 in particular. That's It's a whole lot cheaper to run neural net style systems than to train them. "Somebody on Twitter"[2] got it setup, and broke down the costs, demonstrated some prompts, and what not. Cliff notes being a fraction of a penny per query, with eac…

Apples hardware, and 4 RTX 3090's being able to run a GPT scale model quantized to 4 byte ints are worlds apart. Who knows though, maybe someone manages to get 4byte quantization producing good results and Apple makes a chip that can do the required ops for whatever that looks like with ~100GB of memory attached and then GP's comment might be relevant. I started this reply rather skeptical, but with the boundaries Ap…

Isn’t the research now on using float8? Anyway, it will take time to move battleship, but they’re well aligned and motivated. Their UI team has gone to hell in recent years, but hardware has been strong with nothing to waste the power on. Now there is something to use it for.

Re: OpenAI Status: Multiple engines are down

#167

Post by somenameforme »

earthboundkid wrote: post #166
TOMDM wrote: post #127

Earlier quoted context omitted.

Apples hardware, and 4 RTX 3090's being able to run a GPT scale model quantized to 4 byte ints are worlds apart. Who knows though, maybe someone manages to get 4byte quantization producing good results and Apple makes a chip that can do the required ops for whatever that looks like with ~100GB of memory attached and then GP's comment might be relevant. I started this reply rather skeptical, but with the boundaries Ap…

Isn’t the research now on using float8? Anyway, it will take time to move battleship, but they’re well aligned and motivated. Their UI team has gone to hell in recent years, but hardware has been strong with nothing to waste the power on. Now there is something to use it for.

Are we not putting the cart before the horse? Where's the widespread usage for this that people would actually want? I can think of quite a lot of uses, but oddly enough most are quite negative and not widespread in any case. Media companies can use it to fire employees and increase profit margins. Hollywood can use it to rapidly churn out even more overtly derivative and generic movies. Students can use it to cheat on papers. Governments, and marketing agencies alike, can use it for endless contextually sensitive propaganda on social media.

Even in search I think it will, at most, be a sort of sidebox that says 'Super clippy says the answer to your query is [blah].' Because a single opaque source of information, which will continue to struggle with truthfulness (both inadvertently, and by design) is really just going to supplement endless dynamic content on a topic.

I'm just not seeing the big use for this (outside of the brief period of 'wow' novelty) in anything remotely like its current state.

Post reply on HN