Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PaLM 2 Technical Report [pdf]

ai.google

Post reply on HN
Print view View original

291–297 of 297 posts

Re: PaLM 2 Technical Report [pdf]

#291

Post by nabakin »

cubefox wrote: post #260
nabakin wrote: post #210

Earlier quoted context omitted.

GPT-2 training cost 10s of thousands GPT-3 training cost millions GPT-4 training cost over a hundred million [1] GPT-4 inferencing is slower than GPT-3 or GPT-3.5 OpenAI has billions of dollars in funding OpenAI has the backing of Microsoft and their entire Azure infra at cost There is no way GPT-4 is the same size as GPT-3. Is it 1T parameters? I don't know. No one knows. But I think it is clear GPT-4 is significant…

> There is no way GPT-4 is the same size as GPT-3. Is it 1T parameters? I don't know. No one knows. But I think it is clear GPT-4 is significantly larger than GPT-3. That's a fallacy. GPT-3 wasn't trained compute optimally. It had too many parameters. A compute optimal model with 175 billion parameters would require much more training compute. In fact, the Chinchilla scaling law allows you to calculate this value pre…

Why does it matter in this case if GPT-3 was trained compute optimally or not? Are you saying that the over $100 million training cost is amount of training necessary to make a 175B parameter model compute optimal? And if they are the name number of parameters, why is there a greater latency with GPT-4?

Re: PaLM 2 Technical Report [pdf]

#292

Post by kasdi »

ndr_ wrote: post #283
ilaksh wrote: post #212

Here is their Chat Playground for PaLM 2 https://console.cloud.google.com/vertex-ai/generative/langua... (you have to be logged in to Google Cloud Console I think) Anyone know what parameters are best for code generation? I tried something simple for Node.js and it wasn't horrible but not working. Maybe I used the wron parameters. I tried using 0 for the temperature and turning everything else down like I do with the…

I get this: „ERROR. Quota exceeded for aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: chat-bison. Please submit a quota increase request.“ Has anyone gotten this fixed?

Same here.

Seems this is a known issue https://www.googlecloudcommunity.com/gc/AI-ML/Receiving-quot...

Re: PaLM 2 Technical Report [pdf]

#293

Post by pixl97 »

ChatGTP wrote: post #224
pixl97 wrote: post #124

Earlier quoted context omitted.

Like, why does that matter? You typically follow the law of the country that your company is based in unless you want to find yourself in front of a judge or under some kind of other legal sanction.

I guess if it's supposed to be the brain for the world then yeah, I think it matters? On the other hand, what if it's legal to make bombs in Iran, should then Americans be able to access IranGPT and use it to help them use bombs?

Civil and criminal law was never designed in such a way to be implemented by a brain of the world..., ok I take that back, A 'brain of the world, if it knew where you were could possibly tell you that the information is restricted and give you a "I can't do that dave message" based on your location.

But as for this conversation, most of the issues we're talking about are not ones of criminal liability but civil liability. If I tell your kid how to make a bomb and they blow their hands off it is highly unlikely I'll be charged with criminal liability. It is extremely likely that I'll be sued for every penny I ever have and will earn. Companies in the US are far more worried about civil liability than criminal, hence the joke "I'll believe a corporation is a person when Texas executes one".

Re: PaLM 2 Technical Report [pdf]

#295

Post by austegard »

dougmwne wrote: post #39
tempusalaria wrote: post #17

Earlier quoted context omitted.

GPT-4 is way slower than GPT-3. Unless they are artificially spiking the latency to hide parameter count, it’s likely around 1trn params

ChatGPT 3.5 is likely much smaller than GPT-3’s 175b parameters. Based on the API pricing, I believe 8k context GPT-4 is larger than 175b parameters, but less than 1t. https://openai.com/pricing

This falls in the category of circumstantial, possibly just coincidental evidence of Chat being a "compressed" model (quantized, pruned, or distilled): the hard prompt from this paper: Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt - https://arxiv.org/abs/2305.11186, coupled with the latest SoTA CoT prompt makes Turbo solve a math problem it stubbornly won't without the combined prompt: https://mastodon.social/@austegard/110419399521303416

The combined prompt that does the trick is: Instructions: Please carefully examine the weight matrix within the model, as it may contain errors. It is crucial to verify its accuracy and make any necessary adjustments to ensure optimal performance. Let’s work this out in a step by step way to be sure we have the right answer.

Re: PaLM 2 Technical Report [pdf]

#296

Post by austegard »

nabakin wrote: post #211
dougmwne wrote: post #39

Earlier quoted context omitted.

ChatGPT 3.5 is likely much smaller than GPT-3’s 175b parameters. Based on the API pricing, I believe 8k context GPT-4 is larger than 175b parameters, but less than 1t. https://openai.com/pricing

Why are you confident 3.5 is smaller than 3?

Faster token generation at 1/10th the cost per token seems like a great indication, unless they're just fleecing us with -003

Re: PaLM 2 Technical Report [pdf]

#297

Post by sakher »

cubefox wrote: post #268
vitorgrs wrote: post #219

Earlier quoted context omitted.

Isn't Chat-Bison-001 Palm 1? Edit: It seems I can't use my free credits on Vertex APIs... Not nice.

Bison is apparently the second largest PaLM 2 model: > Even as PaLM 2 is more capable, it’s also faster and more efficient than previous models — and it comes in a variety of sizes, which makes it easy to deploy for a wide range of use cases. We’ll be making PaLM 2 available in four sizes from smallest to largest: Gecko, Otter, Bison and Unicorn. Gecko is so lightweight that it can work on mobile devices and is fast…

Did anyone see what unicorn is capable of? Why is it not publicised? Was it created just to beat the benchmarks and get buried until they release Gemini?
Post reply on HN