Live data from Hacker News

PaLM 2 Technical Report [pdf]

ai.google

271–280 of 297 posts

Re: PaLM 2 Technical Report [pdf]

#271
post #245

Earlier quoted context omitted.

I'm sorry, this is nonsense. Technical reports exist to fill in information that is useful for readers but not necessary to understand the key contributions of the work, and/or that don't fit within the journal or conference's page limit. I'm not sure where you got the idea that it is something people do to avoid competitive baselines; IME, the peer-reviewed portion of the publication is far more likely to contain mi…

I agree that's what TR's are for. However, my point is, if you want to publish academic writing without peer review, a TR is a way to go about that. You can also just publish a preprint somewhere, which - surprise surprise - is also common for these same actors. I should have worded it better, in hindsight.

I get what you're saying, I just think this is more of a Google thing than a TR thing. Their peer reviewed papers have the same issue as their preprints, TRs, and whitepapers, generally speaking--Google researchers feel no incentive to actually share how they did things, perform accurate or up-to-date comparisons to comparable frameworks, or even bother outlining their key contributions, because they know the paper will be published, widely read, widely cited, and influential even if they don't do any of those things. It's to the point that I think it might actually be house policy to neuter their papers of specific details as much as possible, presumably to retain what they perceive as Google's competitive advantage, because it makes no sense otherwise that wildly different papers with different authorship groups coming from so many different areas of CS could all have these same problems.

This is (IMO) quite different from, e.g., the cases of academics publishing misleading benchmarks, which is more often just being wedded to a bad idea because you spent years of work on it and your position is at risk if you didn't end up outperforming existing approaches. Often I can still get a lot out of papers with misleading benchmarks, even if what I get is "don't try this technique, it doesn't work." Whereas I frequently get nothing at all out of Google publications. If I had to describe the way Google seems to view academic publishing in one word, it would be "marketing"--it's advertising for people to either come work at Google or use their products, not something written with the intent of advancing the wider state of the art, or even the less noble goal justifying the time and money they put into whatever they're writing about.

Re: PaLM 2 Technical Report [pdf]

#272
post #23

So how do we actually try out the PaLM 2? The links in their press release just link to their other press release, and if I google "PaLM API" it just gives me more press release, but I just couldn't find the actual document for their PaLM API. How do I actually google the "PaLM API" for a way to test "PaLM 2"?

In Google Cloud Vertex AI, you can play with it straight away

Re: PaLM 2 Technical Report [pdf]

#274

Earlier quoted context omitted.

Btw I've arrived at a different interpretation of the "Open" in OpenAI. It's open in the sense that the generic LLM is exposed via an API, allowing companies to build anything they want on top. Companies like Google have been working on language models (and AI more broadly) for years but have hid the generic intelligence of their models, exposing it only via improvements to their products. OpenAI bucked this trend an…

> Companies like Google have been working on language models (and AI more broadly) for years but have hid the generic intelligence of their models, exposing it only via improvements to their products. OpenAI bucked this trend and exposed an API to generic LLMs. But there is a PaLM API: https://developers.generativeai.google/ Of course this is a reaction to the OpenAI API.

Yeah, Google hopped on the bandwagon. Glad to see a PaLM API.

Re: PaLM 2 Technical Report [pdf]

#275
post #204

Earlier quoted context omitted.

Btw I've arrived at a different interpretation of the "Open" in OpenAI. It's open in the sense that the generic LLM is exposed via an API, allowing companies to build anything they want on top. Companies like Google have been working on language models (and AI more broadly) for years but have hid the generic intelligence of their models, exposing it only via improvements to their products. OpenAI bucked this trend an…

You don't need to "arrive" at an interpretation. Their name is legacy from when they were "open".

That's true. My thought was they're still 'open', in an important way, even though it's not the open source way. If they were smart they'd adopt my interpretation in their PR materials.

Re: PaLM 2 Technical Report [pdf]

#276
post #165

Earlier quoted context omitted.

Btw I've arrived at a different interpretation of the "Open" in OpenAI. It's open in the sense that the generic LLM is exposed via an API, allowing companies to build anything they want on top. Companies like Google have been working on language models (and AI more broadly) for years but have hid the generic intelligence of their models, exposing it only via improvements to their products. OpenAI bucked this trend an…

Sounds like the famous Facebook hoodie. "Open and connected" was one slogan on it. The API can be shut down at any time. https://venturebeat.com/social/facebook-insignia-hoodie/ In the end they just shat all over RSS etc.

This is an important point, although I think incentives don't align at all for OpenAI to close their APIs.

Re: PaLM 2 Technical Report [pdf]

#277
You know you're old when you see a title like this and mistakenly assume that it's related to the Palm Pilot ;)

Why do we have AI things named palm and lora when we already have other things named that? Also, get off of my lawn. /s

Re: PaLM 2 Technical Report [pdf]

#278
Bard still not using PaLM 2. Using it from India

Question: are you on PaLM 2 right now?

Answer: No, I am not on PaLM 2 right now. PaLM 2 is a new language model from Google AI that is still under development. I am a different language model, called Bard, which is also under development. I am trained on a massive dataset of text and code, and I can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way. I am still under development, but I have learned to perform many kinds of tasks.

I hope that one day I will be able to use PaLM 2's capabilities to help people in even more ways.

Re: PaLM 2 Technical Report [pdf]

#279

Earlier quoted context omitted.

1. there's no reason to think OpenAI wouldn't also be going the artificial scarcity route as have so many other companies in the past 2. Microsoft may not like them using too much azure compute and tell them to step off. Rumor has it they're trying to migrate github to it and it's seemingly not going ideal. And they're certainly nothing more than another microsoft purchase at this point.

OpenAI has a 40k token per minute rate limit on their GPT4 API too so I doubt it's artificial scarcity.

Perhaps. I found it was far too easy to hit the API limit with their old codex models, though that may have been limited to a small GPU cluster given it was pretty obscure compared to chatgpt and even davinci.

Re: PaLM 2 Technical Report [pdf]

#280

Earlier quoted context omitted.

You can use gpt-4 for free (toggle "Use best model"), and it'll search the internet and state sources on https://phind.com No idea when they'll start charging, but it's replaced a lot of my googling at work

You can use GPT-4 for free with Bing.

Bing seems dumber than the free tier on OpenAI's chat (I believe it's GPT3.5?). It constantly just falls back to some search results I don't want

I don't even bother using it

Post reply on HN