Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaCode Attention Visualization

alphacode.deepmind.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: AlphaCode Attention Visualization

#31

Post by lossolo »

typon wrote: post #28
lossolo wrote: post #18

Earlier quoted context omitted.

> Has Google/Alphabet publicly released any of their AI models yet? You mean NLP field changing models from Google like BERT [1]? or Transformers paper [2]? or T5 model [3] (used by company doing ChatGPT like search currently on the front page on HN)? 1. https://arxiv.org/abs/1810.04805 code+models: https://github.com/google-research/bert 2. https://arxiv.org/abs/2112.04426 3. https://arxiv.org/abs/1910.10683 code+mo…

I read a lot on Twitter about the so-called "culture" of Google that prevents them from making AI based products, meanwhile OpenAI has made ChatGPT and is going to replace Google search within the next two months or so. I think this is the same narrative that's being expressed by the GP comment. To add to your comment, Google has been using BERT to power Search since 2019: https://blog.google/products/search/search-l…

> if Google offered Search "Premium" using the latest LLMs, how much would people pay for it?

I think they need to solve hallucination problem first, they are already working on optimization eg FLAN-T5 (smaller LLM models same performance) and RETRO (retrieval transformer that can use data index outside of the model) that takes them closer to use it in search.

Re: AlphaCode Attention Visualization

#32

Post by lossolo »

dougmwne wrote: post #25
lossolo wrote: post #18

Earlier quoted context omitted.

> Has Google/Alphabet publicly released any of their AI models yet? You mean NLP field changing models from Google like BERT [1]? or Transformers paper [2]? or T5 model [3] (used by company doing ChatGPT like search currently on the front page on HN)? 1. https://arxiv.org/abs/1810.04805 code+models: https://github.com/google-research/bert 2. https://arxiv.org/abs/2112.04426 3. https://arxiv.org/abs/1910.10683 code+mo…

Right, so that’s a no. Google releases research papers but can’t productize anything. They are like a modern day Xerox Parc.

> Google releases research papers but can’t productize anything

If you mean actual products and not just open sourcing models and code then:

https://cloud.google.com/products/ai

They also implement a lot (the most interesting stuff) of what they publish inside google to power their own products.

Re: AlphaCode Attention Visualization

#33

Post by anu7df »

ldjkfkdsjnv wrote: post #30
jw1224 wrote: post #7

Has Google/Alphabet publicly released any of their AI models yet? I’ve seen plenty of hype surrounding Imagen [1] and Parti [2], but as far as I know, they’re still vaporware. Normally I wouldn’t think too hard about this, but two things come to mind here: Firstly, the speed at which competitors are launching and developing new AI models. Imagen/Parti both seem to rival Stable Diffusion and DALL-E… why can’t we use t…

I cannot wait for the true downfall of Google. Many friends I have had began working there and fell into a blackhole of arrogance. Meanwhile, nothing from a technical perspective, aside from BERT, has been contributed by them in quite a while. Their technical open source (Tensorflow, Angular, Kubernetes) has all followed the same pattern of overly complex garbage. Facebook opensource (Pytorch, React) blows the doors…

I guess it depends on your point of view. Deepmind as far as I am concerned is the one true success that Alphabet is funding. The work they do is truly cutting edge. They may not be good at making products out of their breakthrough research, but you can't argue they are the "beens" on the basis of that.

Re: AlphaCode Attention Visualization

#34

Post by ldjkfkdsjnv »

anu7df wrote: post #33
ldjkfkdsjnv wrote: post #30

Earlier quoted context omitted.

I cannot wait for the true downfall of Google. Many friends I have had began working there and fell into a blackhole of arrogance. Meanwhile, nothing from a technical perspective, aside from BERT, has been contributed by them in quite a while. Their technical open source (Tensorflow, Angular, Kubernetes) has all followed the same pattern of overly complex garbage. Facebook opensource (Pytorch, React) blows the doors…

I guess it depends on your point of view. Deepmind as far as I am concerned is the one true success that Alphabet is funding. The work they do is truly cutting edge. They may not be good at making products out of their breakthrough research, but you can't argue they are the "beens" on the basis of that.

At some point in time they need to either release the models or release a product that leverages AI in an impressive way. When other companies start releasing more, and they dont respond for years, it means they were all hype

Re: AlphaCode Attention Visualization

#35

Post by ipsum2 »

jw1224 wrote: post #7

Has Google/Alphabet publicly released any of their AI models yet? I’ve seen plenty of hype surrounding Imagen [1] and Parti [2], but as far as I know, they’re still vaporware. Normally I wouldn’t think too hard about this, but two things come to mind here: Firstly, the speed at which competitors are launching and developing new AI models. Imagen/Parti both seem to rival Stable Diffusion and DALL-E… why can’t we use t…

The arrogance of this comment is astoundingly funny.

> Firstly, the speed at which competitors are launching and developing new AI models. Imagen/Parti both seem to rival Stable Diffusion and DALL-E… why can’t we use them yet?

Because Google doesn't want you to have access to them? Why do you feel like you're entitled to their internal research?

Google releases papers on robots[0] as well. Do you expect them to ship you a free robotic arm? Or give you the ML model for it?

0: https://ai.googleblog.com/2022/12/talking-to-robots-in-real-...

Re: AlphaCode Attention Visualization

#36

Post by astrange »

lossolo wrote: post #31
typon wrote: post #28

Earlier quoted context omitted.

I read a lot on Twitter about the so-called "culture" of Google that prevents them from making AI based products, meanwhile OpenAI has made ChatGPT and is going to replace Google search within the next two months or so. I think this is the same narrative that's being expressed by the GP comment. To add to your comment, Google has been using BERT to power Search since 2019: https://blog.google/products/search/search-l…

> if Google offered Search "Premium" using the latest LLMs, how much would people pay for it? I think they need to solve hallucination problem first, they are already working on optimization eg FLAN-T5 (smaller LLM models same performance) and RETRO (retrieval transformer that can use data index outside of the model) that takes them closer to use it in search.

Apparently you can avoid hallucination by basically reading the model’s mind instead of asking it questions.

https://arxiv.org/abs/2212.03827

Raises some ethical issues…

Re: AlphaCode Attention Visualization

#37

Post by astrange »

jiggawatts wrote: post #10
jw1224 wrote: post #7

Has Google/Alphabet publicly released any of their AI models yet? I’ve seen plenty of hype surrounding Imagen [1] and Parti [2], but as far as I know, they’re still vaporware. Normally I wouldn’t think too hard about this, but two things come to mind here: Firstly, the speed at which competitors are launching and developing new AI models. Imagen/Parti both seem to rival Stable Diffusion and DALL-E… why can’t we use t…

The cultures at the various leading AI research organisations are wildly divergent. Google is full of people that for a want of a better word are simply arrogant. They think that the purpose of AI is for them to show off their skills and... that's it. At best they'd use it internally for selling you more ads, they don't seem to think other people are worthy of using the output of their efforts in any shape, way, or f…

Language models don’t work unless you “censor” them like OpenAI does with reinforcement learning. You get the opposite result - it starts writing erotica as soon as it sees a woman’s name.

Re: AlphaCode Attention Visualization

#38

Post by DoingIsLearning »

iLoveOncall wrote: post #16

The fact that the amount of text you have to write is greater than the amount of code AlphaCode writes should reassure programmers afraid that their job will ever be taken over by AI.

We should really let go of words per minute or typing effort as a useful metric.

Most of our time is spent thinking about what to write not the actual writing.

Also nothing stopping it from being a voice input rather than typing.

Re: AlphaCode Attention Visualization

#39

Post by Grazester »

ldjkfkdsjnv wrote: post #34
anu7df wrote: post #33

Earlier quoted context omitted.

I guess it depends on your point of view. Deepmind as far as I am concerned is the one true success that Alphabet is funding. The work they do is truly cutting edge. They may not be good at making products out of their breakthrough research, but you can't argue they are the "beens" on the basis of that.

At some point in time they need to either release the models or release a product that leverages AI in an impressive way. When other companies start releasing more, and they dont respond for years, it means they were all hype

Their products are used internally at Google and they don't need to release anything to anyone if they feel their use of their ai gives them and advantage in their own business field.

Re: AlphaCode Attention Visualization

#40

Post by Al-Khwarizmi »

ipsum2 wrote: post #35
jw1224 wrote: post #7

Has Google/Alphabet publicly released any of their AI models yet? I’ve seen plenty of hype surrounding Imagen [1] and Parti [2], but as far as I know, they’re still vaporware. Normally I wouldn’t think too hard about this, but two things come to mind here: Firstly, the speed at which competitors are launching and developing new AI models. Imagen/Parti both seem to rival Stable Diffusion and DALL-E… why can’t we use t…

The arrogance of this comment is astoundingly funny. > Firstly, the speed at which competitors are launching and developing new AI models. Imagen/Parti both seem to rival Stable Diffusion and DALL-E… why can’t we use them yet? Because Google doesn't want you to have access to them? Why do you feel like you're entitled to their internal research? Google releases papers on robots[0] as well. Do you expect them to ship…

I'm not the user you replied to, but I have the same view as them. And it's not that we are entitled to anything, it's that they're losing the race, or at least the image race.

For example, as an AI researcher, I can't consider Imagen/Parti to be the state of the art if all we have are cherry-picked examples and we can't verify anything. For all practical intents and purposes, they are just vaporware, and the state of the art are models like Stable Diffusion or DALL-E.

Of course they are free to keep them that way, but they risk losing their reputation as AI/ML/NLP leaders.

Post reply on HN