Google “We have no moat, and neither does OpenAI” • Hacker News Board

Re: Google “We have no moat, and neither does OpenAI”

#561

Post by pavo-etc » Fri, May 05, 2023, 3:27 AM UTC

xkgt wrote: post #528

seydor wrote: post #302

Earlier quoted context omitted.

If i understand correctly it is also shockingly simple, basically just the first figure in the paper: https://miro.medium.com/v2/resize:fit:730/1*D_i25E9dTd_5HMa4... train 2 matrices, add their product to the pretrained weights, and voila! Someone correct me if i m wrong

Correct me if I am wrong, to use LORA fine-tuned model in inference you would still need the original model + trained additional layers, right? If we can perfect methods to fine-tune large models for specific task while reducing the overall model size, then it can fit into more consumer grade hardware for inference and can be broadly used. The objective is to prune unnecessary trivia and memorization artifacts from t…

Yes you still require the original model weights to use LoRA layers. For many LLaMA based models you need to find the original weight yourself and then apply the LoRA diff on top of that.

Re: Google “We have no moat, and neither does OpenAI”

#562

Post by opisthenar84 » Fri, May 05, 2023, 3:29 AM UTC

Taek wrote: post #490

narrator wrote: post #396

Earlier quoted context omitted.

I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…

The biggest llama model has near 100% fidelity (its like 99.3%) at 4 bit quantization, which allows it to fit on any 40GB or 48GB GPU, which you can get for $3500. Or at about a 10x speed reduction you can run it on 128 GB of RAM for only around $250. The story is not anywhere near as bleak as you paint.

A $3500 GPU requirement is far from democratization of AI.

Re: Google “We have no moat, and neither does OpenAI”

#563

Post by goatlover » Fri, May 05, 2023, 3:34 AM UTC

alfor wrote: post #440

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

How can a company keep up with the speed of what is happening in the open? Open AI had years of advanced that almost vanished in a few months. And we will see the rise of specialized models, smaller but targeted, working in team, delegating (Hugging GPT) I would use a small and fast model that only speak english, is expert at coding an science and not much more. Then you fire up an question to another model if yours…

Will the average use know or want to use different models when they can just go to ChatGPT?

Re: Google “We have no moat, and neither does OpenAI”

#564

Post by darig » Fri, May 05, 2023, 3:35 AM UTC

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

[dead]

Re: Google “We have no moat, and neither does OpenAI”

#565

Post by zztop44 » Fri, May 05, 2023, 3:42 AM UTC

titzer wrote: post #419

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

> It's going to be seamlessly integrated into every-day software. I...kinda don't want this? UIs have already changed in so many different fits, starts, waves, and cycles. I used to have skills. But I have no skills now. Nothing works like it used to. Yeah they were tricky to use but I cannot imagine that a murky AI interface is going to be any easier to use, and certainly impossible to master. Even if it is easier t…

I once volunteered with a older woman. She’d been a computer programmer in the 70s, using punch cards and, later, Pascal.

Then she had kids and stopped working for a while and the technology moved on without her. Now she’s like any other old person, doesn’t know how to use a computer and gets flustered when eg: trying to switch from the browser back to Word. Her kids and grandkids clown on her for being hopeless with computers.

I asked her what it was she found difficult about modern computers compared to what she worked with 50 years ago. She said it’s the multitasking. The computers she had worked with just did one thing at once.

Re: Google “We have no moat, and neither does OpenAI”

#566

Post by stu432 » Fri, May 05, 2023, 3:43 AM UTC

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

Yes, bring back Clippy!!!

Re: Google “We have no moat, and neither does OpenAI”

#567

Post by deeplstm » Fri, May 05, 2023, 3:54 AM UTC

I don't think OpenAI and Google have no moat. The biggest moat they have is hardware-software integration, which allows them to serve the models at a very cheap price.

My detailed thoughts in a video format https://youtu.be/cIMlPYI3nz8

Re: Google “We have no moat, and neither does OpenAI”

#568

Post by user_named » Fri, May 05, 2023, 4:11 AM UTC

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

LLMs are just better ML models, are just better statistical models. I agree that they're going to to be in everything, but invisible and in the background.

Re: Google “We have no moat, and neither does OpenAI”

#569

Post by 0xpgm » Fri, May 05, 2023, 4:13 AM UTC

The title is a bit click-bait, since it makes it sound like this is an official position by Google, yet the article clarifies that this the opinion of a Google employee

Re: Google “We have no moat, and neither does OpenAI”

#570

Post by [deleted] » Fri, May 05, 2023, 4:26 AM UTC

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

[deleted]