Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google “We have no moat, and neither does OpenAI”

semianalysis.com

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: Google “We have no moat, and neither does OpenAI”

#391

Post by kashkhan »

tikkun wrote: post #81

The part of the post that resonates for me is that working with the open source community may allow a model to improve faster. And, whichever model improves faster, will win - if it can continue that pace of improvement. The author talks about Koala but notes that ChatGPT is better. GPT-4 is then significantly better than GPT-3.5. If you've used all the models and can afford to spend money, you'd be insane to not use…

aren't androids linux? thats the biggest by far end user platform.

of course google doesnt want to acknowledge it too much.

https://source.android.com/

Re: Google “We have no moat, and neither does OpenAI”

#392

Post by dopeboy »

kyaghmour wrote: post #62

Google's moat is its data set. Imagine training an generative AI LLM on the entire set of YouTube training videos. No one else has this.

This is the glaring omission in this piece.

Googles know _so much_ about me. Is it not reasonable to assume powerful llm + personal data = personal tuned LLM?

Re: Google “We have no moat, and neither does OpenAI”

#393

Post by int_19h »

sharemywin wrote: post #94
FemmeAndroid wrote: post #41

Earlier quoted context omitted.

Charity is only a moat if it’s not profitable.

This is the timeline that's scaring the shit out of them: Feb 24, 2023: Meta launches LLaMA, a relatively small, open-source AI model. March 3, 2023: LLaMA is leaked to the public, spurring rapid innovation. March 12, 2023: Artem Andreenko runs LLaMA on a Raspberry Pi, inspiring minification efforts. March 13, 2023: Stanford's Alpaca adds instruction tuning to LLaMA, enabling low-budget fine-tuning. March 18, 2023: G…

This really ought to mention https://github.com/oobabooga/text-generation-webui, which was the first popular UI for LLaMA, and remains one for anyone who runs it on GPU. It is also where GPTQ 4-bit quantization was first enabled in a LLaMA-based chatbot; llama.cpp picked it up later.

Re: Google “We have no moat, and neither does OpenAI”

#394

Post by sbrother »

eulers_secret wrote: post #331
lysecret wrote: post #144

Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it. One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is…

If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh). So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry…

Will it distribute training across multiple GFX cards? I have a 4x 2080Ti box I would love to be able to use for this sort of thing.

Re: Google “We have no moat, and neither does OpenAI”

#395

Post by reissbaker »

ineedasername wrote: post #384
reissbaker wrote: post #368

Earlier quoted context omitted.

GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using something worse than 3.5... Oof. TBH this feels like cope from Google; Bard is embarrassingly bad and they expected to be able to compete with OpenAI. In my experience, despite their graph in the article that puts them ahead of Vicuna-13B, they're actually behind..…

Yes, I'd readily pay for GPT-4 access, though not the limited 25 requests per 3 hours version. I ponied up $20 for a month of usage to check it out, and it performs head & shoulders above 3.5 in its ability to comprehensively address more complex prompts and provide output that is more nuanced than ChatGPT. I'll also point out that paid api access to 3.5 (davinci-03) is frequently better than ChatGPT's use of 3.5. Yo…

I hear ya! I'm out here dying on the GPT-4 API waitlist too. I use gpt-3.5-turbo's API extensively, and occasionally copy my prompts into GPT-4's web UI and watch as it just flawlessly does all the things 3.5 struggles with. Very frustrating since I don't have GPT-4 API access, but also very, very impressive. It's not even remotely close.

I pay the $20 for ChatGPT Plus (aka, GPT-4 web interface access); personally I find it useful enough to be worth paying for, even in its rate-limited state. It already replaces Google for anything complex for me. I wish I could pay for the API too, and use it in my projects.

Re: Google “We have no moat, and neither does OpenAI”

#396

Post by narrator »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology.

For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The new H100 that is 3x faster goes for about $40,000 and both of these cards can only support a limited number of users, not the tens of thousands of users who can run off a high-end webserver.

I'd imagine Google would lose a lot of money if they put GPT4 level AI into every search, and they are obsessed with cost per search. Multiply that by the billions and it's the kind of thing that will not be cheap enough to be ad supported.

Re: Google “We have no moat, and neither does OpenAI”

#397

Post by kelipso »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

To be fair, the open source model has been what's been working for the last few decades. The concern with LLMs was that open source (and academia) couldn't do what the big companies are doing because they couldn't get access to enough computing resources. The article is arguing (and I guess open source ML groups are showing) you don't need those computing resources to pave the way. It's still an open question whether OpenAI or the other big companies can find a most in AI via either some model, dataset, computing resources, whatever. But then you could ask that question about any field.

Re: Google “We have no moat, and neither does OpenAI”

#398

Post by 4ndrewl »

dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

This is 100% correct - products evolve to become features. Not sure OpenAI faces the existential risk as MS need them to compete with Google in this space.

Re: Google “We have no moat, and neither does OpenAI”

#399

Post by [deleted] »

karmasimida wrote: post #373
reissbaker wrote: post #368

Earlier quoted context omitted.

GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using something worse than 3.5... Oof. TBH this feels like cope from Google; Bard is embarrassingly bad and they expected to be able to compete with OpenAI. In my experience, despite their graph in the article that puts them ahead of Vicuna-13B, they're actually behind..…

GPT-4 is a must if tool using is your goal. GPT-3.5, I think it is mostly suitable for: 1. Quick documentation lookup for non-essential facts 2. Lightweight documents writing and rewriting 3. Translation Other use cases should go straightly to GPT-4

[deleted]

Re: Google “We have no moat, and neither does OpenAI”

#400

Post by airstrike »

narrator wrote: post #396
dahwolf wrote: post #389

The current paradigm is that AI is a destination. A product you go to and interact with. That's not at all how the masses are going to interact with AI in the near future. It's going to be seamlessly integrated into every-day software. In Office/Google docs, at the operating system level (Android), in your graphics editor (Adobe), on major web platforms: search, image search, Youtube, the like. Since Google and other…

I think the problem with AI being everywhere and ubiquitous is that AI is the first technology in a very long time that requires non-trivial compute power. That compute power costs money. This is why you only get a limited number of messages every few hours from GPT4. It simply costs too much to be a ubiquitous technology. For example, the biggest LLama model only runs on an A100 that costs about $15,000 on ebay. The…

Time for a dedicated "AI box" at home with hotswapping compute boards? Maybe put it inside a humanoid or animal-like robot with TTS capabilities?

Sign me up for that kickstarter!

EDIT: based on some quick googling (should I have asked ChatGPT instead?), Nvidia sells the Jetson Xavier Nx dev kit for ~$610 https://www.electromaker.io/shop/product/nvidia-jetson-xavie...

Just need the robot toy dog enclosure

(See https://www.electromaker.io/blog/article/best-sbc-for-ai-sin... for a list of alternatives if that one is too expensive)

Post reply on HN