Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

51–60 of 337 posts

Re: MiniGPT-4

#51

Post by mewpmewp2 »

sho wrote: post #14
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

Only because of big complicated sounding terms, that also exist in web dev.

Re: MiniGPT-4

#52

Post by blurbleblurble »

sho wrote: post #14
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

Arf!

Re: MiniGPT-4

#53

Post by mewpmewp2 »

redox99 wrote: post #46
causi wrote: post #26

Earlier quoted context omitted.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

It's pretty simple actually. Get a 3090 or 4090. Forget about AMD.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games?

Do I need dualboot? Or is Windows good?

Re: MiniGPT-4

#54

Post by thewataccount »

causi wrote: post #26
Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

Nvidia and the highest amount of vram you can get.

Currently the 4090, the rumor is the 4090ti will have 48gb of vram, idk if its worth waiting or not.

The more VRAM the higher paremeter count you can run all in memory (fastest by far).

AMD is almost a joke in ML. The lack of CUDA support (which is nvidia proprietary) is straight lethal, and also even though ROCM does have much better support these days, from what I've seen it's still a fraction of the performance of what it should be. I'm also not sure if you need projects to support it or not, I know pytorch has backend support for it but I'm not sure how easy it is to drop in.

Re: MiniGPT-4

#55

Post by thewataccount »

mewpmewp2 wrote: post #53
redox99 wrote: post #46

Earlier quoted context omitted.

It's pretty simple actually. Get a 3090 or 4090. Forget about AMD.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?

Windows generally works but there may be a somewhat small performance hit. IMO linux is much easier to get to work judging by all the github issue threads I see able SD/LLaMa stuff on windows - but I don't use windows so I dont have personal experience.

4090 24GB is 1800USD, The Ada A6000 48GB is like 8000USD and idk where you buy it? So if you want to run games and models locally the 4090 is honestly the best option.

EDIT: I forgot - there is a rumored 4090ti with 48gb of vram, no idea if thats worth waiting for.

Re: MiniGPT-4

#56

Post by dannyw »

mewpmewp2 wrote: post #53
redox99 wrote: post #46

Earlier quoted context omitted.

It's pretty simple actually. Get a 3090 or 4090. Forget about AMD.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?

You could also get more detailed help from https://reddit.com/r/buildapc/

Re: MiniGPT-4

#57

Post by KaoruAoiShiho »

guax wrote: post #48
mjburgess wrote: post #17

Earlier quoted context omitted.

In practice, it's a lot more like web dev than you might imagine. The above means that the approach is web-dev like gluing, almost literally just, from existingliba import someop from existinglibb import anotherop from someaifw import glue a = someop(X) b = glue(a) Y = anotherop(b)

And just like webdev, each of those were done in a different platform and require arcane incantations and 5h of doc perusing to make it work on your system.

Buddy this ain't 2022 anymore, ask chatgpt (with a plugin that can read docs).

Re: MiniGPT-4

#58

Post by pansa2 »

sho wrote: post #14
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

Web stuff probably makes ML devs feel the same way.

ML is just a different field, using a different set of technologies from those you’re familiar with.

Re: MiniGPT-4

#59

Post by valine »

niemandhier wrote: post #33
causi wrote: post #21

Earlier quoted context omitted.

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

That's because the decided they do not need lidar.

Near term it’s a frustrating decision, but if these gpt4 vision LLMs are anything to go by it will prove to be the right decision in the long term.

Re: MiniGPT-4

#60

Post by rafram »

valine wrote: post #59
niemandhier wrote: post #33

Earlier quoted context omitted.

That's because the decided they do not need lidar.

Near term it’s a frustrating decision, but if these gpt4 vision LLMs are anything to go by it will prove to be the right decision in the long term.

Why wouldn’t LIDAR in addition to computer vision with cameras be a strictly better idea?
Post reply on HN