Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

81–90 of 337 posts

Re: MiniGPT-4

#81

Post by l33tman »

causi wrote: post #26
Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

You don't need a GPU, alpaca.cpp runs purely on the CPU if you have 32+ GB of RAM for the 30B Llama model. It's a bit slow, but you can toy with it..

Re: MiniGPT-4

#82

Post by anonzzzies »

kramerger wrote: post #15
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Can any of this realistically run on CPU at some point? (Not training obviously)

It will do, probably quite soon. Many people are trying.

Re: MiniGPT-4

#83

Post by mk_stjames »

causi wrote: post #26
Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

Ask: how many parameters is the model? At how many bits? and has it been quantized to a smaller number of bits or if not can it be without losing quality? multiply number of bits by the number of parameters, convert to GB, and then add so amount of overhead for however much else needs to load and/or how horrible/amazing of an implementation the inference code is. Done.

Example: A 30B parameter model trained at 16bit FP gets quantized down to 4 bit ints. 4 bits = 0.5 byte. 30 billion * 0.5 byte = 15GB of VRAM (plus a GB or few of other overhead)

For more real world discussion see

https://github.com/ggerganov/llama.cpp/issues/13

Re: MiniGPT-4

#84

Post by alex_duf »

dom96 wrote: post #62

Can this decode CAPTCHAs? Are we finally in a place where CAPTCHAs are useless?

would it be a good things for captchas to be useless? Because in my career I've been a few times on the receiving end of spam attacks and various nastiness and it's nice to be able to integrate such a thing - as unpleasant as it is for the user - and solve the problem at hand.

if this disappears then spammers and the various botnets will have the upper hand again.

Re: MiniGPT-4

#85

Post by egeozcan »

relativeadv wrote: post #50
sho wrote: post #14

Earlier quoted context omitted.

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

> like a dog trying to read Tolstoy this got a chuckle out loud from me. great visual.

This could be a great prompt to test the limits of txt2img models. The astronaut riding a horse got boring already :)

Re: MiniGPT-4

#86

Post by ilteris »

avereveard wrote: post #43
causi wrote: post #26

Earlier quoted context omitted.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

Depends on whether you just want to play around or have actual business needs, but to play around I'm waiting for the next generation of unified memory architectures

Can you point any source re: next generation of unified memory architectures? How far are they away?

Re: MiniGPT-4

#87

Post by astrange »

causi wrote: post #21
omeid2 wrote: post #9

Earlier quoted context omitted.

What did you expect?

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

That's a well known result from the last AI hype era.

https://en.wikipedia.org/wiki/Moravec%27s_paradox

Although, I'm not surprised AI can describe paintings, but I'm still pretty surprised it can generate them. How'd we get that to work!?

Re: MiniGPT-4

#88

Post by sebzim4500 »

HarHarVeryFunny wrote: post #77
Uehreka wrote: post #61

Earlier quoted context omitted.

The word for this is “trademark infringement”. You are specifically not allowed to capitalize on the marketing of another entity’s product to bolster yours by implying through your name that you are somehow related. This is why “DALL-E Mini” had to change their name to craiyon.

It's also just (deliberately) misleading. It's based on the 13B Vicuna/Llama model, not 175B GPT-3 or 1T GPT-4. There is zero justification for calling it MiniGPT-4. A more honest name would be Visual-Vicuna or Son-of-BLIP.

I don't see how it's misleading. MiniGPT-4 makes it sound like a smaller alternative to GPT-4, if it was based on GPT-4 there would be nothing 'mini' about it.

Re: MiniGPT-4

#90

Post by macrolime »

thewataccount wrote: post #55
mewpmewp2 wrote: post #53

Earlier quoted context omitted.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?

Windows generally works but there may be a somewhat small performance hit. IMO linux is much easier to get to work judging by all the github issue threads I see able SD/LLaMa stuff on windows - but I don't use windows so I dont have personal experience. 4090 24GB is 1800USD, The Ada A6000 48GB is like 8000USD and idk where you buy it? So if you want to run games and models locally the 4090 is honestly the best option…

The A6000 is actually the old generation, Ampere. The new Ada generation one is called 6000. Seems many places still sell A6000 (Ampere) for the same price as RTX 6000 (Ada) though, even though the new one is twice as fast.

Seems you can get used RTX A6000s for around $3000 on ebay.

Post reply on HN