Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

101–110 of 337 posts

Re: MiniGPT-4

#101

Post by lhl »

causi wrote: post #26
Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

For a general guide, I recommend: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...

There's a subreddit r/LocalLLaMA that seems like the most active community focused on self-hosting LLMs. Here's a recent discussion on hardware: https://www.reddit.com/r/LocalLLaMA/comments/12lynw8/is_anyo...

If you're looking just for local inference, you're best bet is probably to buy a consumer GPU w/ 24GB of RAM (3090 is fine, 4090 more performance potential), which can fit a 30B parameter 4-bit quantized model that can probably be fine-tuned to ChatGPT (3.5) level quality. If not, then you can probably add a second card later on.

Alternatively, if you have an Apple Silicon Mac, llama.cpp performs surprisingly well, it's easy to try for free: https://github.com/ggerganov/llama.cpp

Current AMD consumer cards have terrible software support and IMO isn't really an option. On Windows you might be able to use SHARK or DirectML ports, but nothing will run out of the box. ROCm still has no RDNA3 support (supposedly coming w/ 5.5 but no release date announced) and it's unclear how well it'll work - basically, unless you would rather be fighting w/ hardware than playing around w/ ML, it's probably best to avoid (the older RDNA cards also don't have tensor cores, so perf would be hobbled even if you could get things running. Lots of software has been written w/ CUDA-only in mind).

Re: MiniGPT-4

#102

Post by wincy »

thewataccount wrote: post #55
mewpmewp2 wrote: post #53

Earlier quoted context omitted.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?

Windows generally works but there may be a somewhat small performance hit. IMO linux is much easier to get to work judging by all the github issue threads I see able SD/LLaMa stuff on windows - but I don't use windows so I dont have personal experience. 4090 24GB is 1800USD, The Ada A6000 48GB is like 8000USD and idk where you buy it? So if you want to run games and models locally the 4090 is honestly the best option…

If I was going to spend $8000 on a video card I’d hunt on eBay for an A100 80GB rather than settle for the A6000

Re: MiniGPT-4

#103

Post by kerkeslager »

MattPalmer1086 wrote: post #63
guax wrote: post #48

Earlier quoted context omitted.

And just like webdev, each of those were done in a different platform and require arcane incantations and 5h of doc perusing to make it work on your system.

You can just ask GPT how to do it. Much like a lot of web dev!

And the code GPT gives you won't work, much like a lot of web dev? ;P

Re: MiniGPT-4

#104

Post by HarHarVeryFunny »

sebzim4500 wrote: post #88
HarHarVeryFunny wrote: post #77

Earlier quoted context omitted.

It's also just (deliberately) misleading. It's based on the 13B Vicuna/Llama model, not 175B GPT-3 or 1T GPT-4. There is zero justification for calling it MiniGPT-4. A more honest name would be Visual-Vicuna or Son-of-BLIP.

I don't see how it's misleading. MiniGPT-4 makes it sound like a smaller alternative to GPT-4, if it was based on GPT-4 there would be nothing 'mini' about it.

It has more in common with GPT-3 than GPT-4 in terms of size, but in reality it's based on Vicuna/Llama which is 10x smaller than either, so as far as the LLM part of it goes its not mini-anything - it's just straight-up Vicuna 13B.

The model as a whole is just BLIP-2 with a larger linear layer, and using Vicuna as the LLM. If you look at their code it's literally using the entire BLIP-2 encoder (Salesforce code).

https://arxiv.org/pdf/2301.12597.pdf

Re: MiniGPT-4

#105

Post by [deleted] »

sebzim4500 wrote: post #88
HarHarVeryFunny wrote: post #77

Earlier quoted context omitted.

It's also just (deliberately) misleading. It's based on the 13B Vicuna/Llama model, not 175B GPT-3 or 1T GPT-4. There is zero justification for calling it MiniGPT-4. A more honest name would be Visual-Vicuna or Son-of-BLIP.

I don't see how it's misleading. MiniGPT-4 makes it sound like a smaller alternative to GPT-4, if it was based on GPT-4 there would be nothing 'mini' about it.

[deleted]

Re: MiniGPT-4

#106

Post by causi »

niemandhier wrote: post #33
causi wrote: post #21

Earlier quoted context omitted.

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

That's because the decided they do not need lidar.

Sure, sure, but would it have killed them to drop in a few five dollar "don't hit this object" ultrasonic proximity sensors?

Re: MiniGPT-4

#107

Post by tantony »

kramerger wrote: post #15
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Can any of this realistically run on CPU at some point? (Not training obviously)

Someone is probably going to port it to llama.cpp soon.

Re: MiniGPT-4

#108

Post by tudorw »

amelius wrote: post #76
mjburgess wrote: post #17

Earlier quoted context omitted.

In practice, it's a lot more like web dev than you might imagine. The above means that the approach is web-dev like gluing, almost literally just, from existingliba import someop from existinglibb import anotherop from someaifw import glue a = someop(X) b = glue(a) Y = anotherop(b)

It's more like gardening: 1. plant seed 2. ...wait a very long time... 3. observe completely unexpected but cool result The unexpected part of step 3 is what makes this very different from any kind of engineering, even webdev. Of course, there is a lot of engineering involved in good ML, but that is more comparable to agricultural engineering in the sense that it's just a lot of dumb plumbing that any engineer can do…

count me in :)

Re: MiniGPT-4

#109

Post by ikurei »

sho wrote: post #14
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

> This ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

Just like any discussion between advanced web devs would make any humble woodworker feel?

And just like any discussion between advanced woodworkers would make a humble web dev feel?

"It's really simple, they're just using a No. 7 jointer plane with a high-angle frog and a PM-V11 blade to flatten those curly birch boards, then a No. 4 smoother plane with a Norris-type adjuster and a toothed blade for the final pass."

Whut?

"You could use Webpack to bundle your HTML, CSS and Babel-transpiled TypeScript 5 down to shim-included Ecmascript 4", "They're just using OAuth2 authentication with Passport.js and JWT tokens, which easily gets you CSRF protection", "Our e-learning platform uses LMS.js and xAPI.js, plus SCORM for course packaging and Moodle as the LMS backend.", ...

There was a time you didn't know what any of that meant.

Just because you don't know what the words mean shouldn't make it sound difficult. Not saying AI is easy, just that the jargon is not a good indication of difficulty and we should know better than to be so easily mystified.

Re: MiniGPT-4

#110

Post by alchemist1e9 »

wwalexander wrote: post #75
MrGilbert wrote: post #19

Earlier quoted context omitted.

Just get rid of all the abbreviations in your mind - they seem to be very intimidating. I really liked the explanation that Stephen Wolfram did on ChatGPT: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... Maybe someone has resources to understand machine-learning on an ELI5 level.

Wow, he waits until halfway through the article to mention A New Kind of Science. Usually he works it into the first couple of paragraphs!

I known it’s hard to believe but I sense LLMs have slightly knocked his ego down and injected a small dose of humility.

https://youtu.be/z5WZhCBRDpU

I pick that up in above video and also in the post above.

Definitely healthy for him which just to be clear I’m a huge Wolfram fan and the ego doesn’t really bother me, it’s just part of who he is, however I do find it nice that LLMs are having him self reflect more than typical.

Post reply on HN