Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

131–140 of 337 posts

Re: MiniGPT-4

#131

Post by AbrahamParangi »

cs702 wrote: post #115
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Indeed, really simple. And yes, the results are shockingly good. But what I find most remarkable about this is that the ViT-L+Q-former's hidden states are related by only a linear projection (plus bias) to the Vicuna-13B's token embeddings: emb_in_vicuna_space = emb_in_qformer_space @ W + B These two models are trained independently of each other, on very different data (RGB images vs integer token ids representing s…

I think it’s just that affine transforms in high dimensions are surprisingly expressive. Since the functions are sparsely defined they’re much less constrained compared to the low dimensional affine transformations we usually think of.

Re: MiniGPT-4

#132

Post by pc86 »

sho wrote: post #14
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

Big "a monad is just a monoid in the category of endofunctors" vibes from this one.

Re: MiniGPT-4

#133

Post by MrGilbert »

wwalexander wrote: post #75
MrGilbert wrote: post #19

Earlier quoted context omitted.

Just get rid of all the abbreviations in your mind - they seem to be very intimidating. I really liked the explanation that Stephen Wolfram did on ChatGPT: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... Maybe someone has resources to understand machine-learning on an ELI5 level.

Wow, he waits until halfway through the article to mention A New Kind of Science. Usually he works it into the first couple of paragraphs!

I just happen to read this article of him, which I found easy to understand. I'm neither a huge proponent nor opponent of the likes of his work. Or, bluntly speaking: I don’t know much else about his reputation in the community.

Re: MiniGPT-4

#134

Post by Auracle »

BeefWellington wrote: post #129
lhl wrote: post #101

Earlier quoted context omitted.

For a general guide, I recommend: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni... There's a subreddit r/LocalLLaMA that seems like the most active community focused on self-hosting LLMs. Here's a recent discussion on hardware: https://www.reddit.com/r/LocalLLaMA/comments/12lynw8/is_anyo... If you're looking just for local inference, you're best bet is probably to buy a consumer GPU w/ 24GB of RAM (309…

FWIW I had no real issues getting StableDiffusion to run on a 6800 I have in one of my systems. I haven't tried with LLaMA at all.

Running is one thing. Running with any sort of speed is another.

Re: MiniGPT-4

#135

Post by bugglebeetle »

causi wrote: post #21
omeid2 wrote: post #9

Earlier quoted context omitted.

What did you expect?

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

Don’t worry. That’s valuation has always been ZIRP nonsense and will fall with Musk’s addiction to publicly demonstrating his incompetence.

Re: MiniGPT-4

#136

Post by pc86 »

kerkeslager wrote: post #103
MattPalmer1086 wrote: post #63

Earlier quoted context omitted.

You can just ask GPT how to do it. Much like a lot of web dev!

And the code GPT gives you won't work, much like a lot of web dev? ;P

Maybe it's because of how I use it, but the code ChatGPT gives me has always been super helpful and 99% correct. But, we have a policy at work not to use it for work product so I have to spend time changing enough of it where it's different, and I'm never copy/pasting anything. Enough changes to the structure and variables to make it sufficiently different that it can't be considered pasting company data into GPT, ask my question(s), see what comes back out, refactor/type manually into my IDE, test. I'd say one out of every 8-9 times I get something objectively wrong - a method that doesn't exist, something not compiling, etc. But it's faster than using google/DDG, especially with some prompting so that it just spits back code and not 5th-grade level explanatory paragraphs before and after. And well over half the time it does exactly what I need or sufficiently close that my initial refactoring step gets me the rest of the way.

Re: MiniGPT-4

#137

Post by 22c »

valine wrote: post #64
tarruda wrote: post #18

Looking forward to the next generation of cheap GPUs with enough VRAM to run models like Vicuna-13 locally.

Cheap is relative I suppose. I’m running Vicuna 13b 16f locally and it needs 26GB of VRAM, which won’t even fit on a single RTX 4090. The next gen RTX Titan might have enough vram but that won’t come cheap. I’m expecting a price point above $2500.

I'm not sure if it's the point GP is trying to make, but I would like to see GPUs with extra VRAM that don't have the extra compute. eg. similar performance of a 4070Ti but with 24GB or 32GB of VRAM.

I don't see a really good reason why OEMs couldn't do that now, in the past there have been OEM cards that have more VRAM than the reference design. I'm sure there's an appetite for cards like that for people who don't want to refinance their home loan to get 2 x RTX 4090 cards.

Re: MiniGPT-4

#138

Post by alsodumb »

Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

It's pretty clear that they're doing it for citation farming. I've seen another paper from KAUST that uses fancy title on a trivial paper.

Re: MiniGPT-4

#139

Post by aabajian »

From a radiology world this is fascinating. I'm not worried about job security as I'm an interventionalist. What I'm wondering is about go-to-market strategies for diagnostics. I do some diagnostic reads and I would love to have something like this pre-draft reports (especially for X-Rays). There are tons of "AI in rads" companies right now, none of which have models that come anywhere close to GPT-4 or even this. Personally, I would not start such a company because the barrier to entry is now so low. I think there needs to be an open-source project that just subtrains these models on rads images and is available to all radiologists.

Re: MiniGPT-4

#140

Post by seydor »

152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Then it's an impressive demonstration of how modular neural networks can be. Maybe we don't even need to train monoliths
Post reply on HN